CN108476258A - Method and electronic equipment for electronic equipment control object - Google Patents
Method and electronic equipment for electronic equipment control object Download PDFInfo
- Publication number
- CN108476258A CN108476258A CN201680075716.5A CN201680075716A CN108476258A CN 108476258 A CN108476258 A CN 108476258A CN 201680075716 A CN201680075716 A CN 201680075716A CN 108476258 A CN108476258 A CN 108476258A
- Authority
- CN
- China
- Prior art keywords
- electronic equipment
- attribute
- user
- information
- exemplary embodiment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 203
- 238000004891 communication Methods 0.000 claims description 100
- 230000006854 communication Effects 0.000 claims description 100
- 238000003384 imaging method Methods 0.000 claims description 43
- 238000001514 detection method Methods 0.000 claims description 6
- 230000008569 process Effects 0.000 description 129
- 241001465382 Physalis alkekengi Species 0.000 description 90
- 230000006870 function Effects 0.000 description 40
- 238000012549 training Methods 0.000 description 34
- 238000003860 storage Methods 0.000 description 20
- 230000008859 change Effects 0.000 description 18
- 230000009471 action Effects 0.000 description 16
- 238000005286 illumination Methods 0.000 description 16
- 230000005611 electricity Effects 0.000 description 15
- 230000001815 facial effect Effects 0.000 description 14
- 239000010408 film Substances 0.000 description 13
- 238000005516 engineering process Methods 0.000 description 12
- 238000012545 processing Methods 0.000 description 12
- 238000005406 washing Methods 0.000 description 11
- 238000010586 diagram Methods 0.000 description 10
- 210000001508 eye Anatomy 0.000 description 10
- 241000406668 Loxodonta cyclotis Species 0.000 description 9
- 230000006399 behavior Effects 0.000 description 9
- 230000004048 modification Effects 0.000 description 9
- 238000012986 modification Methods 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 8
- 238000005452 bending Methods 0.000 description 8
- 238000010295 mobile communication Methods 0.000 description 8
- 230000004044 response Effects 0.000 description 8
- 238000004378 air conditioning Methods 0.000 description 7
- 230000033001 locomotion Effects 0.000 description 7
- 235000013305 food Nutrition 0.000 description 5
- 230000003068 static effect Effects 0.000 description 5
- 230000004913 activation Effects 0.000 description 4
- 230000033228 biological regulation Effects 0.000 description 4
- 210000005252 bulbus oculi Anatomy 0.000 description 4
- 238000013519 translation Methods 0.000 description 4
- 241001465754 Metazoa Species 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 3
- 238000007791 dehumidification Methods 0.000 description 3
- 235000013601 eggs Nutrition 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000008570 general process Effects 0.000 description 3
- 238000007689 inspection Methods 0.000 description 3
- 235000013336 milk Nutrition 0.000 description 3
- 239000008267 milk Substances 0.000 description 3
- 210000004080 milk Anatomy 0.000 description 3
- 230000005236 sound signal Effects 0.000 description 3
- 230000000007 visual effect Effects 0.000 description 3
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 3
- 241000208340 Araliaceae Species 0.000 description 2
- 240000004160 Capsicum annuum Species 0.000 description 2
- 235000005035 Panax pseudoginseng ssp. pseudoginseng Nutrition 0.000 description 2
- 235000003140 Panax quinquefolius Nutrition 0.000 description 2
- 238000004140 cleaning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 235000008434 ginseng Nutrition 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000013507 mapping Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 239000000843 powder Substances 0.000 description 2
- 235000013580 sausages Nutrition 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 241000282326 Felis catus Species 0.000 description 1
- RSPISYXLHRIGJD-UHFFFAOYSA-N OOOO Chemical compound OOOO RSPISYXLHRIGJD-UHFFFAOYSA-N 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003466 anti-cipated effect Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000007175 bidirectional communication Effects 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 235000013353 coffee beverage Nutrition 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 239000000428 dust Substances 0.000 description 1
- 238000005265 energy consumption Methods 0.000 description 1
- 235000015114 espresso Nutrition 0.000 description 1
- 230000007717 exclusion Effects 0.000 description 1
- 239000004744 fabric Substances 0.000 description 1
- 239000011888 foil Substances 0.000 description 1
- 230000036541 health Effects 0.000 description 1
- 238000010191 image analysis Methods 0.000 description 1
- 230000001939 inductive effect Effects 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 230000014759 maintenance of location Effects 0.000 description 1
- 230000032696 parturition Effects 0.000 description 1
- 238000000926 separation method Methods 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000010897 surface acoustic wave method Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 239000010409 thin film Substances 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
- 230000003442 weekly effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
- H04M1/72415—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/0482—Interaction with lists of selectable items, e.g. menus
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/752—Contour matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/20—Scenes; Scene-specific elements in augmented reality scenes
-
- G—PHYSICS
- G08—SIGNALLING
- G08C—TRANSMISSION SYSTEMS FOR MEASURED VALUES, CONTROL OR SIMILAR SIGNALS
- G08C17/00—Arrangements for transmitting signals characterised by the use of a wireless electrical link
- G08C17/02—Arrangements for transmitting signals characterised by the use of a wireless electrical link using a radio link
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L41/00—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks
- H04L41/22—Arrangements for maintenance, administration or management of data switching networks, e.g. of packet switching networks comprising specially adapted graphical user interfaces [GUI]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4126—The peripheral being portable, e.g. PDAs or mobile phones
- H04N21/41265—The peripheral being portable, e.g. PDAs or mobile phones having a remote control device for bidirectional communication between the remote control device and client device
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4131—Peripherals receiving signals from specially adapted client devices home appliance, e.g. lighting, air conditioning system, metering devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/43615—Interfacing a Home Network, e.g. for connecting the client to a plurality of peripherals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/436—Interfacing a local distribution network, e.g. communicating with another STB or one or more peripheral devices inside the home
- H04N21/4363—Adapting the video stream to a specific local network, e.g. a Bluetooth® network
- H04N21/43637—Adapting the video stream to a specific local network, e.g. a Bluetooth® network involving a wireless protocol, e.g. Bluetooth, RF or wireless LAN [IEEE 802.11]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2201/00—Electronic components, circuits, software, systems or apparatus used in telephone systems
- H04M2201/42—Graphical user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M2250/00—Details of telephonic subscriber devices
- H04M2250/22—Details of telephonic subscriber devices including a touch pad, a touch sensor or a touch detector
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computer Networks & Wireless Communication (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Automation & Control Theory (AREA)
- Health & Medical Sciences (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
A kind of method for electronic equipment control object includes：Recognize the first object and the second object, identify the first attribute of the first object and the second attribute of the second object, the object of control is selected based on the first attribute and the second attribute, generates the operation signal of the object for selection, and the operation signal generated is sent to the object of selection.
Description
Technical field
The device and method consistent with exemplary embodiment are related to controlling selected by user at least one for electronic equipment
The method and electronic equipment of a object.
Background technology
Mobile terminal can be configured to perform various functions.The example of various functions include data and function of voice communication,
Pass through the function of camera capture photo or video, sound storing function, the function by speaker system playing music, figure
Picture or video display function etc..
Some mobile terminals have the additional function for executing game, and some other mobile terminals can be implemented as
Multimedia player.Moreover, mobile terminal provides the distant control function for remote control miscellaneous equipment.However, since equipment has
There is different control interfaces, therefore it is inconvenient that user controls miscellaneous equipment by mobile terminal.
Invention content
Technical problem
Exemplary embodiment can at least solve problem above and/or disadvantage and other disadvantages not described above.Moreover,
Exemplary embodiment may not solve the problems, such as above-described any without overcoming disadvantages described above.
One or more exemplary embodiments are provided for being based on identifying by camera or short-range wireless communications interfaces
Multiple objects in the attribute of each, automatically control the method for at least one object according to the user's intention and electronics is set
It is standby.
Advantageous effects
According to one exemplary embodiment, attribute of the electronic equipment based on the multiple objects identified by electronic equipment, according to
The intention at family determines target object and the object run of control, so that user can easily control external equipment.
Description of the drawings
Certain exemplary embodiments are described by referring to accompanying drawing, above and/or other aspect will become clearer, attached
In figure：
Fig. 1 is the figure for illustrating object control system according to one exemplary embodiment；
Fig. 2 is the flow chart for illustrating the method for electronic equipment control object according to one exemplary embodiment；
Fig. 3 is relating attribute table between object according to one exemplary embodiment；
Fig. 4 is the flow chart for illustrating the method that object is distinguished by imaging sensor according to one exemplary embodiment；
Fig. 5 A and 5B are that the electronic equipment illustrated according to one exemplary embodiment is received for the input by user of selecting object
The figure of operation；
Fig. 6 is the flow chart for illustrating the method that object is distinguished by short haul connection according to one exemplary embodiment；
Fig. 7 is the figure for illustrating object map according to one exemplary embodiment；
Fig. 8 be illustrate when all multiple objects are controllable devices according to one exemplary embodiment be used for electronic equipment control
The sequence chart of the method for object processed；
Fig. 9 be illustrate when electronic equipment be connected to multiple objects for communication when according to one exemplary embodiment for electricity
Sub- equipment controls the sequence chart of the method for an object based on the attribute of multiple objects；
Figure 10 A, Figure 10 B and Figure 10 C are the attributes for illustrating electronic equipment according to one exemplary embodiment and being based on TV (TV)
The figure of the operation of lamps and lanterns is controlled with the attributes of lamps and lanterns；
Figure 11 is the basis illustrated when the only one in the multiple objects identified by electronic equipment is controllable device
The sequence chart of the method for electronic equipment control object of exemplary embodiment；
Figure 12 is to illustrate the attribute of attribute and TV of the electronic equipment based on people according to one exemplary embodiment to control TV's
The figure of operation；
Figure 13 be illustrate the attribute of attribute and audio system of the electronic equipment based on surrounding according to one exemplary embodiment come
Control the figure of the operation of audio system；
Figure 14 is to illustrate electronic equipment according to one exemplary embodiment based on the attribute of transaction attribute and dehumidifier and to control
The figure of the operation of dehumidifier processed；
Figure 15 be illustrate when object is characterized as uncontrollable object by electronic equipment according to one exemplary embodiment for electricity
The sequence chart of the method for sub- equipment control extra objects；
Figure 16 is to illustrate the attribute of attribute and doll of the electronic equipment based on people according to one exemplary embodiment to control TV
Operation figure；
Figure 17 be illustrate attribute of the electronic equipment based on the people identified by electronic equipment according to one exemplary embodiment and
The attribute of doll carrys out the figure of the operation of broadcasting content；
Figure 18 is attribute of the electronic equipment based on the refrigerator identified by electronic equipment illustrated according to one exemplary embodiment
The figure of the operation of notice is shown with the attribute of people；
Figure 19 is attribute of the electronic equipment based on the refrigerator identified by electronic equipment illustrated according to one exemplary embodiment
The figure of the operation of the display of external equipment is controlled with the attribute of people；
Figure 20 is the figure for the operation for illustrating electronic equipment remote control external equipment according to one exemplary embodiment；
Figure 21 is the method for illustrating the attribute that electronic equipment according to one exemplary embodiment shows multiple objects identified
Flow chart；
Figure 22 is the operation for illustrating electronic equipment according to one exemplary embodiment and showing the attribute of TV and the attribute of lamps and lanterns
Figure；
Figure 23 is the flow chart for illustrating the method that electronic equipment according to one exemplary embodiment recommends multiple operations；
Figure 24 be illustrate attribute and lamps and lanterns of the electronic equipment based on TV according to one exemplary embodiment attribute it is more to recommend
The figure of the operation of a operation；
Figure 25 is the figure for illustrating application execution system according to one exemplary embodiment；
Figure 26 is the sequence chart for illustrating the method that server according to one exemplary embodiment generates object recognition model information；
Figure 27 is the figure for illustrating classification and keyword according to one exemplary embodiment；
Figure 28 is the figure for the operation for illustrating the image that generation according to one exemplary embodiment is used for object recognition；
Figure 29 is the modification object recognition model information illustrated when there are object recognition mistake according to one exemplary embodiment
Method sequence chart；
Figure 30 is the figure for illustrating the situation that generating object in the electronic device distinguishes mistake；
Figure 31 is the sequence for the method for illustrating the personalized object recognition model information of generation according to one exemplary embodiment
Figure；
Figure 32 is the figure for the operation for illustrating linked object and application according to one exemplary embodiment；
Figure 33 A are the figures of the operation of the electronic equipment for the video for illustrating acquisition object according to one exemplary embodiment；
Figure 33 B are the figures for illustrating the operation that electronic equipment distinguishes model information from server downloaded object；
Figure 34 is the figure for the operation for illustrating linked object and controller according to one exemplary embodiment；
Figure 35 is the flow chart for illustrating the method that upgating object according to one exemplary embodiment distinguishes model information；
Figure 36 is that the electronic equipment illustrated according to one exemplary embodiment changes object recognition model information in the case where user asks
Operation figure；
Figure 37 is the flow chart of the method for the electronic equipment or controller that illustrate execution application according to one exemplary embodiment；
Figure 38 is the figure for illustrating the operation that electronic equipment according to one exemplary embodiment shows the controller for corresponding to TV；
Figure 39 be illustrate according to one exemplary embodiment by link memorandum or station address and object generate to as
Distinguish the sequence chart of the method for model information；
Figure 40 is the figure for the operation for illustrating link credit card and memorandum according to one exemplary embodiment；
Figure 41 is the figure for the operation for illustrating link window and station address according to one exemplary embodiment；
Figure 42 is the operation for illustrating the Weather information that electronic equipment according to one exemplary embodiment is shown corresponding to window
Figure；
Figure 43 is the block diagram for the configuration for illustrating electronic equipment according to one exemplary embodiment；And
Figure 44 is the block diagram for the configuration for illustrating server according to one exemplary embodiment；
Specific implementation mode
According to one exemplary embodiment on the one hand, a kind of method of control object includes：Distinguish the first object and the second object,
Identify the first attribute of the first object and the second attribute of the second object, the first attribute and the second object based on the first object
Second attribute come selected between the first object and the second object control object, generate for selected object operation believe
Number, and the operation signal generated is sent to selected object.
According to one exemplary embodiment on the one hand, a kind of electronic equipment includes processor and communication interface, the processor by with
It is set to：It distinguishes the first object and the second object, identifies the second attribute of the first attribute and the second object of the first object, based on the
First attribute of an object and the second attribute of the second object select the object of control between the first object and the second object,
And the operation signal for selected object is generated, which is configured to send generation to selected object
Operation signal.
The pattern of the present invention
Certain exemplary embodiments are more fully described with reference to the accompanying drawings.
In being described below, even if in different drawings, identical reference numeral is also used for identical element.Offer is retouched
Item defined in stating, such as detailed structure and element, to help comprehensive understanding exemplary embodiment.It will be clear, however, that：
Can in the case of the item especially defined without those practical exemplary embodiment.Moreover, because well known function or knot
Structure will use unnecessary details vague description, therefore be not described in detail well known function or structure.
Select to be currently being widely used generic term as far as possible in view of the function in exemplary embodiment as demonstration
The term used in property embodiment, but can be according to the appearance etc. of the intention of those of ordinary skill in the art, precedent, new technology
And change.Particularly, some terms can arbitrarily be selected by applicant, and in this case, it will be old in corresponding description
State the detailed meanings of term.Therefore, should the meaning based on term together with the description in the whole instruction rather than its simple same
Claim to define the term used in exemplary embodiment.
Throughout the specification, when part " comprising " element, unless otherwise described, it otherwise can further comprise another
The presence of element rather than the other elements of exclusion.Moreover, the term instruction of such as " part ", " module " or the like used herein
Unit for handling at least one functions or operations, the wherein unit can be embodied as hardware or software, or can be by hard
Part and software embody.
As it is used herein, term "and/or" includes any and institute of one or more of associated institute's list
There is combination.When before element list, the expression of such as "...... at least one" etc modify entire element list and
It is not the discrete component for modifying list.
Fig. 1 is the figure for illustrating object control system according to one exemplary embodiment.
With reference to figure 1, object control system according to one exemplary embodiment may include electronic equipment 100 and multiple objects.So
And the component of all diagrams is not necessary component.Object control system can be by more more or less than the component count of diagram
Component realize.
Electronic equipment 100 can be the equipment that can control external object.For example, electronic equipment 100 can be incited somebody to action by network
Control information sends external object to.According to one exemplary embodiment, the network available wireless communication technology or mobile communication technology come
It realizes, such as Wireless Fidelity (Wi-Fi), home radio (RF), bluetooth, high speed wireless personal Local Area Network (HR-WPAN), ultra-wide
Band (UWB), low rate wireless personal domain network (LR-WPAN), Institute of Electrical and Electric Engineers (IEEE) 1394 etc., but not
It is limited to this.
Electronic equipment 100 according to one exemplary embodiment can be realized in a variety of manners.For example, electronic equipment 100 can be with
It is digital camera, smart mobile phone, laptop computer, tablet PC, e-book terminal, the terminal for digital broadcasting, a number
Word assistant (PDA), portable media player (PMP), navigation equipment, MP3 player etc., but not limited to this.It is described herein
Electronic equipment 100 can be the wearable equipment of user.Wearable device may include accessory device (such as wrist-watch, ring, hand
Ring, foot ring, necklace, glasses or contact lenses), headset equipment (HMD), with fabric or clothes (such as electronics clothes) integrally
The equipment of formation, body attach equipment (such as pad skin) and implantable devices (such as implantable circuit), but not limited to this.
For ease of description, the case where electronic equipment 100 is mobile terminal will be described as example.
In an exemplary embodiment, object can be the things that can be distinguished by electronic equipment 100, people, equipment (such as Internet of Things
Net (IoT) equipment), animal, environment etc., but not limited to this.For example, object can be display equipment (such as TV), smart mobile phone,
Laptop computer, tablet PC, e-book terminal, digital broadcast terminal, PDA, PMP, navigation equipment, MP3 player, consumption electricity
Son (CE) equipment (such as lamps and lanterns, refrigerator, air-conditioning, water purifier, dehumidifier, humidifier, espresso coffee machine, oven, robotic vacuum
Dust catcher etc.), wearable device (such as belt, wrist-watch, glasses, virtual reality (VR) earphone, shoes, belt, gloves, ring, item
Chain etc.), people (such as kinsfolk, friend etc.), animal (such as cat, dog etc.), things (such as doll, clothes, bed, window
Deng), environment (such as weather, season, temperature, humidity etc.), but not limited to this.
According to one exemplary embodiment, electronic equipment 100 can control external object according to user's input.For example, in order to
Film is watched on TV 200, user may wish to reduce the illumination of lamps and lanterns 300.At this point, with reference to the reference numeral 10 of figure 1, electronics
Equipment 100 can search for surrounding devices by short haul connection (such as bluetooth), and show the list of the equipment detected.When
For user when selecting the identification information for the lamps and lanterns 300 to be controlled among the equipment detected, electronic equipment 100 can perform and lamps and lanterns
300 pairing process, and then show the graphic user interface (GUI) of the illumination for adjusting lamps and lanterns 300.When user passes through
When the illumination of lamps and lanterns 300 is arranged to 20% by GUI, electronic equipment 100 can be sent to lamps and lanterns 300 illumination is arranged to 20% finger
It enables.However, the identification information for searching lamps and lanterns 300 is found in the list of devices of detection for a user for pairing process
It is very difficult.Moreover, it is inconvenient to execute several processes for a user with the illumination for adjusting lamps and lanterns 300.
Therefore, according to one exemplary embodiment, electronic equipment 100 can provide interface 20, and interface 20 allows users at a glance
It distinguishes object and is easy to manipulate the function of object as shown in Figure 1.For example, when user is in the camera of electronic equipment 100
When TV 200 is connected to the line of lamps and lanterns 300 by drafting in live view (live view), electronic equipment 100 can control lamps and lanterns 300
Illumination is automatically adjusted according to the type of the content played on TV 200.
The method for automatically controlling object according to the user's intention hereinafter with reference to Fig. 2 detailed description electronic equipments 100.
Fig. 2 is the flow chart for the method for illustrating electronic equipment control object according to one exemplary embodiment.
In operating S210, electronic equipment 100 can distinguish the first object and the second object.
In an exemplary embodiment, " distinguishing object " can indicate that object is identified and screen is in possible display and corresponds to
The state of the image (such as real image or alternate image) of object.
According to one exemplary embodiment, imaging sensor or camera discrimination can be used to be present in electronic equipment for electronic equipment 100
The first object outside 100 and the second object.For example, electronic equipment 100 can obtain the original including the first object and the second object
Beginning frame, and the first object and the second object are identified by the image procossing to primitive frame.For example, electronic equipment 100 can lead to
The analysis to primitive frame is crossed to detect the profile of the profile and the second object of the first object.Then, electronic equipment 100 can will detect
To the first object and the second object profile compared with predefined template, with type, the title etc. for detecting object.For example,
When the profile of the first object is similar to the template of TV, electronic equipment 100 can be included within the first object recognition in primitive frame
For TV, and can be refrigerator by the second object recognition when the profile of the second object is similar to the template of refrigerator.
According to one exemplary embodiment, electronic equipment 100 can also execute face recognition to the object being included in primitive frame.Example
Such as, electronic equipment 100 can detect the facial area of the people in primitive frame.The method for detecting facial area includes Knowledge based engineering side
Method, the method for feature based, template matching method, the method etc. based on appearance, but not limited to this.
Electronic equipment 100 can extract feature (such as the shape of eyes, nose and mouth of face from the facial area of detection
Deng).Gabor filter, local binary (LBP) etc. can be used to extract facial feature from facial area for electronic equipment 100,
But the method for the feature from facial area extraction face is without being limited thereto.
Electronic equipment 100 can be by the facial characteristics ratio of the feature and pre-registered user of the face extracted from facial area
Compared with.For example, when the facial characteristics of extraction is similar to the face of the first user (such as Tom) registered in advance, electronic equipment 100
Can be the first user (such as Tom) by the process identification.Moreover, the face of the facial characteristics and the children registered in advance when extraction
When similar, which can be children by electronic equipment 100.
Moreover, electronic equipment 100 can be by some region of primitive frame compared with colour map (color histogram), and carry
Take the visual signature of such as color alignment of image, pattern, atmosphere or the like as image analysis information.In this case,
Electronic equipment 100 can recognize the rain or snow fallen outside window.
In addition, electronic equipment 100 can execute optical character identification (OCR) to the character being included in object.OCR indicates to use
Being converted into the alphabet of various human languages, the digital font being included in image document etc. can in electronic equipment 100
The technology of the character code of editor.Therefore, electronic equipment 100 can identify object by character shown in identification objects.
According to one exemplary embodiment, electronic equipment 100 can also pass through short haul connection identification objects.For example, electronic equipment
100 advertisement packets that can be broadcasted by object (or being attached to bluetooth low energy consumption (BLE) label of object) by BLE communications receptions,
And the identification information being included in advertisement packet is extracted with identification objects.Electronic equipment 100 can also be based on the intensity of BLE signals
To calculate with a distance from object.
When the first object is tablet PC, electronic equipment 100 can be communicated to receive from the first object (tablet PC) by BLE and be wrapped
Include the advertisement packet of the identification information (such as code corresponding to tablet PC) of the first object.Electronic equipment 100 can analyze advertisement
Grouping, and it is tablet PC to pick out the first object.Moreover, when the second object is doll, electronic equipment 100 can be analyzed from attached
It is connected to the advertisement packet that the BLE labels of doll receive, and it is doll to pick out the second object.
According to one exemplary embodiment, extremely by being communicated by light fidelity (Li-Fi), in Wi-Fi communications and Bluetooth communication
The identification information of the identification information and the second object of the first object of a few reception, electronic equipment 100 can pick out the first object
With the second object, but the method for the identification information that receives the first and second objects is without being limited thereto.
Li-Fi can be for using the wavelength of the light emitted from light emitting diode (LED) come transmit one kind of information can
Light-exposed communication (VLC) technology.Data transmission is executed by the flicker of light.Flicker is carried out with the interval of microsecond,
And because without being noticed by human eye.In other words, light bulb, which seems, opens, but actually just in transmission data.Therefore,
Li-Fi can be used for having illumination and it is harmless from anywhere in.
Moreover, according to one exemplary embodiment, when electronic equipment 100 is connected to home network, electronic equipment 100 can recognize
It is connected to other objects of home network.For example, by the way that from home gateway, (it can be expressed as transponder, server or IoT line concentrations
Device) identification information, location information, status information, the functional information etc. of other objects are received, electronic equipment 100 can recognize other
Object.Electronic equipment 100 can show the icon image corresponding to the object picked out on the screen.
According to one exemplary embodiment, the first object and the second object picked out by electronic equipment 100 can be controllable
Equipment.For example, " controllable device " can indicate can to have the communication link established with electronic equipment 100 and according to passing through communication chain
The control signal that road is transmitted from electronic equipment 100 is come the equipment that operates.
According to one exemplary embodiment, the only one in the first object and the second object that are picked out by electronic equipment 100
Can be controllable device, and another can be uncontrollable object.For example, uncontrollable object can not be anticipated in technology
The things of one-way or bi-directional communication, people, animal, environment (weather, season, temperature etc.) in justice, but not limited to this.
Meanwhile the first object picked out by electronic equipment 100 and the second object can be uncontrollable object (such as rings
Border, things etc.).Such case is described in detail hereinafter with reference to Figure 15.
According to one exemplary embodiment, when three or more objects are picked out by camera or communication interface, electronics
Equipment 100 can show the object picked out on the screen, and be received from user for selecting the among the object picked out
The input of an object and the second object.
Electronic equipment 100 can show object in a variety of manners.It shows to distinguish by camera for example, electronic equipment 100 can be used
The live view for the main body known shows the real image of object.Pass through hereinafter with reference to Fig. 4, Fig. 5 A and Fig. 5 B detailed descriptions
Live view shows the operation of the electronic equipment 100 of object.
According to one exemplary embodiment, it includes virtual image (such as icon figure corresponding to object that electronic equipment 100, which produces,
Picture, text image etc.) object map, and show object map on the screen.It is described in detail hereinafter with reference to Fig. 6 and Fig. 7
Object map.
According to one exemplary embodiment, electronic equipment 100 can show the object picked out with VR patterns.For example, electronic equipment
100 can use some in the object that real image or Real-time image display pick out, and virtual image (such as VR images or suitable are used in combination
When recognize icon) show some objects.
In operation S220, electronic equipment 100 can recognize that the attribute of the attribute and the second object of the first object.
Here, the attribute of object may include the identifier (ID) of the object, type of object, the function of being provided by object, right
The current state etc. of elephant, but not limited to this.The attribute of object can change according to the type of object.
When object is equipment, the attribute of object may include the function of being provided by object, the current state of object, object
Position, the communication means etc. supported by object.For example, when object is TV, the attribute of object may include the work(provided by TV
Can (such as brightness regulation, volume adjustment, power control, application execution etc.), the content shown on TV type (such as electricity
Shadow, news, game, music etc.), the style of content (such as action, romantic, illusion etc.), current channel, current volume level,
Current brightness level, TV are in whether on or off state, TV are in operating mistake state etc., but not limited to this.
When object is people, the attribute of object may include the ID of people, the age of people, the gender of people, the channel of people's preference, people
The content (such as film, music etc.) of preference, the facility information of people, people biometric data etc., but not limited to this.Work as object
When being things, the attribute of object may include class of things type, the title of things, function of things etc., but not limited to this.Work as object
When being environment, the attribute of object may include weather, temperature, humidity, season etc., but not limited to this.
According to one exemplary embodiment, when the first object and the second object are the equipment that can be communicated, electronic equipment 100 can
The attribute information of the first object is received from the first object, and the attribute information of the second object is received from the second object.Electronics is set
Standby 100 can directly receive the attribute information of object by short haul connection (such as bluetooth, Wi-Fi direct (WFD) etc.) from object
Or the attribute information of object is received by gateway.
According to one exemplary embodiment, relating attribute table comes between electronic equipment 100 can refer to object stored in memory
Identify the attribute of the attribute and the second object of the first object.For example, when the identification information of the first object is acquired, electronic equipment
100 can search for the first attribute of the identification information corresponding to the first object between object in relating attribute table.
With reference to Fig. 3, relating attribute table 310 between description object.Between object relating attribute table 310 can be wherein for example based on
The usage history of user, the preference of user or any other method appropriate define the combination of the attribute corresponding to multiple objects
Control the table of information.For example, control information may include about equipment (hereinafter referred to as control device to be controlled among multiple objects
Or target object) and by the information of the operation executed by control device (hereinafter referred to as control operate).For example, working as electronic equipment
Between 100 references object when relating attribute table 310, the attribute of TV is " content (film) " and " brightness ", and the attribute of lamps and lanterns is " bright
Degree " corresponds to the first control letter of the combination of " brightness " of " content " of one of attribute as TV and the attribute as lamps and lanterns
Breath is：" control device：Lamps and lanterns, operation：Brightness is arranged to 20% ", and corresponding to the " bright of another attribute as TV
Second control information of the combination of " brightness " of degree " and attribute as lamps and lanterns is：" control device：TV, operation：Adjust TV's
Brightness is with the brightness corresponding to illumination ".
Meanwhile when each object has multiple attributes, relating attribute table 310 selects between electronic equipment 100 can refer to object
An attribute is selected for object control.For example, when the first object and the second object are selected, electronic equipment 100 can refer to
Relating attribute table 310 selects the first attribute among multiple attributes of the first object between object, and in the more of the second object
The second attribute is selected among a attribute.
According to one exemplary embodiment, electronic equipment 100 can select to have among multiple attributes corresponding to object higher
The attribute of priority.In other words, multiple attributes of object can have priority order.For example, when object is TV and TV
Whens attribute includes playing back content, volume, brightness etc., electronic equipment 100 can select have highest priority among the attribute of TV
" playing back content ".That is, usage history that for example can be based on user, the preference of user or any other method appropriate, to return
Put content, volume, brightness priority order arrangement TV attribute.
According to one exemplary embodiment, the priority order of multiple attributes of the first object can according to the type of the second object and
Variation.For example, when the first object is TV and the second object is people, electronic equipment 100 can be among the attribute of the first object
It selects " playing back content ".On the other hand, when the first object is TV and the second object is lamps and lanterns, electronic equipment 100 can be
" brightness " is selected among the attribute of an object.
According to one exemplary embodiment, electronic equipment 100 can select multiple attributes corresponding to object based on user's input
Among one.For example, electronic equipment 100 can will be shown corresponding to the attribute of each in multiple objects together with multiple objects
On the screen, and one user input in the attribute for being used for selecting display is received.It is detailed hereinafter with reference to Figure 21 and Figure 22
Operation of the thin description display corresponding to the electronic equipment 100 of the attribute of at least one object.
In operating S230, electronic equipment 100 can be based on the attribute of the attribute of the first object and the second object come first
The object of control is selected between object and the second object.For ease of description, " object to be controlled " will be referred to as " target object
(or control device) ".
According to one exemplary embodiment, electronic equipment 100 can be selected based on the attribute of the attribute of the first object and the second object
The controllable pair between the first object and the second object is selected as target object.For example, when the attribute and the of the first object of analysis
When the result of the attribute of two objects indicates that the first object is things and the second object is display equipment, electronic equipment 100 can be
Select the second object (display equipment) as target object between first object and the second object.
According to one exemplary embodiment, relating attribute table 310 is in the first object and second between electronic equipment 100 can refer to object
The object of control is selected between object.For example, when the first object is TV, the attribute of the first object is " content (film) ", second
Object is lamps and lanterns, and when the attribute of the second object is " brightness ", and electronic equipment 100 can select in object relating attribute table 310
The first control information is selected (for example, control device：Lamps and lanterns, operation：20%) brightness is arranged to, the first control information, which corresponds to, to be made
The combination of " brightness " of the attribute for " content (film) " of the attribute of TV and as lamps and lanterns.Electronic equipment 100 can analyze first
Control information simultaneously checks whether lamps and lanterns are target objects.
In operating S240, electronic equipment 100 produces the operation signal of the object for selection.For example, operation signal
It can be the signal of the specific operation for the object (target object) for indicating selection, and be expressed as control signal.For example, electronics
Equipment 100 can determine the operation that will be executed by the object selected, and generate the operation signal of the determining operation of instruction.According to showing
Exemplary embodiment, when the object of selection has infrared (IR) transmitter, electronic equipment 100 can search for the object corresponding to selection
IR transmitters model IR signals, and using detection IR signals generate operation signal.
According to one exemplary embodiment, the attribute for the attribute and the second object that electronic equipment 100 can be based on the first object is come really
It is fixed that the operation executed by the object selected (is hereinafter referred to as controlled and operated).For example, control operation can be the object by selecting
One in the function of offer.For example, when target object is TV, control operation can be " channel change ", " video on demand
(VOD) play ", " brightness regulation ", one in " volume adjustment " and " application execution ", but not limited to this.Moreover, working as target pair
As if when lamps and lanterns, control operation can be one in " brightness regulation ", " color adaptation " and " adjusting of flicker cycle ", but not
It is limited to this.
According to one exemplary embodiment, when the second object is chosen as the target object between the first object and the second object,
Functional information among the attribute of status information and the second object that electronic equipment 100 can be among the attribute based on the first object come
Determine the control operation of the second object.For example, when the first object is environment and the second object (target object) is TV, electronics
Equipment 100 can be based on the first object status information (for example, current weather：Rain) and the second object functional information (such as frequency
Road changes) it is operated to determine that " current channel of TV is changed to weather channel " is used as to control.
According to one exemplary embodiment, electronic equipment 100 can be determined based on relating attribute table 310 between object corresponding to first
The operation of the attribute of the attribute of object and the second object.For example, when the first object is TV, the attribute of the first object is " content
(film) ", the second object are people, and when the attribute of the second object is " children ", and the second control letter may be selected in electronic equipment 100
Breath is (for example, control device：TV, operation：VOD is played for children), the second control information corresponds to the " content of the attribute as TV
The combination of " children " of (film) " and attribute as people.Electronic equipment 100 can analyze the second control information and inspection will be by TV
The operation of execution is " playing VOD for children ".
According to one exemplary embodiment, electronic equipment 100 can generate operation signal according to the control protocol of target object.Example
Such as, according to the control protocol of target object, electronic equipment 100 produce include the control command that can be read by target object behaviour
Make signal.
Meanwhile according to one exemplary embodiment, electronic equipment 100 can select the first object being displayed on the screen based on user
Operation signal is generated with the sequence of the second object.For example, when user selects the first object and then selects the second object,
Electronic equipment 100 produces the first operation signal and the first object is made to execute the first operation, and when user selects the second object
And when then selecting the first object, electronic equipment 100 produces the second operation signal and the first object is made to execute the second operation.
For example, when user touches the first object (children) being displayed on the screen with his or her finger and then drags the first object
When moving to the second object (audio system), it is the operation letter that children play music that electronic equipment 100, which produces instruction audio system,
Number.On the other hand, when user touches the second object (audio system) for being displayed on the screen and then with his or her finger
When second object is dragged to the first object (children), electronic equipment 100 produces instruction audio system by volume adjustment to use
In the operation signal for the grade 2 that children fall asleep.
According to one exemplary embodiment, when user selects the first object and then selects the second object, electronic equipment 100
The third operation signal for controlling the second object is produced, and when user selects the second object and then selects first pair
As when, electronic equipment 100 produces the 4th operation signal for controlling the first object.On the other hand, when user selects first
Object and when then selecting the second object, electronic equipment 100 produces the 5th operation signal for controlling the first object, and
And when user selects the second object and then selects the first object, electronic equipment 100 is produced for controlling the second object
The 6th operation signal.
For example, when user touches the TV being displayed on the screen with his or her finger and then touches lamps and lanterns with finger
When, electronic equipment 100 can generate the operation signal of the level of illumination for adjusting lamps and lanterns according to the content played on TV.And
And when user touches the lamps and lanterns being displayed on the screen with his or her finger and then touches TV with finger, electronic equipment
100 can generate the operation signal of the brightness for adjusting TV according to the level of illumination of lamps and lanterns.
In operating S250, electronic equipment 100 can send the operation signal generated to the object of selection.
Electronic equipment 100 according to one exemplary embodiment can be sent to target object by wired and or wireless communications and be wrapped
Include the operation signal of control command.For example, electronic equipment 100 can be used short haul connection (bluetooth, WFD, Li-Fi, UWB etc.) or
Mobile communication sends operation signal to target object.
According to one exemplary embodiment, electronic equipment 100 can or directly or by transponder device (such as server, family
Gateway, IoT hubs etc.) to target object send operation signal.For example, being grasped when electronic equipment 100 is sent to transponder device
When making identification information of the signal together with target object, transponder device can send operation signal to target object.According to demonstration
Property embodiment, the convertible operation signal of transponder device with the control protocol corresponding to target object, and by the operation of conversion believe
Number send target object to.
According to one exemplary embodiment, target object can analyze the operation signal of reception and execute the behaviour corresponding to operation signal
Make.For example, when audio system reception includes the operation signal for the instruction that volume is arranged to grade 3, audio system can be by sound
Amount is adjusted to grade 3 from grade 10.
Electronic equipment 100 further detailed below is received for selecting the first object and the among multiple objects
The operation input by user of two objects.
Fig. 4 is the flow chart for illustrating the method by imaging sensor identification objects according to one exemplary embodiment.
For example, the operation S410 and S420 of Fig. 4 can correspond to the operation S210 of Fig. 2.
In operating S410, electronic equipment 100 can show the multiple objects picked out by imaging sensor.For example, electric
Sub- equipment 100, which can be used, shows the live view of the main body picked out by imaging sensor to show the practical figure of multiple objects
Picture.
In operating S420, electronic equipment 100 can receive for selecting the first object and second pair among multiple objects
The user of elephant inputs.
For example, for selecting the user of the first object and the second object input alterable.For example, for selecting the first object
At least one of user's input with the second object can be touch input, voice inputs, eyes input and bending inputs, but
It is without being limited thereto.
Throughout the specification, " touch input " expression is made on the touchscreen by user with control electronics 100
Gesture etc..For example, the touch input described in exemplary embodiment can be tap, touch and keep, double-click, dragging, translating,
It flicks, drag and drop etc..
" tap " indicate the his or her finger of user or touch tool (such as electronic pen) touch screen and immediately from
Lift finger or touch tool on screen without its mobile action.
" touch and keep " indicates the his or her finger of user or touch tool (such as electronic pen) touch screen and will
Touch input maintains threshold time (such as two seconds) or longer action.For example, this is wherein to touch (touch-in) time
Point and difference of the touch-off (touch-out) between time point are threshold time (such as two seconds) or longer situation.In order to make
User discrimination, which goes out touch input, to be tap or touches and keep, when touch input be maintained threshold time or it is longer when, can be with
The mode of the sense of hearing or tactile provides feedback signal.Threshold time can change according to the example in time limit.
" double-click " indicates that the his or her finger of user or touch tool (such as electronic pen) touch the action of screen twice.
" dragging " indicates the his or her finger of user or touch tool touch screen and will while maintaining to touch
Finger or touch tool are moved to the action of another position in screen.Due to drag action, execution will be described below pair
As mobile or translation motion.
" translation " indicates that user carries out action of the drag action without selecting object.Due to translation motion do not select it is any right
As, therefore no object moves in the page, and page itself is moved in screen or object group moves in the page.
" flicking " indicate user using his or her finger or touch tool with threshold velocity (such as 100 pixels/second) or
Quickly carry out the action of drag action.Can be threshold velocity (such as 100 pictures based on the movement speed of finger or touch tool
Element/second) it still faster distinguishes dragging (or translation) action and flicks action.
" drag and drop " indicate that object is dragged to the position in screen simultaneously by user using his or her finger or touch tool
And put down the action of object.
" pinching " indicates that user moves the action of two fingers in different directions while touching screen with two fingers.
Pinch can be for amplifying (folder is opened) or reducing the gesture of (clamping) object, and value of magnification or diminution value according to two fingers it
Between distance determine.
" sliding " indicates while user's finger or touch tool touch the object in screen in horizontal or vertical direction
The his or her finger of upper movement or touch tool reach the action of a distance.Diagonally adjacent movement is not identified as sliding
Event.
According to one exemplary embodiment, electronic equipment 100 can receive for connecting dragging for the first object and the second object with line
Dynamic input.Moreover, electronic equipment 100 can receive tap, double-click or touch and keep every in the first object and the second object
One input, but touch input is without being limited thereto.
According to one exemplary embodiment, electronic equipment 100 can analyze the voice of user and identification is included in the voice of user
The first object identification information and the second object identification information.Moreover, when electronic equipment 100 includes flexible display,
Electronic equipment 100 can receive bending part region to select the bending of the first object and the second object to input.
According to one exemplary embodiment, when the first object and the second object are not shown in a screen, user can be
The first object is selected in first live view and the second object is selected in the second live view.At this point, being taken in real time second
In the subregion (such as upper left) of scape, electronic equipment 100 can be shown corresponding to the selected in the first live view
The icon of an object.
For example, after touching children in the first live view, user's mobile electronic equipment 100, and it includes TV to work as
The second live view when being displayed in electronic equipment 100, the tangible TV of user.It is further described hereinafter with reference to Fig. 5 B
In the first object and the second object be not shown in the situation in a screen.
In operating S430, electronic equipment 100 can recognize that the attribute of the attribute and the second object of the first object.
According to one exemplary embodiment, electronic equipment 100 can be identified by image procossing and be selected in live view by user
The first object and the second object.For example, by the image procossing of the image including the first object and the second object, electronic equipment
100 can will be TV corresponding to the first Object identifying of the first touch location, and will be corresponding to the second couple of the second touch location
As being identified as children.
According to one exemplary embodiment, electronic equipment 100 can recognize that the first object and the second object selected by user, and
Then the attribute for identifying the attribute and the second object of the first object, as described in the operation S220 above by reference to Fig. 2.
Fig. 5 A and Fig. 5 B are to illustrate electronic equipment according to one exemplary embodiment to receive the user's input for being used for selecting object
Operation figure.
With reference to the first screen 510 of figure 5A, electronic equipment 100 can be shown including TV, lamps and lanterns, audio frequency apparatus and washing machine
Live view.By the image procossing of the image corresponding to live view, electronic equipment 100 can be distinguished in live view TV,
The position of lamps and lanterns, audio frequency apparatus and washing machine.
Electronic equipment 100 can mark in live view by user option object.For example, when by user option
When object is TV, lamps and lanterns and audio frequency apparatus, electronic equipment 100 can show the first icon close to TV, the second figure close to lamps and lanterns
Be marked with and close to audio frequency apparatus third icon.
According to one exemplary embodiment, electronic equipment 100 can receive the input for selecting TV and lamps and lanterns in live view.
It touches and drags close to the first icon that TV is shown to the second icon shown close to lamps and lanterns for example, electronic equipment 100 can receive
Dragging input.By identifying that the starting points and end point of dragging input, electronic equipment 100 can identify user and have selected for
TV and lamps and lanterns.
With reference to the second screen 520, electronic equipment 100 can receive the input for drawing closed figures (such as circle, rectangle etc.).
The object in the object that closed figures are passed through and closed figures may be selected in electronic equipment 100.For example, when user takes in real time
When drawing across the circle of TV, lamps and lanterns and audio frequency apparatus on scape, electronic equipment 100 can identify user and have selected for TV, lamp
Tool and audio frequency apparatus.
According to one exemplary embodiment, electronic equipment 100 can receive for selecting three among three or more objects
Or more object and control the inputs of two or more objects.For example, electronic equipment 100 can be based on TV attribute, lamps and lanterns
Attribute and the attribute of audio frequency apparatus lamps and lanterns and audio frequency apparatus are determined as target object, send instruction according to existing to lamps and lanterns
First operation signal of the content changing illumination played on TV, and send content adjustment of the instruction according to broadcasting to audio frequency apparatus
Second operation signal of volume.
For ease of description, one feelings that wherein electronic equipment 100 will be controlled in two objects identified below
Condition is described as example.
With reference to third screen 530, electronic equipment 100 can receive the input carried out with electronic pen.For example, electronic equipment 100
It is defeated to can receive the dragging touched and dragged with electronics style of writing close to the first icon that TV is shown to the second icon shown close to lamps and lanterns
Enter.By checking that the starting points and end point of dragging input, electronic equipment 100 can identify user and have selected for TV and lamp
Tool.
With reference to the 4th screen 540, electronic equipment 100 can receive the input for drawing closed figures (such as circle, rectangle etc.).
For example, electronic equipment 100 can receive the circular input drawn with electronic pen across audio frequency apparatus and washing machine.
With reference to the 5th screen 550, electronic equipment 100 can receive for selecting the eyes of TV and lamps and lanterns defeated in live view
Enter.Eyes input indicate user adjust blink, watch attentively position, his or her eyeball movement speed etc. with control electronics
100 input.For example, electronic equipment 100 can be sensed looks at TV tri- seconds or longer on live view, his or her eyes are moved
Lamps and lanterns are moved, and then look at three seconds or longer users of lamps and lanterns input.What electronic equipment 100 can analyze user watches position attentively
And determine that user has selected for TV and lamps and lanterns.
With reference to figure 5B, when the first object and the second object are not shown in together on one screen, user is removable
Electronic equipment 100.In this case, the visual field for being included in the image lens in electronic equipment 100 is moved.
For example, when the first object is children 501 and the second object is TV 502, children 501 and TV 502 may that
This is separate, and is not shown in together in the live view of electronic equipment 100.In this case, when TV 502 is shown
When showing in the second live view 570, user can select children 501, mobile electronic device first in the first live view 560
100, and TV 502 is then selected in the second live view 570.According to one exemplary embodiment, when in user selection children
After 501 when mobile electronic device 100, electronic equipment 100 can show correspondence in some region of the screen of electronic equipment 100
In the icon 503 of children 501.
Meanwhile when the first object in a live view and the second object are not shown together, user is adjustable to be regarded
Angle.For example, by expanding visual angle (such as reducing), user may make the first object and the second object are displayed in take in real time
Jing Zhong.
Fig. 6 is the flow chart for illustrating the method that object is distinguished by short haul connection according to one exemplary embodiment.
In operating S610, electronic equipment 100 can distinguish multiple objects by short haul connection.For example, electronic equipment 100
The object in Bluetooth communication search Bluetooth communication radius can be used.Moreover, electronic equipment 100 can be by via BLE communications receptions
Object in BLE communication radius is distinguished by the BLE ad datas packet of object broadcast.Other than being communicated in addition to bluetooth and BLE
In various short-range communication methods, electronic equipment 100 can distinguish the object within the scope of some.For example, electronic equipment 100 can be used
WFD, near-field communication (NFC), Li-Fi etc. distinguish object.
According to one exemplary embodiment, electronic equipment 100 can determine multiple objects and electronics based on the intensity of signal of communication
The distance between equipment 100.For example, when the intensity of the signal of communication from the first object is more than the communication letter from the second object
Number intensity when, electronic equipment 100 can determine that the first object is more closer than the second object.Moreover, electronic equipment 100 can calculate pair
It should be in the distance of the intensity of signal of communication.
According to one exemplary embodiment, when electronic equipment 100 is connected to home network, electronic equipment 100 can receive about
It is connected to the information (such as object identity value, the current state of object, position etc.) of the object of home network.
In operating S620, electronic equipment 100 can show object map, the object map be included in it is one-to-one on the basis of
Corresponding to the image of multiple objects.
Object map can be the image by user option object among the object shown around electronic equipment 100
Mapping.For example, the image corresponding to multiple objects on the basis of one-to-one can be icon image, thumbnail image, mark
Will, figure, text image etc., but not limited to this.
According to one exemplary embodiment, object map can be realized in a variety of manners.For example, object map can be in plan view
On show storage object images form, or the image of equipment around electronic equipment 100 is shown in magnetic mapping
Form, but not limited to this.
It according to one exemplary embodiment, can be according to the image arranged with a distance from electronic equipment 100 corresponding to object.Example
Such as, when electronic equipment 100 than audio frequency apparatus closer to TV when, TV can be before audio frequency apparatus in object map.
In operating S630, electronic equipment 100 can receive for selecting the first object and second pair among multiple objects
The user of elephant inputs.
Here, it is used to select the user of the first object and the second object input alterable.For example, for selecting the first object
At least one of user's input with the second object can be touch input, voice inputs, eyes input and bending inputs, but
It is without being limited thereto.
According to one exemplary embodiment, electronic equipment 100 can receive in object map with line connect the first object and
The dragging of second object inputs.Moreover, electronic equipment 100 can receive tap, double-click or touch and keep the first object and the
The input of each in two objects, but touch input is without being limited thereto.
According to one exemplary embodiment, electronic equipment 100 can analyze the voice of user and discrimination is included in the voice of user
The first object identification information and the second object identification information.Moreover, when electronic equipment 100 includes flexible display,
Electronic equipment 100 can receive bending part region to select the bending of the first object and the second object to input.
Fig. 7 is the figure for illustrating object map according to one exemplary embodiment.
According to one exemplary embodiment, electronic equipment 100 can show the object map of the virtual image including surroundings
700.For example, object map 700 may include image (such as the TV icons for distinguishing the object of (such as sensing) by short haul connection
701, lamps and lanterns icon 702, dog 704, audio frequency apparatus icon 706, window 708, vacuum cleaner icon 710, computer icon
712, the image 714 and microwave icon 716 of electronic equipment 100).
According to one exemplary embodiment, when mobile phone is identified to be gone out, electronic equipment 100 can be shown in object map 700
Show the face-image of mobile phone user.For example, when identifying the phone of the first user (such as mother), it can be in object map
The face-image 703 of the first user (such as mother) is shown in 700.
According to one exemplary embodiment, even if when not identifying predefined object by short haul connection, electronic equipment
100 can also show the image of predefined object in object map 700.For example, when user be set such that it is respective
When the image of kinsfolk is displayed in object map 700, electronic equipment 100 can show the first use in object map 700
The face-image 703 at family (such as mother), the face-image of second user (such as daughter), doggie image etc..Moreover, according to
Exemplary embodiment, electronic equipment 100 can always show the window icons 708 for indicating environment in object map 700.
According to one exemplary embodiment, user can be checked selectable object by object map 700 and selectable right
Two objects are selected as among.It touches for example, electronic equipment 100 can receive and drags the dragging to lamps and lanterns icon 702 of TV icons 701
Dynamic input.The location information for the object map icon being included in object map 700 can be used to distinguish user for electronic equipment 100
Have selected for TV and lamps and lanterns.
Electronic equipment 100 described in detail below controls first based on the attribute of the attribute of the first object and the second object
One operation in object and the second object.
Fig. 8 is the electronic equipment control pair illustrated when all multiple objects are controllable devices according to one exemplary embodiment
The sequence chart of the method for elephant.In fig. 8, wherein the first object 801 and the second object 802 will can both be set with electronics
The case where equipment of standby 100 communication, is described as example.
In operating S810, electronic equipment 100 can distinguish the first object 801 and the second object 802.
According to one exemplary embodiment, imaging sensor or camera discrimination can be used to be present in electronic equipment for electronic equipment 100
The first object 801 outside 100 and the second object 802.Moreover, electronic equipment 100 can distinguish first pair by short haul connection
As 801 and second object 802.
Meanwhile when identifying three or more objects by camera or communication interface, electronic equipment 100 can be based on using
Family input selects the first object 801 and the second object 802 among the object identified.
Since operation S810 corresponds to the operation S210 of Fig. 2, detailed description thereof will not be repeated.
In operating S820, the communication connection with the first object 801 and the second object 802 can be arranged in electronic equipment 100.Example
Such as, when electronic equipment 100 is not attached to the first object 801 and the second object 802 for communication, electronic equipment 100 can be built
Vertical and the first object 801 the first communication link and the second communication link of foundation and the second object 802.
Electronic equipment 100 can by with 802 exchange identification information of the first object 801 and the second object (such as device id, matchmaker
The address body access control (MAC), device name etc.), functional information (such as the support to BLE, bluetooth, Ant+, Wi-Fi and
NFC), information about preferred communication method (such as bluetooth) etc. establishes the first communication link and the second communication link.
According to one exemplary embodiment, the first communication link and the second communication link may include blueteeth network, BLE networks, nothing
Line LAN (WLAN and/or Wi-Fi network), WFD networks, UWB network and mobile communications network (such as the second generation (2G), 3G,
The networks such as 4G, 5G), but not limited to this.
In operating S830, the first object 801 can send the first attribute information to electronic equipment 100.For example, the first object
801 can send the first attribute information (such as current state of the first object 801 by the first communication link to electronic equipment 100
Information, about the information etc. for the function of being supported by the first object 801).
According to one exemplary embodiment, the first object 801 periodically can send the first attribute information to electronic equipment 100,
Or the first attribute information can be sent to electronic equipment 100 when receiving request from electronic equipment 100.
In operating S840, the second object 802 can send the second attribute information to electronic equipment 100.For example, the second object
802 can send the second attribute information (such as current state of the second object 802 by the second communication link to electronic equipment 100
Information, about the information etc. for the function of being supported by the second object 802).
According to one exemplary embodiment, the second object 802 periodically can send the second attribute information to electronic equipment 100,
Or the second attribute information can be sent to electronic equipment 100 when receiving request from electronic equipment 100.
In operating S850, relating attribute table can be used between object to determine target object and operation in electronic equipment 100.
According to one exemplary embodiment, relating attribute table 310 comes in the first object 801 between electronic equipment 100 can refer to object
The target object of control is selected between the second object 802.Moreover, electronic equipment 100 can determine corresponding to the first object 801
The operation of the attribute of attribute and the second object 802.For example, when the first object 801 is TV, the attribute of the first object 801 is " content
(film) ", the second object 802 are lamps and lanterns, and when the attribute of the second object 802 is " brightness ", and electronic equipment 100 can be in object
Between in relating attribute table 310 the first control of selection information (for example, control device：Lamps and lanterns, operation：20%) brightness is arranged to,
The first control information corresponds to the combination of " brightness " of " content (film) " of the attribute as TV and the attribute as lamps and lanterns.
Electronic equipment 100 can analyze the first control and check that lamps and lanterns are target objects, and be " by brightness by the operation executed by lamps and lanterns
Grade is arranged to 20% ".
In operating S860, electronic equipment 100 produces and sends operation signal.For example, when the first object 801 is determined
For target object when, electronic equipment 100 can send in operating S870 to the first object 801 the corresponding to the first object 801
One operation signal.In operating S875, executable the first operation corresponding to the first operation signal of the first object 801.
When the second object 802 is targeted object, electronic equipment 100 can be in operating S880 to the second object
802 send the second operation signal corresponding to the second object 802.In operating S885, the second object 802 is executable to correspond to the
Second operation of two operation signals.
According to one exemplary embodiment, some in operation S810 to S885 can be omitted, or the suitable of some operations can be changed
Sequence.For example, operation S820 can be executed before operating S810.Moreover, can only omit operation S820 to S840.
Hereinafter with reference to Fig. 9 detailed descriptions wherein electronic equipment 100 object is collected before the object that user selects control
Attribute information the case where.
Fig. 9 is to illustrate when electronic equipment is connected to multiple objects to communicate being set for electronics according to one exemplary embodiment
It is standby that the sequence chart of the method for an object is controlled based on the attribute of multiple objects.In fig.9, it will will wherein the first object
901 and second object 902 can both be communicated with electronic equipment 100 equipment the case where be described as example.
Electronic equipment 100 can establish the first communication link with the first object 901 in operating S905, and operate
The second communication link with the second object 902 is established in S910.
According to one exemplary embodiment, electronic equipment 100 can directly by short haul connection (such as bluetooth, Wi-Fi etc.) or
Person is connected indirectly to the first object 901 and the second object 902 by home gateway.
Since operation S905 and S910 corresponds to the operation S820 of Fig. 8, detailed description thereof will not be repeated.
In operating S920, the attribute information of 100 collectible object of electronic equipment.For example, in operating S925, electronics is set
Standby 100 can receive the first attribute information by the first communication link from the first object 901.For example, the first attribute information may include
Identification information, status information and the functional information of first object 901, but not limited to this.
In operating S930, electronic equipment 100 can receive the second attribute letter by the second communication link from the second object 902
Breath.For example, the second attribute information may include at least one in identification information, status information and the functional information of the second object 902
It is a, but not limited to this.
According to one exemplary embodiment, electronic equipment 100 can with certain intervals (such as 10 minutes) from the first object 901 or
Second object 902 receives the first attribute information or the second attribute information.Moreover, when particular event occurs, electronic equipment 100 can
The first attribute information or the second attribute information are received from the first object 901 or the second object 902.For example, particular event may include
The event that wherein electric power of the first object 901 or the second object 902 is switched on, wherein the first object 901 or the second object 902 are built
The vertical event with the communication link of electronic equipment 100, wherein the thing that the state of the first object 901 or the second object 902 is changed
Part etc., but not limited to this.
According to one exemplary embodiment, electronic equipment 100 can be new based on being received from the first object 901 or the second object 902
The first attribute information or the second new attribute information come the data of attribute information library (DB) of upgating object.
In operating S940, electronic equipment 100 can recognize that the first object 901 and the second object 902.
According to one exemplary embodiment, imaging sensor or camera identification can be used to be present in electronic equipment for electronic equipment 100
The first object 901 outside 100 and the second object 902.Moreover, short haul connection can be used to identify first in electronic equipment 100
Object 901 and the second object 902.
Meanwhile when three or more objects are by by camera or communication interface identification, electronic equipment 100 can be based on using
Family input selects the first object 901 and the second object 902 among the object of identification.Since operation S940 corresponds to the behaviour of Fig. 2
Make S210, therefore detailed description thereof will not be repeated.
In operating S950, electronic equipment 100 can extract the first attribute information and the second attribute information.
According to one exemplary embodiment, electronic equipment 100 can be in the attribute information DB of the attribute information for the object that storage is collected
Middle search first attribute information corresponding with the identification information of the first object 901.Moreover, electronic equipment 100 can be in attribute information
Second attribute information corresponding with the identification information of the second object 902 is searched in DB.
In operating S960, using relating attribute table between object, the first object 901 can be determined as controlling by electronic equipment 100
The target object of system.According to one exemplary embodiment, relating attribute table can be included in attribute information DB between object, or can be with
Attribute information DB separation.
According to one exemplary embodiment, relating attribute table 310 checks and the first object between electronic equipment 100 can refer to object
The corresponding first control information of second attribute information of 901 the first attribute information and the second object 902.When the first object 901
When being defined as control device in the first control information, electronic equipment 100 may be selected the first object 901 and be used as target pair
As.
In operating S970, electronic equipment 100 produces the first operation signal.
According to one exemplary embodiment, electronic equipment 100 can determine and the first attribute information of the first object 901 and second pair
As the 902 corresponding operation of the second attribute information.Moreover, electronic equipment 100 produces the first behaviour corresponding to determining operation
Make signal.The first operation signal can be generated based on the control protocol information of the first object 901.
Since operation S970 corresponds to the operation S240 of Fig. 2, detailed description thereof will not be repeated.
In operating S980, electronic equipment 100 can send the first operation signal to the first object 901.For example, electronic equipment
100 can send the first operation signal by the first communication link to the first object 901.
In operating S990, the first object 901 can execute the first operation according to the first operation signal.According to exemplary implementation
Example, when the first operation is completed, the first object 901 can be sent completely message to electronic equipment 100.
Hereinafter with reference to Figure 10 A, 10B and 10C detailed description wherein attribute information of the electronic equipment 100 based on multiple objects
To control the exemplary embodiment of special object.
Figure 10 A, 10B and 10C are the categories for illustrating attribute and lamps and lanterns of the electronic equipment based on TV according to one exemplary embodiment
Property controls the figure of the operation of lamps and lanterns.
With reference to figure 10A, electronic equipment 100 can show the live view for including multiple objects.The recognizable packet of electronic equipment 100
It includes multiple objects in live view and shows instruction of the instruction in multiple identified objects of corresponding data collection
It accords with (such as icon).Moreover, electronic equipment 100 can show the status information of each in multiple objects in live view.
For example, electronic equipment 100 may recognize that TV 1001, lamps and lanterns 1002, audio frequency apparatus 2224, washing machine 2226 and machine
Device people vacuum cleaner 2228.Moreover, electronic equipment 100 can show the status information of TV 1001 (for example, viewing, volume：
60), the status information of lamps and lanterns 1002 is (for example, brightness：70%), the status information of audio frequency apparatus 2224 is (for example, wait for, volume：
60), the status information of washing machine 2226 is (for example, cleaning, remaining time：30 minutes), the shape of robotic vacuum cleaner 2228
State information is (for example, cleaning, battery：It is remaining 60%) etc..
When user wants to watch the film played on TV 1001 in dark surrounds, electronic equipment 100 can pass through reality
When find a view reception for select the user of TV 1001 and lamps and lanterns 1002 input.It touches and drags for example, electronic equipment 100 can receive
The dragging of dynamic TV 1001 to lamps and lanterns 1002 inputs.
In this case, electronic equipment 100 can recognize that the attribute (for example, playing movie contents) and lamps and lanterns of TV 1001
1002 attribute is (for example, brightness：70%) relating attribute table 310 determines target object and target behaviour, and between reference object
Make.For example, lamps and lanterns 1002 can be determined as target object by electronic equipment 100, and by " brightness：It is arranged to 10% " and is determined as target
Operation.
With reference to figure 10B, electronic equipment 100 can show pop-up window 1003, which is based on TV
The target object (lamps and lanterns 1002) and object run (" brightness that 1001 attribute and the attribute of lamps and lanterns 1002 determine：It is arranged to
10% ").For example, electronic equipment 100 can show " whether you want to adjust the illumination for film mode including message" bullet
Go out window.
With reference to figure 10C, when user selects "Yes" in pop-up window, electronic equipment 100 produces instruction by brightness etc.
Grade is arranged to 10% operation signal.Moreover, electronic equipment 100 can send the operation signal generated to lamps and lanterns.According to reception
Operation signal, lamps and lanterns can be adjusted brightness degree to 10% (reference numeral 1004) from 70%.
Figure 11 is illustrated when the only one in the multiple objects identified by electronic equipment is controllable device according to demonstration
The sequence chart of the method for electronic equipment control object of property embodiment.In fig. 11, will be wherein the first object 1101
Controllable device and the case where the second object 1102 is uncontrollable things is described as example.
In operating S1110, electronic equipment 100 can distinguish the first object 1101 and the second object 1102.
According to one exemplary embodiment, imaging sensor or camera discrimination can be used to be present in electronic equipment for electronic equipment 100
The first object 1101 and the second object 1102 of 100 outside.Moreover, electronic equipment 100 can distinguish the by short haul connection
At least one of an object 1101 and the second object 1102.
Meanwhile when three or more objects are by by camera or communication interface discrimination, electronic equipment 100 can be based on using
Family input selects the first object 1101 and the second object 1102 in the object identified.
Since operation S1110 corresponds to the operation S210 of Fig. 2, detailed description will not be repeated.
In operating S1120, the communication connection with the first object 1101 can be arranged in electronic equipment 100.
For example, when electronic equipment 100 is not attached to the first object 1101 for communication, electronic equipment 100 can ask
Communication connection from the first object 1101 is to establish communication link.
According to one exemplary embodiment, communication link may include blueteeth network, BLE networks, WLAN (Wi-Fi network), WFD nets
At least one of network, UWB network and mobile communications network (such as the networks such as 2G, 3G, 4G, 5G), but not limited to this.
Meanwhile second object 1102 can be things (such as people or environment), and thus electronic equipment 100 cannot be established
With the communication link of the second object 1102.
In operating S1130, the first object 1101 can send the first attribute information to electronic equipment 100.
For example, the first object 1101 can send the first attribute information (such as first by communication link to electronic equipment 100
The current state information of object 1101, about the information etc. for the function of being supported by the first object 1101).
According to one exemplary embodiment, the first object 1101 periodically can send the first attribute information to electronic equipment 100,
Or the first attribute information can be sent to electronic equipment 100 when receiving request from electronic equipment 100.
Simultaneously as the second object 1102 is things (or people or environment), therefore electronic equipment 100 can not be from second
Object 1102 receives the second attribute information of the second object 1102.Therefore, according to one exemplary embodiment, electronic equipment 100 can be from
Reservoir obtains the second attribute information of the second object 1102.For example, electronic equipment 100 can be in being stored in reservoir attribute
Second attribute information of the search corresponding to the identification information of the second object 1102 in information DB.
In operating S1140, first object 1101 is determined as mesh by relating attribute table between object can be used in electronic equipment 100
Mark object.
For example, since the first object 1101 is controllable device and the second object 1102 is uncontrollable things, electricity
Sub- equipment 100 can refer between object relating attribute table 310 by the first object between the first object 1101 and the second object 1102
1101 are selected as the target of control.
In operating S1150, electronic equipment 100 produces the operation signal for the first object 1101.
According to one exemplary embodiment, electronic equipment 100 can determine corresponding to the first object 1101 the first attribute information and
The operation of second attribute information of the second object 1102.Then, electronic equipment 100 produces the operation corresponding to determining operation
Signal.Operation signal can be generated based on the control protocol information of the first object 1101.
Since operation S1150 corresponds to the operation S240 of Fig. 2, detailed description thereof will not be repeated.
In operating S1160, electronic equipment 100 can send operation signal to the first object 1101.For example, electronic equipment
100 can send operation signal by the short-range communication link pre-established to the first object 1101.
In operating S1170, the executable operation corresponding to operation signal of the first object 1101.According to exemplary implementation
Example, when that operation is complete, the first object 1101 can be sent completely message to electronic equipment 100.
The exemplary implementation of the first object 1101 is controlled hereinafter with reference to the detailed descriptions of Figure 12 to 14 wherein electronic equipment 100
Example.
Figure 12 is to illustrate the attribute of attribute and TV of the electronic equipment based on people according to one exemplary embodiment to control TV's
The figure of operation.
With reference to the screen 1210 of figure 12, electronic equipment 100 can show live view, which includes wherein people 1201
It is sitting in the image on 1202 sides TV.For example, electronic equipment 100 can be shown in live view by image processing techniques to identify
In people 1201 and TV 1202.
Electronic equipment 100 can receive the dragging input of the image of image and TV 1202 for connecting people 1201.Electronics is set
The standby attribute (such as children) of 100 recognizable people 1201 and the attribute (such as content) of TV 1202.
Between object in relating attribute table 310, electronic equipment 100 can search for control information (for example, control device：TV, behaviour
Make：Play the VOD of children), which corresponds to " content " of the attribute as TV 1202 and as the attribute of people 1201
" children " combination.By analysis and Control information, TV 1202 can be determined as target object by electronic equipment 100, and check by
The operation executed by TV is " to play VOD for children.”
With reference to figure 12, electronic equipment 100 produces the operation letter for the control command for including instruction " playing VOD for children "
Number.Then, electronic equipment 100 can send the operation signal generated by short haul connection to TV 1220.
Based on the operation signal received from electronic equipment 100, TV 1220 can be children search for or play VOD content (such as
Animated content).
According to one exemplary embodiment, without complex process, user can be shown in electronics by connection and set
Children and TV 1220 in standby 100 live view make on TV 1220 for children's broadcasting VOD content.
Figure 13 be illustrate the attribute of attribute and audio system of the electronic equipment based on surrounding according to one exemplary embodiment come
Control the figure of the operation of audio system.
With reference to the screen 1310 of figure 13, electronic equipment 100 can show live view, which includes that its moderate rain is shown
Go out outside window 1301 and audio system 1302 is located at the image of adjacent window 1301.For example, electronic equipment 100 can pass through
Image processing techniques identifies the window being shown in live view 1301 and audio system 1302.
The dragging that electronic equipment 100 can receive the image of image and audio system 1302 for connecting window 1301 is defeated
Enter.Electronic equipment 100 can recognize that the attribute (weather) of window 1301 and the attribute (music content) of audio system 1302.For example,
When user touches window 1301, weather application can be used to check weather (rain) in electronic equipment 100.Moreover, electronic equipment 100
The raindrop fallen outside window 1301 can be recognized by image processing techniques.
Between object in relating attribute table 310, electronic equipment 100 can search for the " weather with the attribute as window 1301：
The combination of " music content " of rain " and attribute as audio system 1302 for control information (for example, control device：Sound
Display system, operation：Play the music for being suitable for weather).By analyzing the control information, electronic equipment 100 can be by audio system
1302 are determined as target object, and check that by the operation executed by audio system be " music for playing suitable weather ".
With reference to figure 13, electronic equipment 100 produces operation signal, which includes that " it is good on rainy day to play for instruction
That listens recommends music " control command.Then, electronic equipment 100 can be sent by short haul connection to audio system 1320
The operation signal of generation.
When it is correspond to " rainy day it is pleasing to the ear recommend music " music file be stored in audio system 1320
When middle, audio system 1320 can be searched in music servers " rainy day it is pleasing to the ear recommend music " and play the music.
Moreover, when without correspond to " rainy day it is pleasing to the ear recommend music " music file be stored in audio system
When in 1320, electronic equipment 100 can to audio system 1320 send correspond to " rainy day it is pleasing to the ear recommend music " sound
Music file.For example, music file may be stored in the reservoir of electronic equipment 100, or can be in music servers
It is searched in real time.
According to one exemplary embodiment, audio system 1320 can be downloaded and playing music, or can broadcast in streaming fashion
Playback music file.
Figure 14 is to illustrate the attribute of attribute and dehumidifier of the electronic equipment based on things according to one exemplary embodiment to control
The figure of the operation of dehumidifier processed.
With reference to the screen 1410 of figure 14, electronic equipment 100 can show real-time including clothes hanger 1401 and dehumidifier 1402
It finds a view.For example, electronic equipment 100 can identify the clothes hanger being included in live view 1401 by image processing techniques and remove
Wet device 1402.
The dragging that electronic equipment 100 can receive the image of image and dehumidifier 1402 for connecting clothes hanger 1401 is defeated
Enter.Electronic equipment 100 can recognize that the attribute (such as clothing) of clothes hanger 1401 and the attribute (such as dehumidifying) of dehumidifier 1402.
Between object in relating attribute table 310, electronic equipment 100 can search for the " clothing with the attribute as clothes hanger 1401
The corresponding control information of combination of " dehumidifying " of object " and attribute as dehumidifier 1402 is (for example, control device：Dehumidifier, behaviour
Make：It is powered).By analyzing the control information, dehumidifier 1402 can be determined as target object by electronic equipment 100, and check by
The operation executed by dehumidifier 1420 is " activation dehumidification function ".
With reference to figure 14, electronic equipment 100 produces the operation signal for the control command for including instruction " activation dehumidification function ".
Then, electronic equipment 100 can send the operation signal generated by short haul connection to dehumidifier 1420.
Based on the operation signal received from electronic equipment 100, dehumidifier 1420 can open power supply and execute dehumidification function.
According to one exemplary embodiment, without complex process, user can be shown in electronics by connection and set
The image of clothes hanger 1401 in standby 100 live view and the image of dehumidifier 1402 make the operation of dehumidifier 1420.
Figure 15 is illustrated when all multiple objects are characterized as uncontrollable object by electronic equipment according to exemplary implementation
The sequence chart of the method for the electronic equipment control extra objects of example.It in fig.15, will be wherein the first object 1501 and second pair
The case where as 1502 being all uncontrollable object, is described as example.
In operation S1510, electronic equipment 100 can distinguish the first object 1501 and the second object 1502.
According to one exemplary embodiment, imaging sensor or camera discrimination can be used to be present in electronic equipment for electronic equipment 100
The first object 1501 outside 100 and the second object 1502.Moreover, electronic equipment 100 can distinguish first by short haul connection
At least one of object 1501 and the second object 1502.
Meanwhile when three or more objects are by by camera or communication interface discrimination, electronic equipment 100 can be based on using
Family input selects the first object 1501 and the second object 1502 among the object identified.
Since operation S1510 corresponds to the operation S210 of Fig. 2, detailed description thereof will not be repeated.
In operating S1520, third object is determined as mesh by relating attribute table 310 between object can be used in electronic equipment 100
Mark object.
For example, when the first object 1501 and the second object 1502 are uncontrollable objects, electronic equipment 100 cannot be from first
Selection target object in object 1501 and the second object 1502.Therefore, electronic equipment 100 can be based on the attribute of the first object 1501
Controllable third object is selected with the attribute of the second object 1502.
In operating S1530, electronic equipment 100 can determine that the third object for being targeted object is to be present in electronics
External equipment 1503 or electronic equipment 100 itself outside equipment 100.
In operation S1540, electronic equipment 100 produces operation signal to be sent to external equipment 1503.
Electronic equipment 100 can determine external equipment 1503 based on the attribute of the attribute of the first object 1501 and the second object
Operation.Then, electronic equipment 100 produces the operation signal corresponding to determining operation.It can be based on external equipment 1503
Control protocol information generates operation signal.
In operating S1550, when the third object for being targeted object is external equipment 1503, electronic equipment
100 can establish the communication link with external equipment 1503.
For example, when electronic equipment 100 is not attached to external equipment 1503 for communication, electronic equipment 100 can ask
Communication connection from external equipment 1503 is to establish communication link.According to one exemplary embodiment, communication link may include bluetooth
Network, BLE networks, WLAN (Wi-Fi network), WFD networks, UWB network and mobile communications network (such as 2G, 3G, 4G, 5G etc.
At least one of network), but not limited to this.
According to one exemplary embodiment, electronic equipment 100 can directly by short haul connection (such as bluetooth, Wi-Fi etc.) or
It is connected indirectly to external equipment 1503 by home gateway.
Meanwhile when electronic equipment 100 has been connected to external equipment 1503 for communication in advance, operation can be omitted
S1550。
In operating S1560, electronic equipment 100 can send operation signal to external equipment 1503.For example, electronic equipment
100 can send operation signal by the short-range communication link pre-established to external equipment 1503.
In operating S1570, the executable operation corresponding to operation signal of external equipment 1503.According to exemplary implementation
Example, when executing operation, external equipment 1503 can send operation to electronic equipment 100 and start message.
In operating S1540 and S1580, when the third object for being targeted object is not external equipment 1503,
Electronic equipment 100 can perform determining operation.For example, electronic equipment 100 oneself can execute the attribute based on the first object 1501
The operation determined with the attribute of the second object 1502, without generating the operation signal for external equipment 1503.
The exemplary embodiment of third object is controlled hereinafter with reference to the detailed descriptions of Figure 16 to 20 wherein electronic equipment 100.
Figure 16 is to illustrate the attribute of attribute and doll of the electronic equipment based on people according to one exemplary embodiment to control TV
Operation figure.In figure 16, by wherein TV 1603 is the external equipment being present in outside electronic equipment 100 the case where retouch
It states as example.
With reference to the screen 1610 of figure 16, electronic equipment 100 can show live view, which includes wherein people 1601
The image played together with doll 1602.For example, electronic equipment 100 can be shown in live view by image processing techniques to identify
In people 1601 and doll 1602.
Electronic equipment 100 can receive the dragging input of the image of image and doll 1602 for connecting people 1601.Electronics
Equipment 100 can recognize that the attribute of people 1601 (for example, the age：Children) and doll 1602 attribute (such as role).
Between object in relating attribute table 310, electronic equipment 100 can search for control information (for example, control device：TV, behaviour
Make：Play the animated content for role wherein occur), which corresponds to " children " and the conduct of the attribute as people 1601
The combination of " role " of the attribute of doll 1602.By analysis and Control information, TV 1603 can be determined as mesh by electronic equipment 100
Object is marked, and it is " to play the animated content for role wherein occur to check the operation executed by TV 1603.”
With reference to figure 16, when doll 1602 is the doll of first role, it includes that instruction " plays that electronic equipment 100, which produces,
Wherein there is the animated content of first role " control command operation signal.Then, electronic equipment 100 can pass through short distance
It communicates to TV 1603 and sends the operation signal generated.
Based on the operation signal received from electronic equipment 100, TV 1603 can play in the animation for first role wherein occur
Hold.
When the animated content for not occurring first role wherein is stored in TV 1603, TV 1603 can be carried in content
For searching for animated content and playing animation content in server.Alternatively, electronic equipment 100 is transmittable wherein there is first role
Animated content together with operation signal.
According to one exemplary embodiment, without complex process, user can simply play on TV1603
The animated content that wherein doll 1602 of children occurs.
Figure 17 be illustrate attribute of the electronic equipment based on the people picked out by electronic equipment according to one exemplary embodiment and
The attribute of doll carrys out the figure of the operation of broadcasting content.It in fig. 17, will be wherein electronic equipment 100 is mobile phone and TV is
The case where external equipment, is described as example.
With reference to the screen 1710 of figure 17, electronic equipment 100 can show live view, which includes wherein people 1701
The image played together with doll 1702.For example, electronic equipment 100 can be shown in live view by image processing techniques to identify
In people 1701 and doll 1702.
Electronic equipment 100 can receive the dragging input of the image of image and doll 1702 for connecting people 1701.Electronics
Equipment 100 can recognize that the attribute of people 1701 (for example, the age：Children) and doll 1702 attribute (such as role).
Between object in relating attribute table 310, electronic equipment 100 can search for as people 1701 attribute " children " and
The corresponding control information of combination of " role " of the attribute as doll 1702 is (for example, control device：TV, operation：It plays wherein
There is the animated content of role).By analyzing the control information, TV can be determined as target object by electronic equipment 100, and be checked
It is " to play the animated content for role wherein occur by the operation executed by TV.“
With reference to the screen 1720 of figure 17, electronic equipment 100 can be attempted to establish communication link with TV.However, working as user
When outdoor object for appreciation, electronic equipment 100 may cannot connect to TV for communication.
Electronic equipment 100 can show the acknowledgement window 1703 for being played for oneself and the animated content of role wherein occur.
With reference to the screen 1730 of figure 17, when user selects " mobile to play " in acknowledgement window 1703, electronic equipment 100
Oneself it can play the animated content for role wherein occur.
According to one exemplary embodiment, electronic equipment 100 can play animated content stored in memory, or can including
Hold and search animated content and playing animation content in server are provided.
Figure 18 is attribute of the electronic equipment based on the refrigerator identified by electronic equipment illustrated according to one exemplary embodiment
The figure of the operation of notice is shown with the attribute of people.
With reference to the screen 1810 of figure 18, electronic equipment 100 can show the live view for including refrigerator 1801.For example, electronics
Equipment 100 can distinguish the refrigerator being shown in live view 1801 by image processing techniques.
Moreover, electronic equipment 100 can recognize that the attribute information of refrigerator 1801, and can be in multiple attributes and 1801, refrigerator
Timing display indicates the icon of multiple attributes.For example, when there is " grocery store's shopping " among the attribute corresponding to refrigerator 1801
When " expiration date ", electronic equipment 100 can show grocery store's shopping icon 1811 and expiration date icon 1812.
Meanwhile electronic equipment 100 can show the object panels 1800 including at least one object images.According to exemplary real
Example is applied, object panels 1800 can be located in some region of screen.Moreover, object panels 1800 are not shown usually, but can be
On the side for appearing in screen under the request of user.
According to one exemplary embodiment, the object images being shown in object panels 1800 can frequently be selected by user
The image of object.New object images can be added to object panels 1800 by user, or be removed at least from object panels 1800
One object images.
Electronic equipment 100 can receive for connecting expiration date icon 1812 and corresponding with the user of electronic equipment 100
The dragging of icon 1802 inputs.Electronic equipment 100 can recognize that the attribute (such as expiration date) of refrigerator 1801 and the attribute (example of people
Such as the user of electronic equipment 100).
Between object in relating attribute table 310, electronic equipment 100 can search for " expired with the attribute as refrigerator 1801
The corresponding control information of combination of " user of electronic equipment 100 " of date " and attribute as people is (for example, control device：Electricity
Sub- equipment 100, operation：Show the food list very close to its expiration date).Pass through analysis and Control information, electronic equipment 100
It can check that by the operation executed by electronic equipment 100 be " showing the food list very close to its expiration date ".
With reference to the screen 1820 of figure 18, electronic equipment 100 can show the food list 1803 of its very close expiration date.
For example, electronic equipment 100 can establish the communication link with refrigerator 1801, and received from refrigerator 1801 very close to it
The information of the food (such as milk, sausage, egg, chilli powder etc.) of expiration date.Then, electronic equipment 100 can show pop-out
Mouthful, which includes the letter about the food (such as milk, sausage, egg, chilli powder etc.) of its very close expiration date
Breath.
Figure 19 is attribute of the electronic equipment based on the refrigerator identified by electronic equipment illustrated according to one exemplary embodiment
The figure of the operation of the display of external equipment is controlled with the attribute of people.
With reference to the screen 1910 of figure 19, electronic equipment 100 can show object panels 1800 and including the real-time of refrigerator 1901
It finds a view.
Electronic equipment 100 can recognize that the attribute information of refrigerator 1901, and can be when multiple attributes are matched with refrigerator 1901
Display indicates the icon of multiple attributes.For example, when there is " grocery store's shopping " and " mistake among the attribute corresponding to refrigerator 1901
When date phase ", electronic equipment 100 can show grocery store's shopping icon 1911 and expiration date icon 1912.
Electronic equipment 100 can receive for connecting grocery store's shopping icon 1911 and corresponding to second user (such as wife)
Icon 1902 dragging input.Electronic equipment 100 can recognize that the category of attribute (such as grocery store's shopping) and people of refrigerator 1901
Property (such as second user).
Between object in relating attribute table 310, electronic equipment 100 can search for the " groceries with the attribute as refrigerator 1901
Do shopping in shop " and attribute as people " second user " the corresponding control information of combination (for example, control device：Second user
Mobile phone 1900, operation：Show grocery store's Shopping List).By analyzing the control information, electronic equipment 100 can be used second
The mobile phone 1900 at family is determined as target object, and the operation executed by the mobile phone 1900 of second user is by inspection
" display grocery store's Shopping List ".
With reference to the screen 1920 of figure 19, electronic equipment 100 produces operation signal, which includes instruction " display
The control command of grocery store's Shopping List ".Then, electronic equipment 100 can be by short haul connection to the mobile electricity of second user
Words 1900 send the operation signal generated.In this case, the mobile phone 1900 of second user can show that grocery store does shopping
List 1903 (such as milk, hair conditioner, egg, paper towel box etc.).
According to one exemplary embodiment, electronic equipment 100 can send to the mobile phone 1900 of second user and be stored in storage
Grocery store's Shopping List 1903 in device, or grocery store's Shopping List 1903 can be received from refrigerator 1901 and to second user
Mobile phone 1900 sends grocery store's Shopping List 1903.
Moreover, the mobile phone 1900 of second user can be from refrigerator 1901 or Relay Server (such as home gateway or IoT
Hub) grocery store's Shopping List 1903 is received, and the groceries Shopping List 1903 of reception is shown on the screen.
Figure 20 is the figure for the operation for illustrating electronic equipment remote control external equipment according to one exemplary embodiment.
With reference to the screen 2000-1 of figure 20, electronic equipment 100 can show the sound including playing the first user (such as Jane)
The live view of first TV 2010 of happy meeting video.For example, electronic equipment 100 can be logical by image processing techniques or short distance
Letter distinguishes the first TV 2010 being shown in live view.
Moreover, electronic equipment 100 can show the object panels 1800 including at least one object images.Work as second user
When (user of electronic equipment 100) touch corresponds to the icon 2020 of third user (such as Tom) in object panels 1800, electronics
Equipment 100 can be shown and the matched attribute of third user.For example, electronic equipment 100 can show mirror image icon 2021 and setting figure
Mark 2022.
Electronic equipment 100 can receive the dragging input of the image for connecting the first TV 2010 and mirror image icon 2021.Electricity
Sub- equipment 100 can recognize that the attribute (such as mirror image) of the attribute (such as content) and third user of the first TV 2010.
Between object in relating attribute table 310, electronic equipment 100 can search for control information (for example, control device：Third
The 3rd TV 2030 of user, operation：It is shown in the content played on the first TV 2010), which corresponds to as the
The combination of " mirror image " of " content " of the attribute of one TV 2010 and the attribute as third user.Pass through analysis and Control information, electricity
The 3rd TV 2030 of third user can be determined as target object and check the behaviour that will be executed by the 3rd TV 2030 by sub- equipment 100
Work is " content (concert videos of the first user (such as Jane)) that display is just playing on the first TV 2010 ".
With reference to the reference numeral 2000-2 of figure 20, electronic equipment 100 produces operation signal, which includes instruction
The control command of " concert videos of the first user of display (such as Jane) ".Then, electronic equipment 100 can be logical by long range
Believe to the 3rd TV 2030 of third user and sends operation signal.Electronic equipment 100 can be sent out to the 3rd TV 2030 of third user
It send the concert videos file of the first user (such as Jane) or the concert videos of the first user (such as Jane) is shown
Screen.
The 3rd TV 2030 of third user can show the music of the first user (such as Jane) according to the operation signal of reception
It can video.
According to one exemplary embodiment, electronic equipment 100 can receive image and setting icon for connecting the first TV 2010
2022 dragging input.Electronic equipment 100 can recognize that the category of attribute (such as showing content) and third user of the first TV 2010
Property (such as setting).
Between object in relating attribute table 310, electronic equipment 100 can search for " aobvious with the attribute as the first TV 2010
Show content " and attribute as third user " setting " the corresponding control information of combination (for example, control device：Third is used
3rd TV 2030 at family, operation：Reflect the setting information of the first TV 2010).Electronic equipment 100 analysis and Control information and can be controlled
Make the first TV 2010 so that the screen picture shown on the 3rd TV 2030 for being shown in third user on the first TV 2010.
Moreover, electronic equipment 100 can receive the setting of the screen picture for being shown on the first TV 2010 from second user.For example,
When screen picture includes the interface for display ratio to be arranged, electronic equipment 100 can receive for display ratio to be arranged to
90% input.
In this case, electronic equipment 100 produces operation signal, which includes instruction " by the 3rd TV
2030 display ratio is arranged to 90% " control command.Then, electronic equipment 100 can be used by long haul communication to third
3rd TV 2030 at family sends operation signal.The 3rd TV 2030 of third user can will show ratio according to the operation signal of reception
Rate is arranged to 90%.
According to one exemplary embodiment, second user can simply and remotely control feel to be difficult to commanding apparatus he or
The equipment of her parent.
Figure 21 is the method for illustrating the attribute that electronic equipment according to one exemplary embodiment shows multiple objects identified
Flow chart.
In operation S2110, electronic equipment 100 can show the attribute of multiple objects.For example, electronic equipment 100 can be in screen
At least one attribute of the upper display corresponding at least one attribute of the first object and corresponding to the second object.
According to one exemplary embodiment, electronic equipment 100 can show the attribute of multiple objects.For example, electronic equipment 100 can be shown
Show the indicator of the attribute corresponding to multiple objects.For example, indicator can be icon, mark, text etc., but not limited to this.
In operating S2120, the user that electronic equipment 100 can receive at least one of attribute for selecting display is defeated
Enter.
For example, electronic equipment 100 can receive the second attribute of the first attribute and the second object for selecting the first object
User input.Moreover, when there is an attribute corresponding to the second object, electronic equipment 100 can receive for selecting the
The user of first attribute of an object and the second object inputs.
In operating S2130, electronic equipment 100 can determine target object and operation corresponding at least one attribute.
For example, electronic equipment 100 can be based on the second attribute of the first attribute of the first object and the second object come first
The target object of control is determined between object and the second object.Alternatively, electronic equipment 100 can be based on the first object first belongs to
Property and the second object the second attribute come determine third object as control target object.
According to one exemplary embodiment, relating attribute table 310 can be used between object to determine corresponding to first in electronic equipment 100
The operation of the combination of first attribute of object and the second attribute of the second object.Then, electronic equipment 100, which produces, corresponds to really
The operation signal of fixed operation.
Figure 22 is the operation for illustrating electronic equipment according to one exemplary embodiment and showing the attribute of TV and the attribute of lamps and lanterns
Figure.
With reference to figure 22, electronic equipment 100 can be shown including TV, lamps and lanterns, audio frequency apparatus 2224, washing machine 2226 and robot
The live view of vacuum cleaner 2228.By the image procossing of the image corresponding to live view, electronic equipment 100 is recognizable
The position of TV, lamps and lanterns, audio frequency apparatus, robotic vacuum cleaner and washing machine in live view.Moreover, logical by short distance
Letter, electronic equipment 100 can recognize that the position of TV, lamps and lanterns, audio frequency apparatus, robotic vacuum cleaner and washing machine in live view
It sets.
Electronic equipment 100 can recognize that the attribute information of object, and show the indicator of the attribute of object.For example, when more
When a attribute (such as brightness, style and volume) matches with TV 2210, electronic equipment 100 can show the first brightness icon 2211,
Style icon 2212 and volume icon 2213.Moreover, when multiple attributes (such as color and brightness) are matched with lamps and lanterns 2220, electricity
100 displayable colour icon 2221 of sub- equipment and the second brightness icon 2222.
Meanwhile the dragging that electronic equipment 100 can receive for connecting style icon 2212 and the second brightness icon 2222 is defeated
Enter.Electronic equipment 100 can search for control information (for example, control device in object relating attribute table 310：Lamps and lanterns, operation：Root
Brightness is adjusted according to style), group of the control information corresponding to the attribute (style) of TV 2210 and the attribute (brightness) of lamps and lanterns 2220
It closes.
By analysis and Control information, lamps and lanterns 2220 can be determined as target object by electronic equipment 100, and checking will be by lamps and lanterns
The operation of execution is " adjusting brightness according to style ".Electronic equipment 100 can be checked currently based on the status information of TV2210
Film is played on TV 2210, and is generated and indicated for brightness to be contoured to correspond to 10% operation signal in film mode.So
Afterwards, electronic equipment 100 can send the operation signal generated to lamps and lanterns.
Figure 23 is the flow chart for illustrating the method that electronic equipment according to one exemplary embodiment recommends multiple operations.
In operating S2310, electronic equipment 100 can be recommended more based on the attribute of the attribute of the first object and the second object
A operation.
For example, when two attributes correspond to the first object and two attributes correspond to the second object, it is understood that there may be right
It should be in four operations of the combination of the attribute of the attribute and the second object of the first object.Therefore, electronic equipment 100 can provide correspondence
In the list of multiple operations of the first object selected by user and the second object.For example, electronic equipment 100 can show it is all
Two of recommendation in four operations or four operations.
According to one exemplary embodiment, including the recommendation list of the operation of recommendation can be in advance arranged by user.Moreover, electronics is set
Standby 100 can learn the use pattern of user and generate recommendation list according to the use pattern of user.For example, electronic equipment 100 can
Generation includes the recommendation list of the operation frequently used by user.It includes that user uses weekly once that electronic equipment 100, which produces,
Or the recommendation list of multiple operation.
Meanwhile when a combination operated corresponding to the attribute of the first object and the attribute of the second object, electronic equipment
100 can be immediately generated the operation signal corresponding to the operation without showing the list recommended.
In operating S2320, electronic equipment 100 can receive for selecting one user in multiple recommendation operations defeated
Enter.
Here, alterable for selecting one user in multiple operations to input.For example, for selecting in multiple operations
One user input can be touch input (such as tap, touch and keep, double-click, dragging, translating, flicking, drag and drop),
At least one of voice input, eyes input and bending input, but not limited to this.
In operating S2330, electronic equipment 100 produces the operation signal of the operation corresponding to selection.Then, electronics is set
Standby 100 can send operation signal to target object.For example, operation letter can be generated based on the control protocol information of target object
Number.
Figure 24 be illustrate attribute and lamps and lanterns of the electronic equipment based on TV according to one exemplary embodiment attribute it is more to recommend
The figure of the operation of a operation.
With reference to figure 24, electronic equipment 100 can show true including TV 2410, lamps and lanterns 2420, audio frequency apparatus 2224, robot
The live view of vacuum cleaner 2228 and washing machine 2226.Electronic equipment 100 can pass through the figure of the image corresponding to live view
As processing come identify TV 2410, lamps and lanterns 2420, audio frequency apparatus 2224, robotic vacuum cleaner 2228 and washing machine 2226 it
At least one of position in live view.Moreover, electronic equipment 100 can by short haul connection come identify TV 2410,
Lamps and lanterns 2420, audio frequency apparatus 2224, robotic vacuum cleaner 2228 and washing machine 2226 at least one of taken in real time
Position in scape.
When user wants to watch the content played on TV 2410 in dark surrounds, electronic equipment 100 can pass through reality
When find a view reception for select the user of TV 2410 and lamps and lanterns 2420 input.It touches and drags for example, electronic equipment 100 can receive
The dragging of dynamic TV 2410 to lamps and lanterns 2420 inputs.
Electronic equipment 100 can recognize that the attribute of the attribute and lamps and lanterns 2420 of TV 2410.Then, electronic equipment 100 can be right
As search corresponds to the control information of the combination of the attribute of TV and the attribute of lamps and lanterns in relating attribute table 310.
However, due to multiple attributes (such as brightness, style and volume) matched with TV and multiple attributes (such as color and
Brightness) it is matched with lamps and lanterns, therefore a plurality of control information can be searched in relating attribute table 310 between object.
Therefore, electronic equipment 100 can be shown among the operation that a plurality of control information includes recommends operating list 2430.
For example, recommend operating list 2430 may include the first operation (such as according to drama adjust illumination brightness), second operation (such as
According to the briliancy of the brightness regulation TV 2410 of illumination) and third operation (such as adjusted and illuminated according to the briliancy of TV 2410), it should
First operation corresponds to the combination of " brightness " of " style " of the attribute as TV 2410 and the attribute as lamps and lanterns 2420, should
Second operation corresponds to the combination of " brightness " of the attribute as TV 2410 and " brightness " of the attribute as lamps and lanterns 2420, should
Third operation corresponds to the combination of " brightness " of the attribute as TV 2410 and " brightness " of the attribute as lamps and lanterns 2420.
When user selects to recommend the first operation in operating list 2430, electronic equipment 100 can be sent to lamps and lanterns 2420
The operation signal of the control command of brightness degree is adjusted including instruction.
Figure 25 is the figure for illustrating application execution system according to one exemplary embodiment.
With reference to figure 25, application execution system 2500 according to one exemplary embodiment may include electronic equipment 100 and multiple right
As (such as lamps and lanterns 2501, TV 2502, robotic vacuum cleaner 2503, air-conditioning 2504 etc.).Moreover, according to exemplary implementation
Example, other than electronic equipment 100 and multiple objects, application execution system may also include server (not shown).
Electronic equipment 100 can be can perform and the equipment of each related application in multiple objects.Using can table
Show one group of computer program for being designed to execute particular task.For example, using that can be that schedule management application, address book are answered
With, video player application, map application, broadcasted application, temper management application, payment application, application of giving birth to children, health care answers
With, e-book application etc., but not limited to this.
Meanwhile according to one exemplary embodiment, using the controller application that may include for control object.For the ease of retouching
It states, controller application will be known as " controller " below.
According to one exemplary embodiment, electronic equipment 100 can execute application related with special object under user request.
However, since there are various applications in electronic equipment 100, it is difficult to for a user quickly in electronic equipment 100
Execute application related with special object.For example, user may wish to execute the first application of the illumination for changing lamps and lanterns 2501.So
And when there are many in application, user is likely difficult to be quickly found out the first application for installation on electronic equipment 100.
Therefore, electronic equipment 100 explained below by using object recognition model information fast search or execute with it is right
Convenient method is provided a user as related application.
In an exemplary embodiment, object recognition model information can be indicated about for distinguishing each in multiple objects
And execute the information for the model for corresponding to the function of each in multiple objects (such as apply or controller).For discrimination pair
The model (it will be referred to as " object recognition model " below) of elephant may include the image of object for identification.For example, for distinguishing
First object recognition model of the first object (such as TV) may include the first image (such as TV images) corresponding to the first object,
And for distinguishing that the second object recognition model of the second object (such as refrigerator) may include the second figure corresponding to the second object
As (such as refrigerator image).
Meanwhile according to one exemplary embodiment, object recognition model information may include every in multiple objects about being linked to
The information of one application (or controller).For example, object recognition model information may include about be linked to third object (such as
Loud speaker) third application (such as music player application) information.
According to one exemplary embodiment, object recognition model information can be generated by electronic equipment 100 or server.It below will ginseng
The operation of the object recognition model information of general object is generated according to Figure 26 detailed description servers.
Figure 26 is the sequence chart for illustrating the method that server according to one exemplary embodiment generates object recognition model information.
In operating S2610, server 2600 can determine multiple classifications and multiple keywords.
For example, with reference to figure 27, product grouping (such as TV, air-conditioning, vacuum cleaner etc.) can be determined as by server 2600
Classification 2710.Moreover, server 2600 will can wherein use place (such as living room, kitchen, office etc.), the product of product
Manufacturer title, using the form (such as stand-type, wall-mounted and mobile) etc. of product be determined as keyword 2720, but
It is that keyword 2720 is without being limited thereto.
In operating S2620, server 2600 can search for the image for corresponding to the object of multiple classifications and multiple keywords.
For example, when classification 2710 is " TV " and keyword 2720 is " living room ", server 2600 can stand in search
Image of the search corresponding to living room TV in point.Moreover, when classification 2710 is " TV " and keyword 2720 is " office ",
Server 2600 can search for the image corresponding to the TV in office in search site.
According to one exemplary embodiment, server 2600 can search for application or the controller corresponding to classification and keyword.Example
Such as, when classification 2710 is " TV " and keyword 2720 is " living room ", server 2600 can be searched in website to be corresponded to
The controller of living room TV.Moreover, when classification 2710 is " water bottle " and keyword 2720 is " intelligence ", server 2600 can
The application of intelligent water bottle is searched in website.
According to one exemplary embodiment, server 2600 can generate file and the image found of classifying according to classification.Showing
In exemplary embodiment, file can be for showing the use with some basic (classification) related content (such as image) together
Family interface.
In operating S2630, the image of the object found can be used to generate object recognition model information in server 2600.
According to one exemplary embodiment, in order to increase object recognition rate, depth learning technology next life can be used in server 2600
At the training image of the image corresponding to the object found.Deep learning is the branch of machine learning, and by simulating the mankind
Brain makes it possible to Complex Modeling.Machine learning be enable a computer to be trained with data and understood as people target or
The artificial intelligence technology of situation.
For example, the angle of the changeable image found of server 2600 is to generate the training image for distinguishing object.Ginseng
Figure 28 is examined, when living room TV images 2800 are acquired, server 2600 is produced by by 2800 inverse time of living room TV images
Needle rotates 30 degree and the first training image 2801 of acquisition, is obtained by living room TV images 2800 are rotated clockwise 5 degree
The second training image 2802, the third training image that is obtained by living room TV images 2800 are rotated clockwise 20 degree
2803, the 4th training image 2804 etc. by obtaining the symmetrical left and right transformation of living room TV images 2800.
According to one exemplary embodiment, server 2600 produces multiple object recognition models, multiple object recognition model packets
Include the training image of the image of the object found and the image corresponding to the object found.For example, server 2600 produces
Including original living room TV images 2800, the first training image 2801, the second training image 2802,2803 and of third training image
The object recognition model of the living room TV of 4th training image 2804.
According to one exemplary embodiment, server 2600 can be by by each and the application in multiple object recognition models
(or controller) information is mutually matched to generate object recognition model information.For example, the object recognition generated by server 2600
Model information may include：By the way that the object recognition model of living room TV is matched with the controller information for controlling living room TV
And the first match information, the object recognition model by matching refrigerator and the refrigerator obtained manages second obtained by application message
Match information etc..
In operation S2640, server 2600 can distinguish model information to 100 sending object of electronic equipment.
According to one exemplary embodiment, server 2600 can be sent out when receiving request from electronic equipment 100 to electronic equipment 100
Object recognition model information is sent, or can distinguish model information to 100 sending object of electronic equipment at a regular interval.And
And when particular event occurs, server 2600 can distinguish model information to 100 sending object of electronic equipment.For example, working as object
When distinguishing that model information is updated, server 2600 can send newer object recognition model information from trend electronic equipment 100.
In operation S2650, electronic equipment 100 can store the object recognition model information received from server 2600.
According to one exemplary embodiment, electronic equipment 100 can in electronic equipment 100 memory or electronic equipment 100 outside
Storage object identification model information in the storage device in portion.
According to one exemplary embodiment, when obtaining the image of special object by imaging sensor, electronic equipment 100 can
Execute the application corresponding to special object automatically using the process identification model information of storage.It is described in detail hereinafter with reference to Figure 37
Electronic equipment 100 executes the operation of application using process identification model information.
Figure 26 will be illustrated as example the case where the wherein execution of server 2600 operation S2610 extremely operation S2640, but demonstrate
Property embodiment is unrestricted.According to one exemplary embodiment, the executable operation S2610 of electronic equipment 100 is to operating S2630.
Moreover, according to one exemplary embodiment, some operations among operation S2610 to operation S2650 can be omitted.For example,
When determining multiple classifications and multiple keywords in advance, operation S2610 can be omitted.Moreover, when electronic equipment 100 executes operation
When S2610 to operation S2630, operation S2640 can be omitted.
Meanwhile when the general pattern of 2600 object search of server and when generating process identification model information, it is included in pair
As the image in identification model information may differ from the image of the object actually used by user.In this case, in electronics
It may generating object identification mistake in equipment 100.Therefore, server 2600 explained below change process identification model information with
The method of calibration object identification mistake.
Figure 29 is illustrated changes process identification model information according to one exemplary embodiment when there are process identification mistake
The sequence chart of method.
In operating S2910, electronic equipment 100 can obtain the image of the first object by imaging sensor.
For example, electronic equipment 100 can activate imaging sensor and using the imaging sensor of activation capture (shooting) in electricity
The image of the first object outside sub- equipment 100, thus obtains the image of the first object.
In operating S2920, image and the process identification model information of the first object can be used to identify for electronic equipment 100
First object.
For example, electronic equipment 100 by the image of the first object of imaging sensor acquisition and can be included in by comparing
Image in process identification model information identifies the first object.As an example, as the first couple obtained by imaging sensor
When the image of elephant is similar to the second image being included in process identification model information and the second image is the image of TV, electronics
First Object identifying can be TV by equipment 100.
In operating S2930, electronic equipment 100 can show the identification information of the first object.For example, the identification of the first object
Information may include the title of the first object, the icon of the first object, the first object trade mark etc., but be included in identification information
Information it is without being limited thereto.
For example, when the first object is identified as TV, electronic equipment 100 can show the pop-up for including TV icons on the screen
Window.
In operation S2940, electronic equipment 100 can receive the request of the identification information to changing the first object.
For example, when the first Object identifying be refrigerator but electronic equipment 100 is TV by the first object, electronic equipment 100 can
Receive the request of the modification to the identification information of the first object.Electronic equipment 100 can receive the title etc. of the first object from user.
In operating S2950, when the request that there is the identification information to changing the first object, electronic equipment 100 can ask
Server 2600 is asked to change process identification model information.
For example, when the request that there is the identification information to changing the first object, the process identification model of the first object is
Falsely generated, and thus electronic equipment 100 can to server 2600 send to change the first object process identification mould
The request of type.Electronic equipment 100 can be sent to server 2600 by the identification information of imaging sensor and the first object (such as
The title etc. of first object) obtain the first object image.
In operating S2960, server 2600 can change process identification model information.
According to one exemplary embodiment, the image of the first object can be used to generate corresponding to the first object in server 2600
Training image.Then, newly-generated training image can be by being added to the process identification model of the first object by server 2600
Carry out upgating object identification model.Moreover, in process identification model information, server 2600 can be stored by by the first object
Process identification model is matched with the identification information of the first object and the information that obtains.
Meanwhile when not having to define the process identification model of the first object in process identification model information, server
2600 can change process identification model by newly defining the process identification model of the first object in process identification model information
Information.
According to one exemplary embodiment, when the modification of process identification model information is completed, server 2600 can be set to electronics
Standby 100 send modified process identification model information.In this case, the process identification rate of electronic equipment 100 can increase.
Figure 30 be illustrate wherein in the electronic device generating object distinguish mistake situation figure.
With reference to the first screen 3010 of figure 30, electronic equipment 100 can receive the input for touching catching press-button 3001.This
In the case of, imaging sensor can be used to shoot external object 3000 for electronic equipment 100.For example, when external equipment 3000 is TV,
Electronic equipment 100 can obtain the image including TV.
With reference to the second screen 3020 of figure 30, electronic equipment 100 can by by the image of acquisition with include in process identification
Image in model information relatively identifies external equipment 3000, and shows the identification information of external equipment 3000.For example, electronics
External equipment 3000 can be determined as air-conditioning and show the icon and text of air-conditioning by equipment 100.However, the external equipment of shooting
3000 be actually not air-conditioning but TV, thus the tangible upload button for asking modification process identification model information of user
3002。
With reference to the third screen 3030 of figure 30, electronic equipment 100 can be shown for inquiring that user has responded in touch
It passes the input of button 3002 and has taken the frame 3003 of which product.Electronic equipment 100 can receive instruction external equipment 3000
The input of TV, and request is repaiied while sending title (that is, " TV ") of image and external equipment 3000 to server 2600
Change process identification model information.
According to one exemplary embodiment, when shooting outside with electronic equipment 100 after process identification model information is changed
When equipment 3000, the process identification model information of modification can be used to be correctly to determine external equipment 3000 for electronic equipment 100
TV。
Meanwhile other than the process identification model for general object generated by server 2600, electronic equipment 100
Or server 2600 can also be according to the personalized process identification model of user requested to generate for special object.It below will be detailed
Thin description electronic equipment 100 or server 2600 generate the operation of personalized process identification model.
Figure 31 is the sequence for the method for illustrating the personalized object recognition model information of generation according to one exemplary embodiment
Figure.
In operating S3110, electronic equipment 100 can obtain the image of the first object by imaging sensor.First object
Image can be static image or video.When the image of the first object is video, the image of the first object may include multiple
Video frame.Since operation S3110 corresponds to the operation S2910 of Figure 29, detailed description thereof will not be repeated.
According to one exemplary embodiment, electronic equipment 100 can receive the identification information of the first object from user.For example, electronics
Equipment 100 can receive product grouping, purposes, name of product, the icon of the first object etc. of shooting.
In operating S3120, electronic equipment 100 can receive the input of selection application (or controller).For example, when first pair
When the image of elephant is captured, what electronic equipment 100 can receive that the selection user of user wishes to carry out related with the first object answers
Input.
According to one exemplary embodiment, electronic equipment 100 can be shown mounted in the list of application thereon, and receive and answering
With the input for selecting at least one application in list.
According to one exemplary embodiment, electronic equipment 100 can identify the first object by analyzing the image of the first object, and
And recommend at least one application for corresponding to the first object.Electronic equipment 100 can receive the input for the application that selection is recommended.
According to one exemplary embodiment, electronic equipment 100 can receive the input of search specific application.In this case, electric
Sub- equipment 100 can search for specific application in application shop and install specific application.
In operating S3130, electronic equipment 100 can send the image of the first object to server 2600 and about application
The information of (or controller).For example, the information about application (or controller) may include the title applied, version information, exploitation
Person's information etc., but not limited to this.
For example, the image that the first object obtained by imaging sensor is sent to server 2600 and about by with
While the information of the specific application (or controller) of family selection, the object that electronic equipment 100 can request to generate the first object is distinguished
Know model.According to one exemplary embodiment, electronic equipment 100 can also send the identification information (example of the first object to server 2600
Such as the title of the first object).
In operating S3140, server 2600 produces the process identification model of the first object.For example, server 2600
Produce the training image for the image for corresponding to the first object received from electronic equipment 100.Due to being described with reference to Figure 28
Server 2600 generates the operation of training image, therefore detailed description thereof will not be repeated.It includes that server 2600, which produces,
The image of an object and corresponding to the first object image training image the first object process identification model.
According to one exemplary embodiment, server 2600 produce by by the object recognition model of the first object with about by
The information matches of the application (or controller) of user's selection and the match information of the first object obtained.
In operating S3150, server 2600 can be used the process identification model of the first object and is previously generated to update
Process identification model information.For example, server 2600 can believe the matching of the process identification model of the first object and the first object
Breath is newly added to the process identification model information being previously generated.
According to one exemplary embodiment, when there is no the process identification model information being previously generated, process identification model letter
The update of breath may include newly-generated process identification model information.
In operating S3160, server 2600 can send newer process identification model information to electronic equipment 100.
According to one exemplary embodiment, server 2600 separably sends the personalization of special object selected by user
The general process identification model information of process identification model information and general object.Moreover, server 2600 can will be personalized
Process identification model information and general process identification model information it is integrated, and using integrated information as a file to electronics
Equipment 100 is sent.Meanwhile according to one exemplary embodiment, server 2600 can only send to electronic equipment 100 and be selected by user
Special object personalized process identification model information.
In operating S3170, electronic equipment 100 can be based on the process identification model information received from server 2600 come more
New previously stored process identification model information.
For example, electronic equipment 100 can connect the version of previously stored process identification model information with from server 2600
The edition comparison of the process identification model information of receipts, and be than elder generation in the process identification model information received from server 2600
Changed using the process identification model information received from server 2600 when the version in the process identification model information evening of preceding storage
Become previously stored process identification model information.
The case where wherein server 2600 generates the process identification model of the first object is illustrated by way of example in Figure 31,
But exemplary embodiment is unrestricted.For example, electronic equipment 100 oneself can generate the process identification model of the first object.
The object that the personalization for generating special object selected by user is further described hereinafter with reference to Figure 32 to 34 is distinguished
Know the operation of model information.
Figure 32 is the figure for the operation for illustrating linked object and application according to one exemplary embodiment.As an example, Figure 32 is logical
Cross the case where way of example diagram wherein uses 2600 new registration temperature controller of user orientation server.
With reference to the first screen 3210 of figure 32, it is outer to shoot that electronic equipment 100 may be in response to the input of touch catching press-button
Portion's object 3200.For example, electronic equipment 100 can include the image of external object 3200 by imaging sensor acquisition.
According to one exemplary embodiment, electronic equipment 100 can be identified by analyzing the image obtained by imaging sensor
External object 3200.For example, electronic equipment 100 can be included within the external object 3200 in the image of acquisition characteristic point (or
Feature vector) compared with the characteristic point (or feature vector) for the object being included in previously stored image, it thus will be external right
Temperature controller is identified as 3200.
According to one exemplary embodiment, electronic equipment 100 can receive the identification information of external object 3200 from user.For example,
The external object 3200 that electronic equipment 100 can receive instruction shooting is the input of temperature controller.
With reference to the second screen 3220 of figure 32, electronic equipment 100 can be provided to be wanted to be linked to temperature for receiving selection user
Spend the GUI input by user of the application of controller.
According to one exemplary embodiment, electronic equipment 100 can provide the recommendation window 3201 for showing the application recommended.Example
Such as, electronic equipment 100 can be shown in recommending window 3201 by the application " OOO " of manufacturer's offer of temperature controller.
According to one exemplary embodiment, electronic equipment 100 can be provided wants to be linked to temperature control for inputting user
The input window 3202 of the address (such as uniform resource locator (URL)) of the website of device.Moreover, according to one exemplary embodiment,
Electronic equipment 100 can show the list 3203 for the application being previously installed on electronic equipment 100.
As application " OOO " that user's selection is provided by the manufacturer of temperature controller, temperature can be used in electronic equipment 100
The image of controller, the identification information of temperature controller and the information about application " OOO " generate the object of temperature controller
Identification model information.
For example, electronic equipment 100 can be in the identification for the image, temperature controller for sending temperature controller to server 2600
Information and about application " OOO " information while request to generate the process identification model information of temperature controller.In this feelings
Under condition, the image of temperature controller can be used to generate training image for server 2600.Moreover, server 2600 can be by will be warm
Identification information, training image and the information about application " OOO " for spending controller are mutually matched to generate pair of temperature controller
As identification model information.
When the generation of the process identification model information of temperature controller is completed by server 2600, electronic equipment 100 can
The process identification model information of temperature controller is received from server 2600.
With reference to the third screen 3230 of figure 32, when the generation of the process identification model information of temperature controller is completed, electricity
Sub- equipment 100 can show the completion screen 3204 (such as " temperature controller has been registered ") of display message.
With reference to the 4th screen 2340 of figure 32, electronic equipment 100 can be provided wherein uses external object 3200 for being arranged
Place setting window 3205.Electronic equipment 100 can wherein use external object by the way that the reception setting of window 3205 is arranged
The input in 3200 place.For example, when temperature in use controller at home, electronic equipment 100 can be by being arranged window 3205
Receive the input of selection " family ".
Location information can be added to the process identification model information of temperature controller by electronic equipment 100.In such case
Under, although obtaining the image of same object by imaging sensor, electronic equipment 100 can be according to where electronic equipment 100
Position execute different applications.For example, electronic equipment 100 can be executed when the image of temperature controller is captured at home
Using " OOO ", and application " XXX " can be executed when the image of temperature controller is captured in an office.
Figure 33 A are the figures for the operation for illustrating the video that electronic equipment according to one exemplary embodiment obtains object.Figure 33 B are
Illustrate the figure that electronic equipment distinguishes the operation of model information from server downloaded object.Figure 33 A and Figure 33 B pass through way of example figure
Show the case where wherein using 2600 new registration loud speaker of user orientation server.
With reference to the first screen 3310 of figure 33A, in order to increase process identification rate, electronic equipment 100 can be shown for inducing
User captures the video of external object 3300 (such as loud speaker) rather than the message of static image is (for example, " centered on circle
Target captures video while drawing round ").Electronic equipment 100 can receive the input for touching video catching press-button 3301.
With reference to the second screen 3320 of figure 33A, electronic equipment 100 may be in response to the input of selection video catching press-button 3301
To capture the video of external object 3300.According to one exemplary embodiment, electronic equipment 100 can show that instruction video is just captured
The indicator 3302 of (such as video capture is being carried out or completed), such as bar shaped.
With reference to the third screen 3330 of figure 33A, when the capture of the video of external object 3300 is completed, electronic equipment 100
It can show the input window 3303 of the identification information for inputting external object 3300.
For example, electronic equipment 100 can show the list (such as TV, air-conditioning, loud speaker etc.) of predefined classification.When user exists
When selecting loud speaker in input window 3303, it is loud speaker that electronic equipment 100, which can recognize external object 3300,.
With reference to the 4th screen 3340 of figure 33A, electronic equipment 100 can provide the function of may link to external object 3300
List 3304.It is shown including operating function, application execution function, memo function, photo for example, electronic equipment 100 can be shown
The list 3304 of function etc..When user selects application execution function, electronic equipment 100 can be provided to be raised for inputting to correspond to
The GUI of the specific application of sound device.
In the identification information of video, loud speaker to the transmission loud speaker of server 2600 and about the spy selected by GUI
Surely while the information applied, electronic equipment 100 can request to generate the process identification model information of loud speaker.
With reference to the 5th screen 3350 of figure 33B, when the video of loud speaker is fully transmitted server 2600, electronics is set
Standby 100 can show that instruction uploads the indicator 3305 completed.
With reference to the 6th screen 3360 of figure 33B, electronic equipment 100 can show that instruction server 2600 is generating loud speaker
Process identification model information message.
The video of loud speaker can be used to generate training image for server 2600.Since the video of loud speaker includes multiple videos
Frame, therefore multiple video frame can be used to generate various training images in server 2600.Moreover, server 2600 can be by that will raise
Identification information, training image and the information about specific application of sound device match each other to generate the process identification model of loud speaker
Information.
With reference to the 7th screen 3370 of figure 33B, when the generation of the process identification model information of loud speaker is completed, electronics is set
Standby 100 can download the process identification model information of loud speaker from server 2600 automatically, as shown in bar shaped 3372.
Figure 34 is the figure for the operation for illustrating linked object and controller according to one exemplary embodiment.Figure 34 passes through exemplary
Mode illustrates the case where wherein using 2600 new registration lamp of user orientation server.
With reference to the first screen 3410 of figure 34, it is outer to shoot that electronic equipment 100 may be in response to the input of touch catching press-button
Portion's object 3400.For example, electronic equipment 100 can include the figure of external object 3400 (such as lamp) by imaging sensor acquisition
Picture.
According to one exemplary embodiment, electronic equipment 100 can be identified by analyzing the image obtained by imaging sensor
External object 3400.For example, electronic equipment 100 can be included within the external object 3400 in the image of acquisition characteristic point (or
Feature vector) compared with the characteristic point (or feature vector) for the object being included in previously stored image, it thus will be external right
Lamp is identified as 3400.
According to one exemplary embodiment, electronic equipment 100 can receive the identification information of external object 3400 from user.For example,
The external object 3400 that electronic equipment 100 can receive instruction shooting is the input of amber light.
With reference to the second screen 3420 of figure 34, electronic equipment 100 can receive the input that lamp is linked to controller 3401.Example
Such as, controller 3401 may include the color for adjusting lamp and/or the interface of brightness.Meanwhile controller 3401 may via
The manufacturer of lamp develops.
While sending the image of lamp, the identification information of lamp to server 2600 and about the information of controller 3401,
Electronic equipment 100 can request to generate the process identification model information of lamp.In this case, the figure of lamp can be used in server 2600
As generating training image.Moreover, server 2600 can be by by the identification information of lamp, training image and about controller 3401
Information is mutually matched to generate the process identification model information of lamp.
When the generation of the process identification model information of lamp is completed by server 2600, electronic equipment 100 can be from server
2600 receive the process identification model information of lamp.
With reference to the third screen 3430 of figure 34, when electronic equipment 100 is logical after storing the process identification model information of lamp
When crossing the image of imaging sensor acquisition lamp, the process identification model information of lamp can be used to execute control automatically for electronic equipment 100
Device 3401.Execute the operation of controller 3401 automatically hereinafter with reference to Figure 37 detailed descriptions electronic equipment 100.
Meanwhile user may wish to change application or the controller for being linked to special object.It is detailed hereinafter with reference to Figure 35
Description electronic equipment 100 is linked to the application of specific application or the operation of controller according to the request of user change.
Figure 35 is the flow chart for illustrating the method that upgating object according to one exemplary embodiment distinguishes model information.
In operating S3510, electronic equipment 100, which can receive, changes the defeated of the application or controller for being linked to the first object
Enter.
For example, electronic equipment 100 can receive the input that the first application for being linked to the first object is changed to the second application.
Alternatively, electronic equipment 100 can receive the input that the first application for being linked to the first object is changed to the first controller.
In operating S3520, electronic equipment 100 may be in response to change the defeated of the application or controller for being linked to the first object
Enter upgating object identification model information.
According to one exemplary embodiment, when receive will be linked to the first object first application be changed to the defeated of the second application
It is fashionable, electronic equipment 100 can request server 2600 change the first object process identification model information.
Server 2600 can by by the process identification model information of the first object with second application rather than first application
It matches to change the process identification model information of the first object.
Electronic equipment 100 can receive the process identification model information of the first object of modification, and base from server 2600
Previously stored process identification model information is updated in the process identification model information of the first object of modification.
Meanwhile according to one exemplary embodiment, electronic equipment 100 can pass through the process identification model of oneself the first object of modification
Information updates previously stored process identification model information.
The application of asking change be linked to special object of the electronic equipment 100 according to user will be further described with reference to Figure 36
Or the operation of controller.
Figure 36 is that the electronic equipment illustrated according to one exemplary embodiment changes process identification model letter under user request
The figure of the operation of breath.
With reference to the first screen 3610 of figure 36, electronic equipment 100 can provide its object that process identification model has been generated
List.For example, the identification information (such as TV, loud speaker, photo etc.) of object can be shown in lists and about being linked to pair
The information of the application of each (or controller) as in.
Electronic equipment 100 can receive the input for selecting TV 3601 in lists.For example, TV 3601 may link to control
Device.
With reference to the second screen 3620 of figure 36, electronic equipment 100 can receive coupling between editor TV 3601 and controller
(linkage) request.For example, electronic equipment 100 can receive the input for touching connection Edit button 3602.
With reference to the third screen 3630 of figure 36, electronic equipment 100 may be in response to touch the input of connection Edit button 3602
To provide the list for the function of may link to TV 3601.User can select application execution function 3603 in lists.
With reference to the 4th screen 3640 of figure 36, when user selects application execution function 3603, electronic equipment 100 can provide
GUI for inputting the specific application for corresponding to TV 3601.Electronic equipment 100 can provide the list for the application recommended by GUI
3604 and default application list 3605.
To server 2600 send TV 3601 identification information and about the information of the specific application selected by GUI
While, electronic equipment 100 can ask the process identification model information of modification TV 3601.For example, server 2600 can pass through by
The identification information of TV 3601 and specific application rather than the information matches of controller change the process identification model of TV 3601
Information.
Figure 37 is the flow chart for illustrating the method that electronic equipment according to one exemplary embodiment executes application or controller.
In operating S3710, electronic equipment 100 can obtain the image of the first object by imaging sensor.
For example, electronic equipment 100 can activate imaging sensor and using the imaging sensor of activation capture (shooting) in electricity
The image of the first object outside sub- equipment 100, thus obtains the image of the first object.
In operation S3720, object recognition model information can be used to identify the first object in electronic equipment 100.
For example, electronic equipment 100 by the image of the first object of imaging sensor acquisition and can be included in by comparing
Image in object recognition model information identifies the first object.As an example, as the first couple obtained by imaging sensor
When the image of elephant is similar to the second image being included in object recognition model information and the second image is the image of loud speaker,
First Object identifying can be loud speaker by electronic equipment 100.
In operating S3730, the executable application or controller corresponding to the first object of electronic equipment 100.
According to one exemplary embodiment, electronic equipment 100 can check the match information being included in object recognition model information
And execute the specific application (or controller) with the object recognition Model Matching of the first object.For example, when the first object is to raise
When sound device and loud speaker and music player application are matched with object recognition model information, electronic equipment 100 can perform music
Player application.
Figure 38 is the figure for illustrating the operation that electronic equipment according to one exemplary embodiment shows the controller for corresponding to TV.
With reference to the first screen 3810 of figure 38, electronic equipment 100 can be right by the outside that will be obtained by imaging sensor
As 3800 image and the image that is included in process identification model information relatively identify external object 3800.For example, when outer
The degree of similarity between the image of portion's object 3800 and the image for the TV being included in process identification model information is more than threshold
When being worth (such as about 95%), external object 3800 can be recognized as TV by electronic equipment 100.
With reference to the second screen 3820 of figure 38, when TV and controller 3801 with pair that is stored in electronic equipment 100
When as identification model information matches, electronic equipment 100 can display controller 3801, such as controller GUI on the screen.
Therefore, according to one exemplary embodiment, user can by simply capture the image of the object in electronic equipment 100 come
Quickly execute application related with object.
Meanwhile according to one exemplary embodiment, other than application and/or controller, user can also link special object
To memorandum and/or website.Electronic equipment 100 described in detail below according to the hyperlink request object and memorandum of user (or
Station address) operation.
Figure 39 be illustrate according to one exemplary embodiment by link memorandum or station address and object generate to as
The sequence chart of the method for identification model information.
In operating S3910, electronic equipment 100 can obtain the image of the first object by imaging sensor.First object
Image can be static image or video.When the image of the first object is video, the image of the first object may include multiple
Video frame.Since operation S3910 corresponds to the operation S2910 of Figure 29, detailed description thereof will not be repeated.
According to one exemplary embodiment, electronic equipment 100 can receive the identification information of the first object from user.For example, electronics
Equipment 100 can receive product grouping, purposes, name of product, the icon of the first object etc. of shooting.
In operating S3920, electronic equipment 100 can receive memorandum or station address.
For example, when the image of the first object is captured, electronic equipment 100 can receive storage user think it is to be shown with the
The input of the related memorandum of an object.Alternatively, when the image of the first object is captured, electronic equipment 100 can receive fixed
Adopted user wants the input of the station address related with the first object of connection.
In operating S3930, electronic equipment 100 can send the image of the first object to server 2600 and about memorandum
The information of (or station address).
For example, to server 2600 send obtained by imaging sensor the first object image, about by user
Identification information (such as the title of the first object, the product of the information of the memorandum (or station address) of input and the first object
Grouping etc.) while, electronic equipment 100 can request to generate the process identification model information of the first object.
In operating S3940, server 2600 can generate the first object according to the request received from electronic equipment 100
Process identification model information.
For example, server 2600 produces the training figure for the image for corresponding to the first object received from electronic equipment 100
Picture.Due to describing the operation that server 2600 generates training image with reference to Figure 28, detailed description thereof will not be repeated.
Server 2600 produce include the first object image and corresponding to the first object image training image the first object
Process identification model.Moreover, server 2600 produces the process identification model information of the first object, the process identification model
Information includes being obtained by matching the process identification model of the first object with memorandum input by user (or station address)
The match information obtained.
When the generation of the process identification model information of the first object is completed by server 2600, electronic equipment 100 can be from
Server 2600 receives the process identification model information of the first object.
Figure 40 is the figure for the operation for illustrating link credit card and memorandum according to one exemplary embodiment.
With reference to the first screen 4010 of figure 40, it is outer to shoot that electronic equipment 100 may be in response to the input of touch catching press-button
Portion's object 4000.For example, electronic equipment 100 can include the image of credit card by imaging sensor acquisition.
According to one exemplary embodiment, electronic equipment 100 can be identified by analyzing the image obtained by imaging sensor
External object 4000.For example, external object 4000 can be identified as credit by electronic equipment 100 by analyzing the image obtained
Card, and recognize the text marked on the credit card or number.
According to one exemplary embodiment, electronic equipment 100 can receive the identification information of external object 4000 from user.For example,
The external object 4000 that electronic equipment 100 can receive instruction shooting is the input of credit card.
According to one exemplary embodiment, when external object 4000 is identified, electronic equipment 100, which can provide, may link to outside
The list 4001 of the function of portion's object 4000.For example, electronic equipment 100 can be shown including memorandum registering functional 4001, photo
Connection, the list using connection etc..Electronic equipment 100 can receive selects memorandum registering functional in lists
4001 input.
With reference to the second screen 4020 of figure 40, when user selects memorandum registering functional 4001, electronic equipment 100 can be shown
Show memorandum input window 4002.It includes credit card that electronic equipment 100, which can be received by memorandum input window 4002 from user,
Disbursement account information memorandum.
The same of the image of credit card, the identification information of credit card and information about memorandum is being sent to server 2600
When, electronic equipment 100 can request to generate the process identification model information of credit card.In this case, server 2600 can make
Training image is generated with the image of credit card.Moreover, server 2600 can by by the identification information of credit card, training image and
Memorandum matches each other to generate the process identification model information of credit card.
With reference to the third screen 4030 of figure 40, when the generation of the process identification model information of credit card is complete by server 2600
Cheng Shi, electronic equipment 100 can receive the process identification model information of credit card from server 2600.Moreover, electronic equipment 100 can
Message 4003 (such as " memorandum has been registered ") is completed in display.
When electronic equipment 100 obtains credit after storing the process identification model information of credit card by imaging sensor
When the image of card, electronic equipment 100 can be used the process identification model information of credit card shows to include credit automatically on the screen
The memorandum of the payment account information of card.
Figure 41 is the figure for the operation for illustrating link window and station address according to one exemplary embodiment.
With reference to the first screen 4110 of figure 41, electronic equipment 100 may be in response to touch the input of catching press-button 4101 to clap
Take the photograph external object 4100.For example, electronic equipment 100 can include the image of window by imaging sensor acquisition.
According to one exemplary embodiment, electronic equipment 100 can be identified by analyzing the image obtained by imaging sensor
External object 4100.For example, external object 4100 can be identified as window by electronic equipment 100 by analyzing the image obtained.
According to one exemplary embodiment, electronic equipment 100 can receive the identification information of external object 4100 from user.For example,
The external object 4100 that electronic equipment 100 can receive instruction shooting is the input of window.
With reference to the second screen 4120 of figure 41, position sensor (such as global positioning system can be used in electronic equipment 100
(GPS)) current location information 4102 is shown.Moreover, when the image of window is acquired, electronic equipment 100 can receive user
It is intended to the address information (such as URL) 4103 of the website of connection.For example, electronic equipment 100 can receive the net for providing Weather information
Address information (such as " the http to stand：//OOOO.weather.com”).
To server 2600 send the image of window, window identification information, provide Weather information website address
Whiles information 4103, location information 4102 etc., electronic equipment 100 can request to generate the process identification model information of window.
In this case, the image of window can be used to generate training image for server 2600.Moreover, server 2600 can be by by window
The identification information of mouth, the address information 4103 and location information 4102 mutual of the website of training image, offer Weather information
Match to generate the process identification model information of window.
With reference to the third screen 4130 of figure 41, when the generation of the process identification model information of window is completed by server 2600
When, electronic equipment 100 can receive the process identification model information of window from server 2600.Moreover, electronic equipment 100 can be shown
Complete message 4104 (such as " information has been registered ").
Figure 42 is the operation for illustrating the Weather information that electronic equipment according to one exemplary embodiment is shown corresponding to window
Figure.
With reference to the first screen 4210 of figure 42, electronic equipment 100 may be in response to touch the input of catching press-button 4201 and lead to
Cross imaging sensor shooting external object 4200.Electronic equipment 100 can be by the external object that will be obtained by imaging sensor
4200 image relatively identifies external object 4200 with the image being included in process identification model information.For example, when outside
The degree of similarity between the image of object 4200 and the image for the window being included in process identification model information is more than threshold
When being worth (such as about 95%), external object 4200 can be recognized as window by electronic equipment 100.
With reference to the second screen 4220 of figure 42, when the website that Weather information is provided window and address information and be stored in electricity
In sub- equipment 100 process identification model information matching when, electronic equipment 100 may be connected to provide Weather information website and
Weather information 4202 is shown on screen.Therefore, user quickly can check Weather information by simply capturing the image of window.
Figure 43 is the block diagram for the configuration for illustrating electronic equipment according to one exemplary embodiment.
With reference to figure 43, electronic equipment 100 may include export transmitter 110, communication interface 120, user interface 130, audio/
Video (A/V) input sink 140, memory 150 (such as storage device), sensor 160 and processor 170 are (such as micro-
Processor).However, the component of all diagrams is not necessary component.Electronic equipment 100 can by more more than illustrated components or
Less component is realized.For example, electronic equipment 100 can be realized by processor 170 and communication interface 120, or by handling
Device 170, communication interface 120 and output transmitter 110 realize, but not limited to this.
Output transmitter 110 be intended to exports audio signal, vision signal or vibration signal, and may include display 111,
Sound exports transmitter 112, vibrating motor 113 etc..
Display 111 can show the information handled by electronic equipment 100.For example, display 111 can be shown by electronic equipment
The 100 multiple objects picked out.
When display 111 and touch pads constitute layered structure and are configured as touch screen, display 111 can be used as inputting
Equipment and output equipment.Display 111 may include liquid crystal display (LCD), thin film transistor (TFT) (TFT)-LCD, organic light emission
Diode (OLED) display, flexible display, three-dimensional (3D) at least one of display and electrophoretic display device (EPD).According to electronics
The form of implementation of equipment 100, electronic equipment 100 may include two or more displays 111.In this case, it can be used
Two or more displays are arranged to facing with each other by hinge.
Display 111 can show the multiple objects picked out by electronic equipment 100.Pass through for example, display 111 can be shown
Multiple objects that imaging sensor picks out.Moreover, display 111 can show picked out by short haul connection it is multiple right
As.Display 111 can show the real image or virtual image of multiple objects.For example, display 111 can be shown including multiple right
The object map of the live view of elephant or virtual image including multiple objects.
Display 111 can show at least one attribute of the first object and the second object.For example, display 111 can be shown
Corresponding to the icon, symbol or text of at least one attribute.
Display 111 can show multiple operations that the attribute including attribute and the second object based on the first object determines
Recommend operating list.Moreover, display 111 can show the status information corresponding to multiple objects.
The exportable audio number for receiving or being stored in reservoir 150 from communication interface 120 of sound output transmitter 112
According to.Moreover, exportable related with the function of being executed by electronic equipment 100 voice signal of sound output transmitter 112 (such as exhale
Signal is made to receive sound, message sink sound and notification voice).For example, sound output transmitter 112 may include loud speaker, buzzing
Device etc., but not limited to this.
113 exportable vibration signal of vibrating motor.For example, the exportable audio data that corresponds to of vibrating motor 113 (such as is exhaled
Make signal receive sound, message sink sound etc.) or video data output vibration signal.
Communication interface 120 may include making it possible between electronic equipment 100 and external object or in electronic equipment 100
The one or more components communicated between server.For example, communication interface 120 may include short-range wireless communications interfaces 121,
Mobile communication interface 122 and radio receiver 123.
Short-range wireless communications interfaces 121 may include bluetooth communication interface, BLE communication interfaces, NFC communication interface, WLAN
And/or Wi-Fi communication interfaces, Zigbee communication interface, Infrared Data Association (IrDA) communication interface, WFD communication interfaces, UWB
Communication interface, Ant+ communication interfaces etc., but not limited to this.For example, short-range wireless communications interfaces 121 may include that Li-Fi is communicated
Interface.
Li-Fi can transmit the sub- technology of the VLC technologies of information for using the wavelength from the LED light emitted.Li-
Fi is available exist wherein illumination and it is harmless from anywhere in.Moreover, Li-Fi is due to its short distance and with high steady
Qualitative and safety, and make it possible to inexpensive high-speed communication.
Mobile communication interface 122 is handed over at least one of base station, exterior terminal and the server in mobile communications network
Change wireless signal.For example, according to the exchange of voice call signal, video call signal or text/Multimedia Message, wireless signal
It may include various forms of data.
Radio receiver 123 has by broadcast channel from the external reception broadcast singal of electronic equipment 100 and/or with broadcast
The information of pass.Broadcast channel may include satellite channel and terrestrial channel.For example, electronic equipment 100 does not include radio receiver
123。
Communication interface 120 can recognize positioned at from electronic equipment 100 it is a certain with a distance from multiple objects.
Communication interface 120 can send to target object (such as the first object, the second object or third object) and be based on first
The operation signal that the attribute of the attribute of object and the second object generates.Communication interface 120 can receive operation from target object and complete
Message or operation start message.
User interface 130 indicates the device for the data of control electronics 100 for user's input.For example, user
Interface 130 can be that keypad, dome switch, touch pads (touch capacitive, pressure resistant type, IR light beam sensings type, surface acoustic wave
Type, whole foil gauge type, piezoelectric effect type etc.), jog wheels, microswitch etc., but not limited to this.
User interface 130 selects the first object and among can receive multiple objects for showing on the display 111
The user of two objects inputs.Moreover, user interface 130 can receive for selecting at least one attribute of display over the display
User's input 111.User interface 130 can receive the input of one user in the operation for selecting multiple recommendations.User is defeated
One during it can be touch input to enter, voice inputs, bending inputs and eyes input, but not limited to this.
A/V input sinks 140 are intended to input audio signal or vision signal, and may include camera 141, microphone
142 etc..Camera 141 can obtain the video frame of static image, video etc. under video calling pattern or photograph mode.Pass through phase
The image of machine capture can be handled by processor 170 or additional image processor (not shown).
The video frame handled by camera 141 is storable in reservoir 150 or is sent to by communication interface 120 outer
Portion.It can be configured to provide two or more cameras 141 according to electronic equipment 100.
Microphone 142 receives external voice signal and is processed into electricity voice data.For example, microphone 142 can be from outer
Portion's equipment or loud speaker receive voice signal.Microphone 142 can be used various noise remove algorithms and receive external sound to remove
The noise occurred during sound signal.
Memory 150 can store the program of processing and the control for processor 170, and the input of executable data
And/or output (such as the attribute information of object, communication connection information etc.).
Memory 150 for example may include internal storage or external memory.Internal storage may include volatile storage
Device (such as dynamic random access memory (DRAM), static state RAM (SRAM), synchronous dynamic ram (SDRAM) etc.) and non-volatile
Memory (such as disposable programmable read only memory (OTPROM), programming ROM (PROM), erasable programmable ROM
(EPROM), electrically erasable ROM (EEPROM), mask rom, flash ROM, flash memory (such as NAND Flash storage
Device, NOR flash memory etc.), hard disk drive or solid state drive (SSD)) at least one of.
External memory may include flash drive, such as the storage of compact flash (CF) storage card, secure digital (SD)
Card, micro-SD memory card, mini SD storage cards, extreme digital (XD) storage card, multimedia card (MMC), memory stick etc..Pass through
Various interfaces, external memory functionally and/or can be connected physically to electronic equipment 100.Moreover, electronic equipment 100 can
Management executes the web storages of the store function of reservoir 150 on the internet.
The program being stored in memory 150 can be classified as multiple modules according to its function, such as be classified as pair
As identification module 151, eyeball tracking module 152 etc., but multiple modules are without being limited thereto.
Process identification module 151 may include the image for recognizing the first object and the second object that are included in primitive frame
Processing Algorithm.Process identification module 151 can detect the profile of the profile and the second object of the first object by analyzing primitive frame.
Then, process identification module 151 can be by the profile of the first object of detection and the second object compared with predefined template, with inspection
Survey type, the title etc. of object.
Process identification module 151 may include the facial recognition algorithm for executing facial recognition to object.Process identification mould
Block 151 can be by the feature of the face extracted from facial area compared with the facial characteristics of pre-registered user.Process identification mould
Block 151 may include the OCR algorithm for executing OCR to the character being included in object.
Process identification module 151 can the intensity of object-based signal of communication determine from electronic equipment 100 to by short
The distance of the object of distance communication identification.
Eyeball tracking module 152 can analyze blink, watch the movement speed of position and/or eyeball attentively to explain the eyes of user
Input.
Memory 150 can relating attribute table 153 between storage object, device-specific control information operation 154 and object
Identification model information 155.
Between object relating attribute table 153 or 310 can be the attribute for corresponding to multiple objects defined in it combination control
The table of information processed.For example, control information may include about equipment (the hereinafter referred to as control device that will be controlled among multiple objects
Or target object) and by the information of the operation executed by control device.
The control information operation 154 of device-specific may include the information of the control protocol about device-specific.For example, equipment
Dedicated control information operation 154 may include data format for generating operation signal, be included in operation signal about
The information etc. of control command language.
Object identity model information 155 may include for recognizing each in multiple objects and executing corresponding to multiple
The information of the function of each (such as apply or controller) in object.For example, process identification model information 155 may include it is more
A process identification model, for identification with each matched multiple object and functional information in multiple process identification models
Each in (such as application message, controller information, memo information, web site address information etc.).
Sensor 160 can sensing electronic equipment 100 state or the state around electronic equipment 100, and by sensing
Information sends processor 170 to.
Sensor 160 may include Magnetic Sensor 161, acceleration transducer 162, inclination sensor 163, IR sensors 164,
In gyro sensor 165, position sensor 166, fingerprint sensor 167, proximity sensor 168 and photoelectric sensor 169
It is at least one, but not limited to this.Due to each sensor function can by those of ordinary skill in the art from its title intuitively
Infer, therefore its detailed description will be omitted.
The integrated operation of 170 usual control electronics 100 of processor.For example, being stored in memory 150 by control
Program, the controllable output transmitter 110 of processor 170, communication interface 120, user interface 130, A/V input sinks 140,
All elements in memory 150, sensor 160 etc..
Processor 170 can recognize the first object and the second object and identify the category of the attribute and the second object of the first object
Property.Processor 170 can be selected based on the attribute of the attribute of the first object and the second object between the first object and the second object
The object of control is selected, and generates the operation signal of the object for selection.For example, between the first object and the second object
When selecting the second object, processor 170 can be generated pair based on the functional information of the status information of the first object and the second object
In the operation signal of the second object.
Processor 170 can touch the sequence of the first object and the second object to generate to the object in selection based on user
Operation signal.For example, when drag object on user in a first direction (such as from first object to the direction of the second object), place
Manage device 170 and produce the first operation signal for corresponding to the first operation, and when user in second direction (such as from the second object
To the direction of the first object) on drag object when, processor 170 produce correspond to second operation the second operation signal.
When the first object and the second object are uncontrollable objects, attribute and that processor 170 can be based on the first object
The attribute of two objects selects controllable third object.Processor 170 produces the operation signal for third object, and controls
Communication interface 120 processed is to send the operation signal for third object to third object.
When the first object and the second object are uncontrollable objects, attribute and that processor 170 can be based on the first object
The attribute of two objects selects the operation of electronic equipment 100, and executes the operation of selection.
According to for selecting one in multiple operations that the attribute of attribute and the second object based on the first object recommends
User input, processor 170 produce corresponding to selection operation operation signal.
Figure 44 is the block diagram for the configuration for illustrating server according to one exemplary embodiment.
With reference to figure 44, server 2600 according to one exemplary embodiment may include communication interface 2610,2620 (example of processor
Such as microprocessor) and memory 2630 (such as storage device).However, the component of all diagrams is not necessary component.Clothes
Business device 2600 can be realized by components more more or fewer than the component count of diagram.
Communication interface 2610 may include making it possible to communicated between server 2600 and electronic equipment 100 one or more
A component.For example, communication interface 2610 can receive the image of object, about the application (control for being linked to object from electronic equipment 100
Device, memorandum or station address processed) information, the identification information of object and being asked for generating process identification model information
It asks.
When process identification model information is generated, communication interface 2610 can recognize mould to 100 sending object of electronic equipment
Type information.Communication interface 2610 can periodically or when a specific event occurs be believed to electronic equipment sending object identification model
Breath.
The integrated operation of the usually control server 2620 of processor 2620.For example, processor 2620 produces general object
General process identification model information or special object selected by user personalized process identification model information.
Memory 2630 can store the process for the processing of processor 2620 and store input and/or output data.Example
Such as, memory 2630 can store image search module 2631, training image generation module 2632 etc..
Image search module 2631 can search for the image for corresponding to the object of multiple classifications and multiple keywords.For example, working as
When classification is " TV " and keyword is " living room ", image search module 2631 can search in search server to be corresponded to
The image of room TV.
Depth learning technology can be used to generate the instruction for corresponding to the object images found in training image generation module 2632
Practice image.For example, training image generation module 2632 can be generated by changing the angle of the image found for identification objects
Training image.
Moreover, memory 2630 can build the DB for managing the process identification model information generated by server 2600
2633。
Method according to one exemplary embodiment can be can perform by various calculating instruments and be recorded in computer-readable record
The form of program instruction in medium embodies.Computer readable recording medium storing program for performing can include program instruction, number alone or in combination
According to file, data structure etc..The program instruction being recorded in computer readable recording medium storing program for performing can be designed or be configured to demonstrate
Property embodiment, or can be computer software fields those of ordinary skill it is known and be used by it.Computer-readable record is situated between
The example of matter includes the magnetic medium for being specifically configured to store and execute such as hard disk, floppy disk and tape of program instruction etc, all
Such as the optical medium of compact disk ROM (CD-ROM) and digital versatile disc (DVD) etc, the magnet-optical medium of such as CD etc,
And the hardware device of such as ROM, RAM, flash memory or the like.The example of program instruction includes using explanation by computer
The executable higher-level language code such as device and the machine language code by compiler-creating.
According to one exemplary embodiment, attribute of the electronic equipment 100 based on the multiple objects picked out by electronic equipment 100,
Target object and the object run for determining control according to the user's intention, set so that user can easily control outside
It is standby.
Although a few exemplary embodiment has been shown and described, one exemplary embodiment not limited to this.This field
The skilled person will understand that：The principle and spirit that these exemplary embodiments can be changed without departing from the disclosure, this public affairs
The range opened is in claim and its equivalent middle restriction.
Claims (15)
1. a kind of method for electronic equipment control object, this method include：
Identification includes multiple objects of the first object and the second object；
Identify the first attribute of the first object and the second attribute of the second object；
The target object of control is selected based on the first attribute and the second attribute, which is at least in multiple objects
Selection；
Generate the operation signal for target object；And
Based on the operation signal generated come the operation of control targe object.
2. the method for claim 1, wherein recognizing multiple objects includes：
Show the multiple objects captured by imaging sensor；And
It receives for selecting the user of the first object and the second object to input in multiple objects.
3. the method for claim 1, wherein recognizing multiple objects includes：
Display be located at from electronic equipment it is a certain with a distance from multiple objects image, the multiple object is logical via short distance
Believe and is positioned；And
It receives for selecting the user of the first object and the second object to input in multiple objects.
4. it is uncontrollable right that the method for claim 1, wherein identification, which includes by the first object and the second object detection,
As, also, the method further includes：
Based on the first attribute and the second attribute, select third object as the target object of control in multiple objects；
Generate the operation signal for third object；And
The operation signal for controlling third object generated is sent to third object.
5. it is uncontrollable right that the method for claim 1, wherein identification, which includes by the first object and the second object detection,
As,
Selection include electronic equipment itself is selected as target object, and
The method further includes：
The operation of electronic equipment is selected based on the first attribute and the second attribute；And
Execute selected operation.
6. the method for claim 1, wherein the first attribute be the first object multiple attributes in one,
Second attribute is one in multiple attributes of the second object, and
Identify that the first attribute and the second attribute include：Show that multiple attributes of the first object at least one of work as attribute and
Multiple attributes of two objects at least one of work as attribute.
7. method as claimed in claim 6, wherein the first attribute of identification and the second attribute further comprise：
It receives at least one as the user of the first attribute or the second attribute input for being selected from shown attribute.
8. the method for claim 1, wherein generating and including for the operation signal of target object：
When the second object is chosen as target object, the functional information next life of status information and the second object based on the first object
In pairs in the operation signal of the second object.
9. the method for claim 1, wherein generating and including for the operation signal of target object：
The sequence of the first object and the second object is touched based on user, generates the operation signal for target object.
10. the method for claim 1, wherein generating and including for the operation signal of target object：
Recommend multiple operations based on the first attribute and the second attribute；
Receive one user input in the operation for selecting multiple recommendations；And
Generate the operation signal corresponding to selected operation.
11. a kind of non-transitory computer readable recording medium storing program for performing of storage program, the program make when being executed by computer system
It obtains the computer system and executes the method as described in claim 1.
12. a kind of electronic equipment, including：
Processor is configured to：Identification includes multiple objects of the first object and the second object, and the first of the first object of identification belongs to
Property and the second object the second attribute, based on the first attribute and the second attribute come selected in multiple objects control target pair
As, and generate the operation signal for target object；With
Communication interface is configured to send generated operation signal to target object,
Wherein, target object is the object of electronic device exterior.
13. electronic equipment as claimed in claim 12, further comprises：
Display is display configured to the multiple objects captured by imaging sensor；With
User interface is configured to receive the user for selecting the first object and the second object in the object of multiple displays
Input.
14. electronic equipment as claimed in claim 12, further comprises：
Display, be display configured to be located at from electronic equipment it is a certain with a distance from multiple objects image, the multiple object
Positioned via short haul connection；With
User interface is configured to receive for selecting the user of the first object and the second object to input in multiple objects.
15. electronic equipment as claimed in claim 12, wherein when the first object and the second object are uncontrollable object, institute
Processor is stated to be further configured to：Based on the first attribute and the second attribute come selected in multiple objects third object as
The target object of control generates the operation signal for third object, and controls communication interface and used with being sent to third object
In the operation signal of control third object.
Applications Claiming Priority (5)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| KR10-2015-0185098 | 2015-12-23 | ||
| KR20150185098 | 2015-12-23 | ||
| KR10-2016-0049403 | 2016-04-22 | ||
| KR1020160049403A KR101847200B1 (en) | 2015-12-23 | 2016-04-22 | Method and system for controlling an object |
| PCT/KR2016/007708 WO2017111234A1 (en) | 2015-12-23 | 2016-07-15 | Method for electronic device to control object and electronic device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108476258A true CN108476258A (en) | 2018-08-31 |
| CN108476258B CN108476258B (en) | 2021-08-13 |
Family
ID=59357678
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201680075716.5A Active CN108476258B (en) | 2015-12-23 | 2016-07-15 | Method for controlling object by electronic equipment and electronic equipment |
Country Status (2)
| Country | Link |
|---|---|
| KR (1) | KR101847200B1 (en) |
| CN (1) | CN108476258B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113014658A (en) * | 2021-03-10 | 2021-06-22 | 北京蜂巢世纪科技有限公司 | Device control, device, electronic device, and storage medium |
| US20220116684A1 (en) * | 2020-10-12 | 2022-04-14 | Arris Enterprises Llc | Set-top box ambiance and notification controller |
| US11430028B1 (en) * | 2017-11-30 | 2022-08-30 | United Services Automobile Association (Usaa) | Directed information assistance systems and methods |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR102045347B1 (en) * | 2018-03-09 | 2019-11-15 | 에스케이브로드밴드주식회사 | Surppoting apparatus for video making, and control method thereof |
| KR20200006484A (en) * | 2018-07-10 | 2020-01-20 | 삼성전자주식회사 | Electronic device and operating method for performing an operation based on bending |
| KR102484647B1 (en) * | 2018-07-11 | 2023-01-04 | 삼성전자주식회사 | Method of controlling electronic apparatus and computer-readable recording medium |
| KR20200062864A (en) * | 2018-11-27 | 2020-06-04 | 삼성전자주식회사 | Image display apparatus and operating method for the same |
| KR102040939B1 (en) | 2019-07-15 | 2019-11-27 | 한화테크윈 주식회사 | Surveillance system and operation method thereof |
Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110170787A1 (en) * | 2010-01-12 | 2011-07-14 | Qualcomm Incorporated | Using a display to select a target object for communication |
| CN102340594A (en) * | 2010-07-19 | 2012-02-01 | Lg电子株式会社 | Mobile terminal and method for controlling the same |
| KR20120066879A (en) * | 2010-12-15 | 2012-06-25 | 엘지전자 주식회사 | Mobile terminal and method for controlling the same |
| US20120299709A1 (en) * | 2011-05-27 | 2012-11-29 | Kyocera Corporation | Remote control device, remote control system, and storage medium storing control program, and medium to be attached to electrical device |
| KR20130022687A (en) * | 2011-08-26 | 2013-03-07 | 엘지전자 주식회사 | Method for controlling a multimedia device and digital television |
| CN103581251A (en) * | 2012-08-01 | 2014-02-12 | 鸿富锦精密工业（深圳）有限公司 | Remote control device and control method thereof |
| CN103917941A (en) * | 2011-11-02 | 2014-07-09 | Lg电子株式会社 | Remote controller and method for operating the same |
| US20140266639A1 (en) * | 2013-03-15 | 2014-09-18 | Ebay Inc. | Automated mobile device configuration for remote control of electronic devices |
| CN104267608A (en) * | 2014-10-20 | 2015-01-07 | 珠海格力电器股份有限公司 | Intelligent household equipment control method, device and system |
| US20150066158A1 (en) * | 2013-09-05 | 2015-03-05 | Samsung Electronics Co., Ltd. | Method and apparatus for configuring and recommending device action using user context |
| CN104683436A (en) * | 2013-11-28 | 2015-06-03 | 松下电器（美国）知识产权公司 | Control Method, Communications Terminal, And Control System |
| CN104991458A (en) * | 2015-06-30 | 2015-10-21 | 联想(北京)有限公司 | Control method and electronic equipment |
| CN105137778A (en) * | 2015-07-30 | 2015-12-09 | 四川长虹电器股份有限公司 | Intelligent household control method and control center |
| CN105141913A (en) * | 2015-08-18 | 2015-12-09 | 华为技术有限公司 | Method and system for visually and remotely controlling touch control equipment and relevant equipment |
-
2016
- 2016-04-22 KR KR1020160049403A patent/KR101847200B1/en active IP Right Grant
- 2016-07-15 CN CN201680075716.5A patent/CN108476258B/en active Active
Patent Citations (14)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20110170787A1 (en) * | 2010-01-12 | 2011-07-14 | Qualcomm Incorporated | Using a display to select a target object for communication |
| CN102340594A (en) * | 2010-07-19 | 2012-02-01 | Lg电子株式会社 | Mobile terminal and method for controlling the same |
| KR20120066879A (en) * | 2010-12-15 | 2012-06-25 | 엘지전자 주식회사 | Mobile terminal and method for controlling the same |
| US20120299709A1 (en) * | 2011-05-27 | 2012-11-29 | Kyocera Corporation | Remote control device, remote control system, and storage medium storing control program, and medium to be attached to electrical device |
| KR20130022687A (en) * | 2011-08-26 | 2013-03-07 | 엘지전자 주식회사 | Method for controlling a multimedia device and digital television |
| CN103917941A (en) * | 2011-11-02 | 2014-07-09 | Lg电子株式会社 | Remote controller and method for operating the same |
| CN103581251A (en) * | 2012-08-01 | 2014-02-12 | 鸿富锦精密工业（深圳）有限公司 | Remote control device and control method thereof |
| US20140266639A1 (en) * | 2013-03-15 | 2014-09-18 | Ebay Inc. | Automated mobile device configuration for remote control of electronic devices |
| US20150066158A1 (en) * | 2013-09-05 | 2015-03-05 | Samsung Electronics Co., Ltd. | Method and apparatus for configuring and recommending device action using user context |
| CN104683436A (en) * | 2013-11-28 | 2015-06-03 | 松下电器（美国）知识产权公司 | Control Method, Communications Terminal, And Control System |
| CN104267608A (en) * | 2014-10-20 | 2015-01-07 | 珠海格力电器股份有限公司 | Intelligent household equipment control method, device and system |
| CN104991458A (en) * | 2015-06-30 | 2015-10-21 | 联想(北京)有限公司 | Control method and electronic equipment |
| CN105137778A (en) * | 2015-07-30 | 2015-12-09 | 四川长虹电器股份有限公司 | Intelligent household control method and control center |
| CN105141913A (en) * | 2015-08-18 | 2015-12-09 | 华为技术有限公司 | Method and system for visually and remotely controlling touch control equipment and relevant equipment |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US11430028B1 (en) * | 2017-11-30 | 2022-08-30 | United Services Automobile Association (Usaa) | Directed information assistance systems and methods |
| US20220116684A1 (en) * | 2020-10-12 | 2022-04-14 | Arris Enterprises Llc | Set-top box ambiance and notification controller |
| US11627377B2 (en) * | 2020-10-12 | 2023-04-11 | Arris Enterprises Llc | Set-top box ambiance and notification controller |
| CN113014658A (en) * | 2021-03-10 | 2021-06-22 | 北京蜂巢世纪科技有限公司 | Device control, device, electronic device, and storage medium |
| CN113014658B (en) * | 2021-03-10 | 2023-04-07 | 北京蜂巢世纪科技有限公司 | Device control, device, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| KR20170075625A (en) | 2017-07-03 |
| CN108476258B (en) | 2021-08-13 |
| KR101847200B1 (en) | 2018-04-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3345379B1 (en) | Method for electronic device to control object and electronic device | |
| CN108476258A (en) | Method and electronic equipment for electronic equipment control object | |
| US10552679B2 (en) | Wearable apparatus and methods for causing a paired device to execute selected functions | |
| CN105320428B (en) | Method and apparatus for providing image | |
| US9900498B2 (en) | Glass-type terminal and method for controlling the same | |
| CN111652678A (en) | Article information display method, device, terminal, server and readable storage medium | |
| CN107832784A (en) | A kind of method of image beautification and a kind of mobile terminal | |
| KR102238330B1 (en) | Display device and operating method thereof | |
| TWI544336B (en) | Classes,electronic device and method of pairing thereof and seamless content playback method | |
| KR20180109499A (en) | Method and apparatus for providng response to user's voice input | |
| CN109361852A (en) | A kind of image processing method and device | |
| WO2022199500A1 (en) | Model training method, scene recognition method, and related device | |
| CN113495487A (en) | Terminal and method for adjusting operation parameters of target equipment | |
| KR101481996B1 (en) | Behavior-based Realistic Picture Environment Control System | |
| US20210327417A1 (en) | Electronic device and control method therefor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |