WO2019237965A1

WO2019237965A1 - Human-computer interaction and television operation control method, apparatus and device, and storage medium

Info

Publication number: WO2019237965A1
Application number: PCT/CN2019/089929
Authority: WO
Inventors: 卞文瀚; 姚维; 马骥; 胡乐水
Original assignee: 阿里巴巴集团控股有限公司
Priority date: 2018-06-12
Filing date: 2019-06-04
Publication date: 2019-12-19
Also published as: CN110602559A; TW202001541A

Abstract

The present disclosure provides a human-computer interaction and television operation control method, apparatus and device, and a storage medium. In response to the input of a user, an interface element corresponding to identification information matching the input is acquired from a presentation interface, wherein at least one interface element in the presentation interface is configured with one or more pieces of identification information to represent the interface element; and a preset operation is executed on the acquired interface element. According to this, the interface element corresponding to the input of the user can be identified conveniently, and the degree of intelligence and the interactive experience of the user are improved while operating requirements of the user are reduced.

Description

Human-computer interaction and television operation control method, device, equipment and storage medium

This application claims the priority of a Chinese patent application filed on June 12, 2018 with an application number of 201810599979.7 and an invention name of "Human-Computer Interaction and TV Operation Control Method, Device, Equipment, and Storage Medium", the entire contents of which are incorporated by reference In this application.

Technical field

The present disclosure relates to the field of interface display technology, and in particular, to a method, a device, a device, and a storage medium for human-computer interaction and television operation control related to a display interface.

Background technique

The convenience of human-computer interaction is an important factor in the intelligence of electronic devices. At present, for the interface elements (such as video, audio, information, etc.) in the interface displayed on the screen of an electronic device, the user often needs to input an operation instruction that strictly matches the interface element in order to perform a preset operation on the corresponding interface element. Such as video playback operations. The degree of intelligence is low, and the user's interactive experience is poor.

Taking a smart TV that provides voice operations as an example, the TV desktop generally displays multiple video resources that users can choose to play. Users need to send voice messages that match the name of the video they want to watch in order to watch the corresponding video. This undoubtedly constrains the user's behavior to a certain extent and reduces the user's interactive experience. For example, when displaying multiple videos including "Flash," "Spider-Man," "The Hobbit," "Gorilla Ball Wars," etc., the user needs to say "Hobbit", "Hobbit" and other strong correlations. Vocabulary to watch the film The Hobbit.

Therefore, there is a need for a human-computer interaction solution that is convenient for user operations.

Summary of the Invention

An object of the present disclosure is to provide a human-machine interaction scheme that is convenient for a user to operate.

According to a first aspect of the present disclosure, a human-computer interaction method is provided, including: in response to a user input, obtaining an interface element corresponding to identification information that matches the input in a display interface, where At least one interface element in the display interface is configured with one or more identification information to characterize the interface element; and performing a preset operation on the acquired interface element.

Optionally, the interface element is an information display entry or a function entry.

Optionally, the interface elements include one or more of the following: information; pictures; music; videos; menus; scroll bars.

Optionally, the identification information includes visual identification information, and the visual identification information is displayed in the display interface in association with a corresponding interface element.

Optionally, each of the identification information represents the interface element with one of one or more feature dimensions.

Optionally, the one or more identification information is determined according to attribute information of the interface element corresponding to one or more feature dimensions.

Optionally, the attribute information includes one or more of the following feature dimensions: the name of the interface element; the type of the interface element; the display position of the interface element; the display order of the interface element; the The color of the interface element; the label of the interface element; the child elements contained by the interface element.

Optionally, the human-computer interaction method further includes: in response to a change in the display interface, configuring one or more of the identification information for a newly displayed interface element in the changed display interface.

Optionally, the human-computer interaction method further includes: in response to a change in the display interface, deleting identification information of interface elements that are no longer displayed.

Optionally, the input is a voice input, and the step of obtaining an interface element corresponding to the identification information that matches the input in the display interface includes: parsing the voice input to obtain the text of the voice input Information; obtaining interface elements corresponding to the identification information matching the text information in the display interface.

Optionally, the interface element corresponds to a preset operation, and the step of performing a preset operation on the acquired interface element includes performing a preset operation corresponding to the interface element on the acquired interface element.

Optionally, the interface element corresponds to a plurality of preset operations, and the step of performing the preset operation on the acquired interface element includes: parsing the input to obtain the input instruction information; and for the acquired interface The element performs a preset operation that matches the instruction information.

According to a second aspect of the present disclosure, there is also provided a television operation control method, including: in response to a user input, obtaining an interface element corresponding to the identification information matching the input in a television interface, where: At least one interface element in the television interface is configured with one or more identification information to characterize the interface element; and a preset operation is performed on the acquired interface element.

Optionally, the input is a voice input, and the step of obtaining the interface element corresponding to the identification information matching the input in the television interface may include: parsing the voice input to obtain the text information of the voice input; and obtaining the text information in the television interface. The interface element corresponding to the identification information corresponding to the information.

According to a third aspect of the present disclosure, there is also provided a human-machine interaction device, including: an acquisition module, configured to acquire an interface corresponding to the identification information that matches the input in a display interface in response to a user input An element, wherein at least one interface element in the display interface is configured with one or more identification information to characterize the interface element; and an execution module configured to perform a preset operation on the acquired interface element.

Optionally, the attribute information includes one or more of the following feature dimensions: the name of the interface element; the type of the interface element; the display position of the interface element; the display order of the interface element; the The color of the interface element; the label of the interface element; one or more child elements contained by the interface element.

Optionally, the human-machine interaction device further includes a configuration module, configured to configure one or more of the identification information for a newly displayed interface element in the changed display interface in response to the change in the display interface.

Optionally, the human-computer interaction device further includes: a deleting module, configured to delete identification information of interface elements that are no longer displayed in response to the display interface changing.

Optionally, the input is a voice input, and the device further includes a first parsing module configured to parse the voice input to obtain text information of the voice input, and the obtaining module obtains the information in the display interface. An interface element corresponding to the identification information that matches the text information.

Optionally, the interface element corresponds to a preset operation, and the execution module executes the preset operation corresponding to the interface element with respect to the acquired interface element.

Optionally, the interface element corresponds to a plurality of preset operations, and the device further includes: a second parsing module, configured to parse the input to obtain the input instruction information, and the execution module is directed to the acquired The interface element performs a preset operation that matches the instruction information.

According to a fourth aspect of the present disclosure, there is also provided a television operation control device, including: an obtaining module, configured to obtain an interface corresponding to the identification information matching the input in a television interface in response to a user input An element, wherein at least one interface element in the television interface is configured with one or more identification information to characterize the interface element; and an execution module configured to perform a preset operation on the acquired interface element.

Optionally, the input is a voice input, and the device further includes: a parsing module configured to parse the voice input to obtain text information of the voice input, and the obtaining module obtains the corresponding information in the television interface. The interface element corresponding to the identification information described in the text information.

According to a fifth aspect of the present disclosure, there is also provided an electronic device, including: a display for providing a display interface for a user, wherein the display interface includes one or more interface elements for the display interface At least one interface element is configured with one or more identification information to characterize the interface element; a memory is configured to store the identification information corresponding to the interface element; and a processor is configured to The display interface acquires the interface element corresponding to the identification information that matches the input, and performs a preset operation on the acquired interface element.

According to a sixth aspect of the present disclosure, there is also provided a computing device including: a processor; and a memory having executable code stored thereon, and when the executable code is executed by the processor, causing the processor to The processor executes the method as described in the first aspect or the second aspect of the present disclosure.

According to a seventh aspect of the present disclosure, there is also provided a non-transitory machine-readable storage medium on which executable code is stored, and when the executable code is executed by a processor of an electronic device, the processing is performed The processor performs the method as described in the first aspect or the second aspect of the present disclosure.

Therefore, the present disclosure can easily identify the interface element targeted by the user's input by configuring the interface element with identification information capable of characterizing the interface element. Compared with the existing interface interaction scheme, while reducing the user operation requirements, Can improve the degree of intelligence and enhance the user's interactive experience.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other objects, features, and advantages of the present disclosure will become more apparent through a more detailed description of the exemplary embodiments of the present disclosure with reference to the accompanying drawings. In the exemplary embodiments of the present disclosure, the same reference numerals generally represent Same parts.

FIG. 1 is a schematic block diagram showing a structure of an electronic device 100 according to an embodiment of the present disclosure;

FIG. 2 shows a schematic flowchart of a human-machine interaction method according to an embodiment of the present disclosure;

3 is a schematic diagram of a display interface according to an embodiment of the present disclosure;

4 is a schematic structural diagram of a human-machine interaction device according to an embodiment of the present disclosure;

5 is a schematic structural diagram of a television operation control device according to an embodiment of the present disclosure;

FIG. 6 is a schematic structural diagram of a computing device according to an embodiment of the present disclosure.

detailed description

Hereinafter, preferred embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although the preferred embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

Existing interface interaction solutions have high requirements on user operations. Users need to input instruction information that is strictly consistent with the interface elements that they want to operate in order to perform preset operations on the interface elements. Different users generally have different awareness or concerns about the same interface element, and existing interaction solutions require users to enter semantic information that is strongly related to the interface element in order to achieve the operation of the interface element. This undoubtedly requires all users to The same interface element maintains the same degree of cognition, and enters voice or text information consistent or strongly related to the cognition. Therefore, the existing interaction schemes are less intelligent and the user's interaction experience is poor.

In view of this, the present disclosure proposes that, according to the attribute information of the interface element in the interface corresponding to multiple feature dimensions, a rich label (that is, the identification information described below) is configured for the interface element to represent from multiple feature dimensions Interface elements. When the user inputs instruction information through voice or text, the instruction information can be matched with a pre-configured label to determine an interface element that the user desires to operate.

Therefore, by configuring rich tags for interface elements, in the case where the same interface element is displayed to multiple users, even if the multiple users have different levels of awareness or attention to the interface element, they can also be input by Voice information or text information that is consistent with the level of self-awareness or attention, realizes the operation of the same interface element. The entire interaction process has fewer constraints on the user, which is convenient for the user to operate, and can improve the user's interactive experience while improving the degree of intelligence.

The human-computer interaction scheme provided by the embodiment of the present disclosure can be applied to an electronic device as shown in FIG. 1. FIG. 1 is a schematic block diagram showing a structure of an electronic device 100 according to an embodiment of the present disclosure. The electronic device 100 may be implemented as various types of computer devices, such as a desktop computer, a portable computer, a tablet computer, a smart phone, a personal data assistant (PDA), or other types of computer devices, but is not limited to any specific form. .

As shown in FIG. 1, the electronic device 100 of the present disclosure may include a display 110, a processor 120, and a memory 130. The display 110 may be various types of display devices such as a cathode ray tube display (CRT), a plasma display (PDP), and a liquid crystal display (LCD). The processor 120 may be a multi-core processor, or may include multiple processors. In some embodiments, the processor 120 may include a general-purpose main processor and one or more special co-processors, such as a graphics processor (GPU), a digital signal processor (DSP), and the like. In some embodiments, the processor 120 may be implemented using customized circuits, such as Application Specific Integrated Circuits (ASICs) or Field Programmable Gate Arrays (FPGAs).

The memory 130 may include various types of storage units, such as a system memory, a read-only memory (ROM), and a permanent storage device. The ROM may store static data or instructions required by the processor 120 or other modules of the computer. The persistent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (eg, magnetic or optical disk, flash memory) as the permanent storage device. In other embodiments, the permanent storage device may be a removable storage device (for example, a floppy disk, an optical drive). The system memory can be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory can store some or all of the instructions and data required by the processor while it is running. In addition, the memory 130 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used. In some embodiments, the memory 130 may include a readable and / or writeable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (eg, DVD-ROM, dual-layer DVD-ROM), Read-only Blu-ray discs, ultra-density discs, flash cards (such as SD cards, SD cards, Micro-SD cards, etc.), magnetic floppy disks, etc. Computer-readable storage media does not contain carrier waves and transient electronic signals transmitted by wireless or wire.

In the embodiment of the present disclosure, the display 110 is configured to provide a display interface for a user, wherein the display interface includes one or more interface elements, and all data required for processing by the processor 120 may be stored on the memory 130, and the processor 120 is configured to: The human-computer interaction scheme of the present disclosure is executed to provide interactive services for users.

FIG. 2 shows a schematic flowchart of a human-machine interaction method according to an embodiment of the present disclosure. The method shown in FIG. 2 may be executed by the processor 120 in FIG. 1.

Referring to FIG. 2, in step S210, one or more identification information is configured for at least one interface element in the display interface to characterize the interface element.

The display interface mentioned in the present disclosure refers to an interface displayed on a screen of an electronic device. For example, it can be the interface displayed on the desktop of a smart TV, the screen of an electronic device such as a mobile phone / IPAD, an application interface of a third-party application installed in an electronic device such as a mobile phone / IPAD, such as a video application interface such as Youku or iQiyi.

The display interface may include one or more interface elements, wherein the interface elements mentioned in the present disclosure preferably refer to interface elements capable of performing corresponding operations in response to user input, which may be information display portals or functions Entrance. For example, the display interface may be an interface displayed on the screen of an electronic device such as a mobile phone, an iPad, a notebook, or a smart TV. The interface element may be an information display entrance such as information, pictures, music, videos, etc. in the interface, or a menu or scroll bar , Theme settings, account login, and other functions such as function selection or function settings.

When configuring identification information for interface elements, you can mine as much as possible of the attribute information of the interface elements under multiple feature dimensions to configure multiple interface elements that can represent the interface with different feature dimensions based on the attribute information of multiple feature dimensions. Element identification information. Thus, each identification information can represent the interface element with a predetermined feature dimension.

For example, the identification information may be configured for the interface element according to the attribute information of the various feature dimensions such as the name, type, color, placement position, display order, label, and sub-elements included in the interface element. The label may be an attribute label related to the interface element itself, such as a classification label of the interface element, or may be another type of label, such as a label such as a rating, a number of favorites, a number of views, and a popularity. The child elements contained in the interface element may refer to the child elements obtained by disassembling the interface element. For example, taking an interface element as a picture, the sub-elements of the interface element can be specific content in the picture, such as "a hand with a gun", "a hat", "firefighter", and so on.

In this disclosure, starting from whether the identification information exhibits this characteristic, the identification information configured for the interface element can be divided into visual identification information and non-visual identification information. The visual identification information refers to identification information that can be displayed in a display interface in association with a corresponding interface element. Invisible identification information refers to identification information that is not displayed in the display interface.

In addition, from the characterization characteristics of the identification information, the identification information mentioned in this disclosure can also be divided into explicit identification information (also known as "explicit label") and implicit identification information (also known as "recessive" label"). Explicit identification information refers to identification information configured according to display attributes of interface elements that can be intuitively understood by the user, such as visually displaying according to the color, shape, display position, display order, sub-elements included in the interface element, etc. The identification information configured by the attribute information in the interface element may be referred to as explicit identification information. Implicit identification information refers to identification information configured according to the hidden attributes of interface elements that cannot be intuitively understood by the user. For example, the classification, rating, number of favorites, views, and popularity of interface elements are not displayed in the interface elements intuitively. The identification information configured by the attribute information may be called implicit identification information.

It should be noted that the present disclosure does not limit the specific definition of explicit identification information and implicit identification information. The reason for describing explicit identification information and implicit identification information is only to show that the disclosure can be based on various aspects of interface elements. The attribute information of the interface is configured with rich identification information for the interface elements, so that the configured identification information can cover (match) the input made by different users based on different cognition levels or concerns of the interface elements as much as possible. This can improve the degree of intelligence and user interaction experience.

In addition, whether the identification information is visible and the identification information is explicit identification information or implicit identification information has no direct relationship, that is, the explicit identification information can be displayed in the display interface in association with the interface elements as the visual identification information. Recessive identification information can be displayed in the display interface in association with interface elements as visual identification information.

It should be noted that the step of configuring identification information for the interface element (that is, step S210) may be real-time configuration when the interface element is displayed in the current display interface, or it may be configured with identification information for the interface element in advance. That is, the human-computer interaction scheme of the present disclosure may not include step S210, but step S210 may be performed in advance by an independent processing module to configure identification information for the interface elements.

In step S220, in response to the user's input, an interface element corresponding to the identification information that matches the input is acquired in the display interface. Wherein, one or more identification information is configured for at least one interface element in the display interface to characterize the interface element.

The identification information configured for the interface element can be used as a reference for subsequent matching with the user input to determine the interface element to which the user's input is directed. The user's input can be voice input or text input.

When the user's input is a voice input, the voice input needs to be parsed to obtain the text information of the voice input, and then the interface element corresponding to the identification information that matches the text information is obtained in the display interface. The process of speech analysis is well known to those skilled in the art, and will not be repeated here. When searching for matching identification information based on text information, matching can be based on natural semantics, such as fuzzy matching based on natural semantics. In addition, multiple similarity calculation methods can also be used to calculate the similarity between the identification information and the text information, such as the cosine similarity. The similarity calculation is also well known to those skilled in the art, and will not be repeated here.

In step S230, a preset operation is performed on the acquired interface element.

A preset operation that a user can perform on an interface element may be a fixed operation or multiple operations. For example, for a plurality of playable video elements (for example, pictures in the interface shown in FIG. 3) displayed on a television desktop, a preset operation corresponding to the interface element may be a fixed operation of jumping or playing. For another example, for interface elements such as a scroll bar and a menu bar, there may be multiple preset operations. For example, the preset operations corresponding to the scroll bar may be scrolling up or down.

In the case where there is one preset operation corresponding to the interface element, the operation corresponding to the interface element may be directly performed for the acquired interface element. In the case where there is one preset operation corresponding to the interface element, it is also necessary to parse out the instruction information contained in the user's input to perform an operation that matches the instruction information for the obtained interface element.

As an optional embodiment of the present disclosure, in the case where the display interface changes (such as the interface is updated and the user opens a new interface), one or more interface elements that are newly displayed in the changed display interface may also be configured. Identification information, and can also delete identification information of interface elements that are no longer displayed. Therefore, a real-time updated identification information set can be maintained according to the change of the display interface, and the identification information in the identification information set is configured for the interface elements in the current display interface. In this way, when the interface element corresponding to the identification information matching the user's input is obtained in the display interface, the identification information matching the user's input can be searched directly from the maintenance identification information set, and the search corresponding to the identification information Interface elements are interface elements that users expect to perform operations.

So far, the implementation process of the human-computer interaction method of the present disclosure has been described in detail with reference to FIG. 2.

The human-computer interaction scheme of the present disclosure can be applied to (smart) television. When the human-computer interaction scheme of the present disclosure is applied to a television, the present disclosure can be implemented as a television operation control method. By obtaining interface elements corresponding to identification information that matches the user's input in the television interface, a preset operation can be performed on the acquired interface elements, wherein at least one interface element in the television interface is configured with one or more identifiers Information to characterize interface elements. As a result, the acquisition process can be made more convenient, thereby improving the user's interactive experience. The human-computer interaction method mentioned above can be applied to the television operation control method. For the configuration of the identification information, the acquisition process of the interface elements, and related details, please refer to the description with reference to FIG. 2 above, and will not be repeated here. .

Application example

The implementation process of the human-computer interaction scheme of the present disclosure will be described below using a smart TV desktop that provides a voice operation function as an example.

1. Display of the front-end interface

Before or during the TV desktop display, one or more identification information can be configured for multiple interface elements on the desktop. The identification information configured for each interface element can be stored as an array in association with the interface element. For example, a sequence number, a title, a color, a label of the interface element A, and a child element in the interface element A can be configured for the interface element A. Identification information.

After the TV desktop is displayed, corresponding identification information has been configured for multiple interface elements displayed on the desktop.

When the TV desktop changes, the identification information of the interface elements that are no longer displayed can be eliminated, and new identification information can be configured for the newly displayed interface elements.

As an example, the interface registration process is as follows.

First, the Page interface can be shown by the TV. Here, the TV can display the Page interface in accordance with its own interface display logic. The displayed Page interface can include pictures, text, numbers and other forms of display information. Among them, the text and numbers can be regarded as the logo of the picture, and the numbers can be It is determined according to the layout order of pictures in the display interface, and the text can be determined according to the content of the picture, such as the title of the picture. For example, the Page interface shown in FIG. 3 can be displayed, and the pictures, text, and numbers displayed in the interface can be regarded as a node (that is, the interface element mentioned above), and this node can be used as a corresponding information display entrance.

During the display of the Page interface, you need to register data services for the Page interface (that is, configure identification information for the interface elements). Specifically, a variety of voice input values (ie, identification information) to be opened can be set for each node in the interface through the server. For example, for node A (which can be an information display entrance or a function entrance), the serial number of node A, the interface text, the information text on the picture, and the information unique to node A can be registered as an array. This array is set for node A The opened voice input value of the user can realize the operation on the node A by speaking a voice consistent with the opened voice input value of the node A. Therefore, when the interface is opened and displayed, the voice input values of the nodes in the interface have been registered. When the interface has task elements changed, the registered content can be updated in real time to keep the registered content consistent with the interface displayed in real time.

2. Interactive process

When a user desires to perform a preset operation on an interface element in the TV desktop, the user can send out relevant voice information based on the degree of awareness or focus of the interface element.

The background service receives and parses the user's voice, converts the voice into text information, and matches the text information with the identification information of multiple interface elements in the TV desktop in a natural semantic manner to determine the interface elements that the user expects to perform the operation. The natural semantic matching mentioned here may be fuzzy matching. For example, when the user's voice data is Sansheng, the identification information of interface element 1 is "My Favorites", and the identification information of interface element 2 is "Sansheng III," then it matches interface element 2. In the case where a matching interface element is found on the TV desktop, a corresponding operation can be performed.

As shown in FIG. 3, not only can the visual identification information such as numbers and titles be configured for the pictures in the interface, but also multiple non-visual identification information can be configured according to the attribute information of the multiple characteristic dimensions of the pictures. For example, for the first picture, you can configure the identification information of "Trump" according to the content of the picture, that is, the name of the person; you can also configure the identification information of the picture according to the background color of the picture. According to the sub-elements contained in the picture, such as the Stars and Stripes, white-haired old man, and other identifying information. For another example, for the eighth picture (that is, the picture corresponding to the number 8), according to the position of the picture, the identification information of "second to second line" can be configured; or "fantasy", "fantasy", "Adventure", "Action", "Magic" and other identification information.

In addition, when configuring the identification information for the interface elements in the interface, you can also configure the corresponding identification information according to the specific label of the interface element. For example, you can configure the identification information of "highest score" for the highest-rated interface element in the display interface. It is also possible to configure the identification information of "most viewed" for the interface element with the most views in the display interface, and also to configure the identification information of "most viewed" for the interface element with the most collections in the display interface.

As a result, different users can enter "first picture", "morning news", "Trump", "Stars and Stripes", "white hair" by voice or text according to their own awareness of the interface elements or concerns. "Old man" and other semantic content, open the first picture to watch the corresponding video or information, which greatly facilitates the user's operation.

FIG. 4 is a schematic structural diagram of a human-machine interaction device according to an embodiment of the present disclosure. The functional modules of the human-computer interaction device 400 may be implemented by hardware, software, or a combination of hardware and software that implements the principles of the present invention. Those skilled in the art can understand that the functional modules described in FIG. 4 can be combined or divided into sub-modules, thereby realizing the principle of the above invention. Therefore, the description herein may support any possible combination, or division, or further definition of the functional modules described herein.

The following briefly describes the functional modules that the human-computer interaction device 400 can have and the operations that each functional module can perform. For the details involved, refer to the description above, and will not be repeated here.

Referring to FIG. 4, the human-machine interaction device 400 may include an acquisition module 420 and an execution module 430.

The obtaining module 420 is configured to obtain, in response to a user input, an interface element corresponding to the identification information matching the input in the display interface, wherein at least one interface element in the display interface is configured with one or more identification information to represent Interface elements. The execution module 430 is configured to perform a preset operation on the acquired interface element.

For interface elements and identification information, refer to the related descriptions above, and will not be repeated here.

As shown in FIG. 4, the human-machine interaction device may optionally further include a configuration module 410 shown by a dashed box in the figure. The configuration module 410 may be configured to configure one or more identification information for at least one interface element in the display interface to characterize the interface element, and in response to a change in the display interface, the configuration module 410 may also be a newly displayed one in the changed display interface. The interface element is configured with one or more of the identification information. In addition, as shown in FIG. 4, the human-computer interaction device 400 may optionally further include a deletion module 440 shown by a dashed box in the figure. In response to a change in the display interface, the deletion module 440 may be used to delete interface elements that are no longer displayed Identification information.

In the case where the input is a voice input, the human-computer interaction device 400 may optionally further include a first parsing module 450 shown by a dashed box in the figure, for parsing the voice input to obtain text information of the voice input. The obtaining module 420 may obtain an interface element corresponding to the identification information that matches the text information in the display interface.

When the interface element corresponds to a preset operation, the execution module 430 may perform a preset operation corresponding to the interface element on the acquired interface element. In the case where the interface element corresponds to multiple preset operations, the human-computer interaction device 400 may optionally further include a second parsing module 460 shown by a dashed box in the figure, for parsing the input to obtain the input instruction information. At this time, the execution module 430 may perform a preset operation matching the instruction information on the acquired interface element.

FIG. 5 is a schematic structural diagram of a television operation control device according to an embodiment of the present disclosure. The content of the human-computer interaction method and device mentioned above is also applicable to the smart TV operation control device. Therefore, the functional modules that the television operation control device 500 can have and the operations that can be performed by each functional module are briefly described below. For the details involved, refer to the description above, and will not be repeated here.

As shown in FIG. 5, the television operation control device 500 includes an acquisition module 510 and an execution module 520.

The obtaining module 510 is configured to obtain, in response to a user's input, an interface element corresponding to the identification information matching the input in a television interface (that is, a television desktop), where is at least one interface element in the television interface One or more identification information is configured to characterize the interface element. The interface element can be an information display entry or a function entry.

The execution module 520 is configured to perform a preset operation on the acquired interface element.

As shown in FIG. 5, the user's input may be a voice input, and the television operation control device 500 may optionally further include an analysis module 530 shown by a dashed box in the figure. The analysis module 530 is configured to parse the voice input to obtain the text information of the voice input, and the obtaining module 510 obtains the interface element corresponding to the identification information that matches the text information in the television interface.

Referring to FIG. 6, the computing device 600 includes a processor 610 and a memory 620. Regarding the memory and the processor, reference may be made to the description in conjunction with FIG. 1 above, and details are not described herein again.

The memory 620 stores a processable code, and when the processable code is processed by the processor 610, the processor 610 can execute the human-computer interaction method or the television operation control method mentioned above.

The human-machine interaction and television operation control method, apparatus, and device according to the present disclosure have been described in detail above with reference to the accompanying drawings.

In addition, the method according to the present disclosure may also be implemented as a computer program or computer program product including computer program code instructions for performing the above steps defined in the above method of the present disclosure.

Alternatively, the present disclosure may also be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. ), When the executable code (or computer program, or computer instruction code) is executed by a processor of an electronic device (or computing device, server, etc.), causing the processor to perform each step of the above-mentioned method according to the present disclosure .

Those skilled in the art will also appreciate that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, which contains one or more components for implementing a specified logical function Executable instructions. It should also be noted that in some alternative implementations, the functions labeled in the blocks may occur in a different order than those labeled in the figures. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified function or operation , Or it can be implemented with a combination of dedicated hardware and computer instructions.

The embodiments of the present disclosure have been described above, the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the various embodiments described. The terminology used herein is chosen to best explain the principles of the embodiments, practical applications or improvements to the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

A human-computer interaction method, comprising:

In response to a user input, an interface element corresponding to the identification information matching the input is obtained in a display interface, wherein at least one interface element in the display interface is configured with one or more identification information to characterize the Interface elements; and

Perform preset operations on the acquired interface elements.
The human-computer interaction method according to claim 1, wherein the interface element is an information display entrance or a function entrance.
The human-computer interaction method according to claim 2, wherein the interface element comprises one or more of the following:

Information;

image;

music;

video;

menu;

scroll bar.
The human-computer interaction method according to claim 1, wherein:

The identification information includes visual identification information,

The visual identification information is displayed in the display interface in association with a corresponding interface element.
The human-computer interaction method according to claim 1, wherein:

Each of the identification information characterizes the interface element with one of one or more feature dimensions.
The human-computer interaction method according to claim 1, wherein:

The one or more identification information is determined according to attribute information of the interface element corresponding to one or more feature dimensions.
The method of claim 6, wherein the attribute information comprises one or more of the following feature dimensions:

The name of the interface element;

The type of the interface element;

A display position of the interface element;

Display order of the interface elements;

The color of the interface element;

A label of the interface element;

The interface element contains child elements.
The human-machine interaction method according to claim 1, further comprising:

In response to a change in the display interface, one or more of the identification information is configured for a newly displayed interface element in the changed display interface.
The human-machine interaction method according to claim 1, further comprising:

In response to a change in the display interface, the identification information of the interface element that is no longer displayed is deleted.
The human-computer interaction method according to claim 1, wherein the input is a voice input, and the step of obtaining an interface element corresponding to the identification information matching the input in a display interface comprises:

Parse the voice input to obtain text information of the voice input;

An interface element corresponding to the identification information matching the text information is obtained in the display interface.
The human-computer interaction method according to claim 1, wherein the interface element corresponds to a preset operation, and the step of performing a preset operation on the acquired interface element comprises:

A preset operation corresponding to the interface element is performed on the acquired interface element.
The human-computer interaction method according to claim 1, wherein the interface element corresponds to a plurality of preset operations, and the step of performing a preset operation on the acquired interface element comprises:

Parse the input to obtain the input instruction information;

A preset operation matching the instruction information is performed on the acquired interface element.
A television operation control method, comprising:

In response to a user input, an interface element corresponding to the identification information matching the input is obtained in a television interface, wherein at least one interface element in the television interface is configured with one or more identification information to characterize the interface information. Interface elements; and

Perform preset operations on the acquired interface elements.
The television operation control method according to claim 13, wherein the interface element is an information display entrance or a function entrance.
The television operation control method according to claim 13, wherein the input is a voice input, and the step of obtaining an interface element corresponding to identification information matching the input in a television interface comprises:

Parse the voice input to obtain text information of the voice input;

Obtaining interface elements corresponding to the identification information matching the text information in the television interface.
A human-computer interaction device, comprising:

An obtaining module, configured to obtain, in response to a user input, an interface element corresponding to the identification information matching the input in a display interface, wherein at least one interface element in the display interface is configured with one or more Identifying information to characterize the interface element; and

The execution module is configured to perform preset operations on the acquired interface elements.
The human-computer interaction device according to claim 16, wherein the interface element is an information display entrance or a function entrance.
The human-computer interaction device according to claim 17, wherein the interface element comprises one or more of the following:

Information;

image;

music;

video;

menu;

scroll bar.
The human-computer interaction device according to claim 16, wherein:

The identification information includes visual identification information,

The visual identification information is displayed in the display interface in association with a corresponding interface element.
The human-computer interaction device according to claim 16, wherein:

Each of the identification information characterizes the interface element with one of one or more feature dimensions.
The human-computer interaction device according to claim 16, wherein:

The one or more identification information is determined according to attribute information of the interface element corresponding to one or more feature dimensions.
The human-computer interaction device according to claim 21, wherein the attribute information comprises one or more of the following feature dimensions:

The name of the interface element;

The type of the interface element;

A display position of the interface element;

Display order of the interface elements;

The color of the interface element;

A label of the interface element;

The interface element contains one or more child elements.
The human-computer interaction device according to claim 16, further comprising:

A configuration module is configured to configure one or more of the identification information for a newly displayed interface element in the changed display interface in response to a change in the display interface.
The human-computer interaction device according to claim 16, further comprising:

A deleting module is configured to delete identification information of interface elements that are no longer displayed in response to the display interface changing.
The human-computer interaction device according to claim 16, wherein the input is a voice input, and the device further comprises:

A first parsing module, configured to parse the voice input to obtain text information of the voice input,

The acquiring module acquires an interface element corresponding to the identification information that matches the text information in the display interface.
The human-computer interaction device according to claim 16, wherein:

The interface element corresponds to a preset operation, and the execution module executes the preset operation corresponding to the interface element with respect to the acquired interface element.
The human-computer interaction device according to claim 16, wherein the interface element corresponds to a plurality of preset operations, and the device further comprises:

A second parsing module, configured to parse the input to obtain the input instruction information,

The execution module executes a preset operation matching the instruction information for the acquired interface element.
A television operation control device, comprising:

An obtaining module, configured to obtain, in response to a user input, an interface element corresponding to the identification information matching the input in a television interface, wherein at least one interface element in the television interface is configured with one or more Identifying information to characterize the interface element; and

The execution module is configured to perform preset operations on the acquired interface elements.
The television operation control device according to claim 28, wherein the interface element is an information display entrance or a function entrance.
The television operation control device according to claim 28, wherein the input is a voice input, and the device further comprises:

A parsing module, configured to parse the voice input to obtain text information of the voice input,

The obtaining module obtains an interface element corresponding to the identification information that matches the text information in the television interface.
An electronic device, comprising:

A display for providing a display interface for a user, wherein the display interface includes one or more interface elements, and at least one interface element in the display interface is configured with one or more identification information to characterize the interface element ;

A memory for storing the identification information corresponding to the interface element; and

The processor is configured to obtain, in response to a user input, an interface element corresponding to the identification information matching the input in the display interface, and perform a preset operation on the acquired interface element.
A computing device, comprising:

Processor; and

The memory stores executable code thereon, and when the executable code is executed by the processor, causes the processor to execute the method according to any one of claims 1-15.
A non-transitory machine-readable storage medium having stored thereon executable code that, when executed by a processor of an electronic device, causes the processor to execute any one of claims 1 to 15 The method described.