CN117270732A - Control interaction method and device, electronic equipment and storage medium - Google Patents

Abstract

The present disclosure relates to a control interaction method, device, electronic device and storage medium. The method is applied to an extended reality XR terminal device. The method includes: displaying at least one control in the extended reality space; and displaying at least one control in the extended reality space. Determine a target control; display a visualized hot area corresponding to the target control; and perform a control operation corresponding to the target control in response to a triggering operation on the hot area corresponding to the target control. Since the selection/confirmation operation of the hot area is equivalent to the selection/confirmation operation of the control corresponding to the hot area, the difficulty of triggering the control can be reduced by rationally setting the hot area, thereby reducing the probability of accidental touch.

Description

A control interaction method, device, electronic equipment and storage medium

Technical field

The present disclosure relates to the field of extended reality technology, and in particular, to a control interaction method, device, electronic device, and storage medium.

Background technique

Extended Reality (XR for short) refers to the combination of reality and virtuality through computers to create a virtual environment that allows human-computer interaction. This is also the collective name for various technologies such as AR, VR, and MR. By integrating the three visual interaction technologies, the user can experience a "sense of immersion" that seamlessly transitions between the virtual world and the real world.

When users use extended reality devices, they usually need to trigger controls in the virtual reality space to achieve human-computer interaction. However, the current interaction method between users and controls is not reasonably designed and cannot assist users to quickly complete the interaction with controls.

Contents of the invention

In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a control interaction method, device, electronic device and storage medium.

In a first aspect, the present disclosure provides a control interaction method, which method is applied to an extended reality XR terminal device, and the method includes:

In the extended reality space, display at least one control;

determining a target control in the at least one control;

Display a visualized hot area corresponding to the target control;

In response to a triggering operation on a thermal area corresponding to the target control, a control operation corresponding to the target control is performed.

In a second aspect, the present disclosure also provides a control interaction device,

The device is applied to extended reality XR terminal equipment, and the device includes:

The first display module is used to display at least one control in the extended reality space;

a determining module, configured to determine a target control in the at least one control;

The second display module is used to display the visualized hot area corresponding to the target control;

An execution module, configured to execute a control operation corresponding to the target control in response to a triggering operation on the hot area corresponding to the target control.

In a third aspect, the present disclosure also provides an electronic device, which includes:

one or more processors;

A storage device for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the control interaction method as described above.

In a fourth aspect, the present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the control interaction method as described above is implemented.

Compared with the existing technology, the technical solution provided by the embodiments of the present disclosure has the following advantages:

The technical solution provided by the embodiment of the present disclosure determines a target control in at least one control; displays a visualized hot area corresponding to the target control; and performs a control operation corresponding to the target control in response to a triggering operation on the hot area corresponding to the target control. , since the triggering operation of the hot area is equivalent to the triggering operation of the control corresponding to the hot area, the difficulty of triggering the control can be reduced by reasonably setting the hot area, thereby reducing the probability of accidental touch and assisting the user to quickly complete the interaction with the control. purpose of interaction.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those of ordinary skill in the art, It is said that other drawings can be obtained based on these drawings without exerting creative labor.

Figure 1 is a structural block diagram of an extended reality terminal device provided by an embodiment of the present disclosure;

Figure 2 is a schematic diagram of an application scenario of a helmet-mounted display provided by an embodiment of the present disclosure;

Figure 3 is a flow chart of a control interaction method provided by an embodiment of the present disclosure;

Figure 4 is a schematic diagram of the corresponding relationship between a thermal area and a control provided by an embodiment of the present disclosure;

Figure 5 is a schematic diagram of another corresponding relationship between thermal areas and controls provided by an embodiment of the present disclosure;

Figure 6 is a flow chart of another control interaction method provided by an embodiment of the present disclosure;

Figures 7 and 8 are schematic diagrams of the corresponding relationships between two other thermal areas and controls provided by embodiments of the present disclosure;

Figure 9 is a schematic structural diagram of a control interaction device in an embodiment of the present disclosure;

Figure 10 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure.

Detailed ways

In order to understand the above objects, features and advantages of the present disclosure more clearly, the solutions of the present disclosure will be further described below. It should be noted that, as long as there is no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

Many specific details are set forth in the following description to fully understand the present disclosure, but the present disclosure can also be implemented in other ways different from those described here; obviously, the embodiments in the description are only part of the embodiments of the present disclosure, and Not all examples.

Extended Reality (XR for short) refers to the combination of reality and virtuality through computers to create a virtual environment that allows human-computer interaction. This is also the collective name for various technologies such as AR, VR, and MR. By integrating the three visual interaction technologies, the user can experience a "sense of immersion" that seamlessly transitions between the virtual world and the real world.

Extended reality terminal equipment is a terminal that can achieve extended reality effects. It can usually be provided in the form of glasses, helmet-mounted displays (HMD), and contact lenses to achieve visual perception and other forms of perception. Of course, virtual The form of actual device implementation is not limited to this, and can be further miniaturized or enlarged as needed.

Extended reality terminal devices can create virtual scenes. A virtual scene is a virtual scene displayed (or provided) by an application when it is run on an electronic device. The virtual scene can be a simulation environment of the real world, a semi-simulation and semi-fictional virtual scene, or a purely fictitious virtual scene. The virtual scene may be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene, or a three-dimensional virtual scene. The embodiments of this application do not limit the dimensions of the virtual scene.

Figure 1 is a structural block diagram of an extended reality terminal device provided by an embodiment of the present disclosure. Referring to Figure 1, for example, the extended reality terminal device is a helmet mounted display (Head Mount Display, HMD). Its main functional modules can include but are not limited to the following components: 1) Detection (module): Use various sensors to detect the user's operation commands and act on the virtual environment, such as following the user's line of sight and continuously updating the information displayed on the display screen. Imaging, enabling users to interact with virtual and scenes, such as continuously updating real-life content based on the detected rotation direction of the user's head; 2) Feedback (module): Receive data from sensors and provide real-time feedback to users; 3) Sensor : On the one hand, it accepts the operation commands from the user and applies them to the virtual environment; on the other hand, it provides the results of the operation to the user in the form of various feedbacks; 4) Control module: for sensors and various input/outputs The device controls, including obtaining the user's data (such as actions, voice) and outputting sensory data, such as images, vibrations, temperature and sound, etc., which have an effect on the user, the virtual environment and the real world; 5) Modeling module: construct a virtual environment The three-dimensional model can also include various feedback mechanisms such as sound and touch in the three-dimensional model.

In a virtual reality scene, the user can select a target object through a controller, which can be a handle, and the user selects the target object by operating the buttons of the handle. Of course, in other embodiments, the target object in the extended reality terminal device may be controlled using gestures or voice instead of using a controller. Among them, the target object is an interactive object in the virtual scene and is controlled by a user or a robot program (for example, a robot program based on artificial intelligence). For example, the target object is a control.

Head-mounted displays are relatively lightweight, ergonomically comfortable, and provide high-resolution content with low latency. FIG. 2 is a schematic diagram of an application scenario of a helmet-mounted display provided by an embodiment of the present disclosure. Referring to Figure 2, the virtual reality device is equipped with a posture detection sensor (such as a nine-axis sensor) for real-time detection of posture changes of the virtual reality device. If the user wears the virtual reality device, then when the user's head posture changes, The real-time posture of the head will be transmitted to the processor to calculate the gaze point of the user's line of sight in the virtual environment. Based on the gaze point, the image in the three-dimensional model of the virtual environment within the user's gaze range (i.e., the virtual field of view) will be calculated, and Displayed on the display screen, it gives people an immersive experience as if they are watching in a real environment.

Currently, the triggering of controls (including selection and/or confirmation) is accomplished by emitting laser light from the controller or by touching it directly with hands. The way the laser is fired through the controller is erratic and unintuitive. It is very easy to accidentally touch it by touching it directly with your hands. Therefore, neither of the above two methods can assist the user to quickly complete the interaction with the control.

In response to this, FIG. 3 is a flow chart of a control interaction method provided by an embodiment of the present disclosure, and the method is applied to an extended reality XR terminal device. As shown in Figure 3, the method may specifically include:

S110. Display at least one control in the extended reality space.

In this application, the control may be a two-dimensional control or a three-dimensional image, which is not limited in this application.

S120. Determine the target control in at least one control.

Optionally, the target control is a control triggered by user intent.

If the target control is a control triggered by the user's intention, there are many specific implementation methods for this step, and this disclosure does not limit this. Exemplarily, corresponding user intention judgment conditions are set for each control in advance. The specific implementation method of this step includes: detecting whether each user intention judgment condition is satisfied; if a certain user intention judgment condition is satisfied, the user's intention judgment condition is satisfied. The control corresponding to the intent judgment condition is used as the target control.

Furthermore, in practice, the user intention judgment conditions corresponding to different controls may be the same or different, and this application does not limit this.

In practice, since users may trigger controls through multiple triggering methods (such as triggering by controllers, triggering by gestures, triggering by voice, etc.), different settings can be set for the same control based on different triggering methods. User intent judgment conditions.

Further, corresponding to the same triggering method of the same control, the user intention judgment condition can be set to include one or more sub-judgment conditions.

In one embodiment, for triggering a control through the triggering method of the controller, the user intention information is the distance between the laser emitted by the controller and the control, and the user intention judgment condition is that the distance between the laser emitted by the controller and the control is less than First set the distance threshold. If the user intention information satisfies the user intention judgment condition, it is determined that the user intention triggers the control, and the control is determined as the target control.

In another embodiment, for triggering a control through gesture triggering, the user intention information is the distance between the user's hand and the control, and the user intention determination condition is that the distance between the user's hand and the control is less than the second set distance threshold. . If the user intention information satisfies the user intention judgment condition, it is determined that the user intention triggers the control, and the control is determined as the target control.

In another embodiment, for triggering a control through gesture triggering, the user intention information is the distance between the user's gaze point and the control, and the user intention determination condition is that the distance between the user's gaze point and the control is less than the third set distance threshold. . If the user intention information satisfies the user intention judgment condition, it is determined that the user intention triggers the control, and the control is determined as the target control.

In another embodiment, for triggering a certain control through voice triggering, the user intention information is the distance between the user's gaze point and the control, and the user intention judgment condition is that the distance between the user's gaze point and the control is less than the fourth set distance threshold. . If the user intention information satisfies the user intention judgment condition, it is determined that the user intention triggers the control, and the control is determined as the target control.

S130. Display the visualized hot area corresponding to the target control.

In one embodiment, the size of the hot area is larger than the size of the target control.

A hot area refers to an area that can represent a control. The user's triggering operation (including selection/confirmation operation) on the hot area is equivalent to the selection/confirmation operation of the control corresponding to the hot area.

A visualized thermal area means that the thermal area can be seen by the user. In other words, the user can visually perceive the location and boundary of the hot area in the extended reality space.

S140. In response to the triggering operation on the hot area corresponding to the target control, execute the control operation corresponding to the target control.

If the user triggers through the controller, the triggering operation for the hot area means pressing the button on the controller after entering the hot area. If the user triggers through gestures, the trigger operation for the hot area refers to completing the specified action (such as a click action) after the user's hand enters the hot area. If the user triggers through voice, the triggering operation for the hot area means that the user describes the identification information of the hot area and the name of the triggering action through voice. The identification information of the hot area refers to the information that can uniquely determine the hot area that the user wants to trigger from among multiple hot areas. For example, the identification information of the hot area includes but is not limited to the color, shape, position, number, Name, as well as the color, shape, position, number, name, etc. of the target control corresponding to the hot area. Trigger action names include but are not limited to open, open, enter, enter, click, click, select, etc.

For example, if the function of control a is to open video A in the video player, when the user triggers the hot area corresponding to control a, video A will be opened in the video player.

The above technical solution determines the target control in at least one control; displays the visualized hot area corresponding to the target control; responds to the triggering operation of the hot area corresponding to the target control, and performs the control operation corresponding to the target control. The triggering operation is equivalent to the triggering operation of the control corresponding to the hot area. By reasonably setting the hot area, the difficulty of triggering the control can be reduced, thereby reducing the probability of accidental touch and assisting the user to quickly complete the interaction with the control.

In one embodiment, the hot zone and its corresponding control are at least partially coincident. This setting can facilitate users to quickly identify the corresponding relationship between hot areas and controls, reducing the chance of accidental triggering.

In the extended reality space, controls are mainly divided into two categories: one is 2D controls and the other is 3D controls.

The thermal area corresponding to the 2D control can be a plane graphic or a three-dimensional graphic, and this application does not limit this. If the hot area is a flat graphic, the hot area and its corresponding control at least partially overlap, which means that the hot area and its corresponding control are located on the same plane, and the outline graphic of the hot area and the outline graphic of its corresponding control at least partially overlap. If the thermal area is a three-dimensional figure, the thermal area and its corresponding control at least partially overlap, which means that the plane where the control is located passes through the corresponding thermal area, and the hot area is intercepted by the plane where the control is located, and the resulting cross-sectional outline graphic is the same as the outline of the control The figures at least partially overlap. Exemplarily, FIG. 4 is a schematic diagram of a corresponding relationship between a thermal area and a control provided by an embodiment of the present disclosure. In Figure 4, the control is a button with text information of 5. Referring to Figure 4, if the control is a 2D control, the thermal area is cylindrical, and one bottom surface of the thermal area is in the plane where the control is located, and at least partially coincides with the area occupied by the control.

The thermal area corresponding to the 3D control can be a plane graphic or a three-dimensional graphic, and this application does not limit this. If the hot area is a plane figure, the hot area and its corresponding control at least partially overlap, which means that the plane where the hot area is located passes through the corresponding control, and the control is intercepted by the plane where the hot area is located, and the outline of the section obtained is the same as the thermal area. The contour figures at least partially overlap. If the hot area is a three-dimensional figure, the hot area and its corresponding control at least partially overlap, which means that in space, the portion where the hot area and its corresponding control intersect forms a three-dimensional structure. Exemplarily, FIG. 5 is a schematic diagram of another corresponding relationship between thermal areas and controls provided by an embodiment of the present disclosure. Referring to Figure 5, if the control is a 3D control, the shape of the hot area is a cube or a sphere, and the control is located in the hot area.

Further, the hot area is a three-dimensional figure or a plane figure, and the target control is located in the hot area. This setting can ensure that users can quickly clarify the corresponding relationship between hot areas and controls, and further assist users to quickly complete the purpose of interacting with controls.

Figure 6 is a flow chart of another control interaction method provided by an embodiment of the present disclosure. Figure 6 is a specific example of Figure 3. Referring to Figure 6, the control interaction methods include:

S210. Display at least one control in the extended reality space.

S220. Determine the target control in at least one control.

S230. Based on the preset rules, determine the size and position of the hot area corresponding to the target control.

Preset rules are information preset by developers to determine the location and boundaries of thermal areas in the extended reality space.

Optionally, the preset rules can specify whether the thermal area corresponding to the control is a plane graphic or a three-dimensional graphic, the relative position of the thermal area and the control, and the size of the thermal area boundary line.

Optionally, you can also set different types of controls to correspond to different preset rules.

S240: Render the color and/or outline of the hot area based on the size and position of the hot area.

The purpose of rendering the color and/or outline of a hot area is to enable the user to visually perceive the position and boundary of the hot area in the extended reality space.

In one embodiment, if the visualized hot area is a color-rendered hot area, the color of the hot area is different from the background color of the extended reality space and the color of the control; and/or, if the visualized hot area is a The thermal area after the outline is rendered. The color of the thermal area outline is different from the background color of the extended reality space and the color of the control. This setting can help users visually perceive the location and boundaries of the hot area in the extended reality space.

S250. Display the rendered thermal area.

S260. In response to the triggering operation on the hot area corresponding to the target control, execute the control operation corresponding to the target control.

The above technical solution determines the size and position of the hot area corresponding to the target control by setting preset rules; renders the color and/or outline of the hot area based on the size and position of the hot area; displays the rendered hot area , which allows users to intuitively perceive the location and boundaries of the hot area in the extended reality space through vision, which facilitates subsequent triggering of the hot area accurately and quickly.

Based on the above technical solutions, optionally, in response to the instruction of selecting the hot area, information prompting that the hot area is selected is displayed.

If the user triggers it through the controller, the command for selecting the hot zone refers to the command generated when the controller enters the hot zone and does not press the button on the controller. If the user triggers it through a gesture, the instruction for the hot area to be selected refers to the instruction generated when the user's hand enters the hot area but does not complete the specified action (such as a click action). If the user triggers through voice, the triggering operation for the hot area refers to the instruction generated when the user speaks the identification information describing the hot area but does not say the name of the triggering action. In other words, when the user performs a trigger operation on the hot area corresponding to the target control, a total of two instructions are generated, one is the instruction for the hot area to be selected (i.e., the selection instruction), and the other is the instruction for the hot area to be triggered (i.e., the confirmation instruction). The instruction that the hot area is selected is generated before the instruction that the hot area is triggered. Subsequently, in response to the instruction that the hot area is selected, information is displayed to prompt that the hot area is selected. This information is used to indicate that if you keep the current position, press a button on the controller, or complete a specified action (such as a click action) , or tell the name of the trigger action, which control will be triggered, and then understand whether the user's expected purpose can be achieved. In response to the instruction that the hot zone is triggered, a control operation associated with the control corresponding to the triggered hot zone is performed.

In one embodiment, in response to the instruction that the hot area is selected, displaying information for prompting that the hot area is selected includes: in response to the instruction that the hot area is selected, changing the color of the hot area and/or the outline of the hot area. color. After the controller or the user's hand enters the hot area, or after the user's voice command hits the identification information of the hot area, the color of the hot area and/or the color of the hot area outline changes, and the user can intuitively determine the current situation through the change in color. Which control is selected further reduces the difficulty of triggering the control.

In another embodiment, in response to the instruction that the hot area is selected, displaying information for prompting that the hot area is selected includes: in response to the instruction that the hot area is selected, displaying a dynamic effect of a preset image entering the hot area. The preset image may specifically be a controller graphic, a user hand graphic, or other pre-specified graphics (such as an arrow graphic, a circle image, etc.). Users can intuitively determine which control is currently selected by entering the animation of the hot area through controller graphics, user hand graphics or other pre-specified graphics (such as arrow graphics, circle images, etc.), further reducing the difficulty of triggering the control. the goal of. Figures 7 and 8 are schematic diagrams of the corresponding relationships between two other thermal areas and controls provided by embodiments of the present disclosure. Among them, Figure 7 corresponds to Figure 4, and Figure 8 corresponds to Figure 5. Comparing Figure 4 and Figure 7, or comparing Figure 5 and Figure 8, by showing the controller graphic entering the hot zone, the user can quickly determine whether the controller has entered the hot zone.

It should be noted that for the sake of simple description, the foregoing method embodiments are expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence. Because in accordance with the present invention, certain steps may be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily necessary for the present invention.

Figure 9 is a schematic structural diagram of a control interaction device in an embodiment of the present disclosure. The device is applied to extended reality XR terminal equipment. Referring to Figure 9, the control interaction device specifically includes:

The first display module 310 is used to display at least one control in the extended reality space;

Determining module 320, configured to determine a target control in the at least one control;

The second display module 330 is used to display the visualized hot area corresponding to the target control;

The execution module 340 is configured to execute a control operation corresponding to the target control in response to a triggering operation on the hot area corresponding to the target control.

Further, the size of the thermal area is larger than the size of the target control.

Further, the hot area and the corresponding target control at least partially overlap.

Further, the thermal area is a three-dimensional figure, and the target control is located in the thermal area.

Further, the second display module 330 is used for:

Based on preset rules, determine the size and position of the thermal area corresponding to the target control;

Rendering the color and/or outline of the thermal area based on the size and location of the thermal area;

Shows the thermal area rendered.

Further, if the visualized hot area is a hot area after color rendering, the color of the hot area is different from the background color of the extended reality space and the color of the control; and/or,

If the visualized hot area is a hot area after the outline is rendered, the color of the outline of the hot area is different from the background color of the extended reality space and the color of the control.

Furthermore, it also includes a third display module for:

In response to the instruction that the hot area is selected, information prompting that the hot area is selected is displayed.

Furthermore, it also includes a third display module for:

In response to an instruction that the thermal area is selected, the color of the thermal area and/or the color of the thermal area outline is changed.

Furthermore, it also includes a third display module for:

In response to the instruction of selecting the hot area, a dynamic effect of a preset graphic entering the hot area is displayed.

The control interaction device provided by the embodiment of the present disclosure can perform the steps of the control interaction method provided by the embodiment of the present disclosure, and has the same or corresponding beneficial effects, which will not be described again here.

Figure 10 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. Referring specifically to FIG. 10 below, a schematic structural diagram of an electronic device 1000 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 1000 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals ( Mobile terminals such as vehicle navigation terminals), wearable electronic devices, etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, etc. The electronic device shown in FIG. 10 is only an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

As shown in FIG. 10 , the electronic device 1000 may include a processing device (eg, central processing unit, graphics processor, etc.) 1001 , which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 1002 or from a storage device 1008 . The program in the memory (RAM) 1003 performs various appropriate actions and processes to implement the control interaction method according to the embodiments of the present disclosure. In the RAM 1003, various programs and information required for the operation of the electronic device 1000 are also stored. The processing device 1001, the ROM 1002 and the RAM 1003 are connected to each other via a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.

Generally, the following devices may be connected to the I/O interface 1005: input devices 1006 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speakers, vibration An output device 1007 such as a computer; a storage device 1008 including a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 may allow the electronic device 1000 to communicate wirelessly or wiredly with other devices to exchange information. Although FIG. 10 illustrates electronic device 1000 with various means, it should be understood that implementation or availability of all illustrated means is not required. More or fewer means may alternatively be implemented or provided.

In particular, according to embodiments of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, the computer program including program code for executing the method shown in the flowchart, thereby achieving the above The control interaction method described. In such embodiments, the computer program may be downloaded and installed from the network via communication device 1009, or from storage device 1008, or from ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the above two. The computer-readable storage medium may be, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination thereof. More specific examples of computer readable storage media may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard drive, random access memory (RAM), read only memory (ROM), removable Programmed read-only memory (EPROM or flash memory), fiber optics, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include an information signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code therein. Such propagated information signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device . Program code embodied on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wire, optical cable, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server can communicate using any known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital information in any form or medium ( For example, communication network) interconnection. Examples of communications networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any known or future developed network.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; it may also exist independently without being assembled into the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device:

In the extended reality space, display at least one control;

determining a target control in the at least one control;

Display a visualized hot area corresponding to the target control;

In response to a triggering operation on a thermal area corresponding to the target control, a control operation corresponding to the target control is performed.

Optionally, when one or more of the above programs are executed by the electronic device, the electronic device may also perform other steps described in the above embodiments.

Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages—such as "C" or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In situations involving remote computers, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as an Internet service provider through Internet connection).

The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operations of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, segment, or portion of code that contains one or more logic functions that implement the specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown one after another may actually execute substantially in parallel, or they may sometimes execute in the reverse order, depending on the functionality involved. It will also be noted that each block of the block diagram and/or flowchart illustration, and combinations of blocks in the block diagram and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or operations. , or can be implemented using a combination of specialized hardware and computer instructions.

The units involved in the embodiments of the present disclosure can be implemented in software or hardware. Among them, the name of a unit does not constitute a limitation on the unit itself under certain circumstances.

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, including:

one or more processors;

Memory, used to store one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement any of the control interaction methods provided by this disclosure.

According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the control interaction as described in any one provided by the present disclosure is implemented. method.

Embodiments of the present disclosure also provide a computer program product. The computer program product includes a computer program or instructions. When the computer program or instructions are executed by a processor, the control interaction method as described above is implemented.

It should be noted that in this article, relational terms such as “first” and “second” are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these There is no such actual relationship or sequence between entities or operations. Furthermore, the terms "comprises," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also those not expressly listed other elements, or elements inherent to the process, method, article or equipment. Without further limitation, an element defined by the statement "comprises a..." does not exclude the presence of additional identical elements in a process, method, article, or apparatus that includes the stated element.

The above descriptions are only specific embodiments of the present disclosure, enabling those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be practiced in other embodiments without departing from the spirit or scope of the disclosure. Therefore, the present disclosure is not to be limited to the embodiments described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (12)

1. A control interaction method, characterized in that the method is applied to an augmented reality XR terminal device, the method comprising:

displaying at least one control in the augmented reality space;

determining a target control in the at least one control;

displaying a visualized hot area corresponding to the target control;

and responding to the triggering operation of the thermal area corresponding to the target control, and executing the control operation corresponding to the target control.

2. The method of claim 1, wherein a size of the hot zone is greater than a size of the target control.

3. The method of claim 1, wherein the hot zone and the target control corresponding thereto at least partially coincide.

4. The method of claim 3, wherein the hot zone is a solid or planar graphic and the target control is located within the hot zone.

5. The method of claim 1, wherein the exposing the visualized hot area corresponding to the target control comprises:

determining the size and the position of a thermal area corresponding to the target control based on a preset rule;

rendering a color and/or contour of the thermal region based on the size and location of the thermal region;

the rendered hot region is shown.

6. The method of claim 5, wherein if the visualized hot area is a hot area after rendering the color, the color of the hot area is different from the background color of the augmented reality space and the color of the control; and/or the number of the groups of groups,

if the visualized hot area is the hot area after the outline is rendered, the color of the outline of the hot area is different from the background color of the augmented reality space and the color of the control.

7. The method as recited in claim 1, further comprising:

and responding to the selected instruction of the hot area, and displaying information for prompting the selected hot area.

8. The method of claim 7, wherein the presenting information prompting the hot zone to be selected in response to the instruction that the hot zone is selected comprises:

in response to the selected instruction of the thermal region, changing the color of the thermal region and/or the color of the thermal region contour.

9. The method of claim 7, wherein responsive to the instruction that the hot zone is selected, presenting information prompting that the hot zone is selected comprises:

and responding to the selected instruction of the hot area, and displaying the dynamic effect of entering the hot area by a preset graph.

10. A control interaction device, wherein the device is applied to an augmented reality XR terminal device, the device comprising:

the first display module is used for displaying at least one control in the augmented reality space;

the determining module is used for determining a target control in the at least one control;

the second display module is used for displaying the visualized hot area corresponding to the target control;

and the execution module is used for responding to the triggering operation of the thermal area corresponding to the target control and executing the control operation corresponding to the target control.

11. An electronic device, the electronic device comprising:

one or more processors;

a storage means for storing one or more programs;

the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9.

12. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-9.