CN117270732A - Control interaction method and device, electronic equipment and storage medium - Google Patents
Control interaction method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN117270732A CN117270732A CN202210680232.0A CN202210680232A CN117270732A CN 117270732 A CN117270732 A CN 117270732A CN 202210680232 A CN202210680232 A CN 202210680232A CN 117270732 A CN117270732 A CN 117270732A
- Authority
- CN
- China
- Prior art keywords
- control
- hot
- hot area
- target control
- color
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 55
- 230000003993 interaction Effects 0.000 title claims abstract description 41
- 230000003190 augmentative effect Effects 0.000 claims abstract description 36
- 238000004590 computer program Methods 0.000 claims description 12
- 230000004044 response Effects 0.000 claims description 10
- 238000009877 rendering Methods 0.000 claims description 8
- 230000000694 effects Effects 0.000 claims description 6
- 239000007787 solid Substances 0.000 claims description 3
- 238000012790 confirmation Methods 0.000 abstract description 4
- 238000010586 diagram Methods 0.000 description 19
- 230000009471 action Effects 0.000 description 14
- 230000001960 triggered effect Effects 0.000 description 8
- 238000004891 communication Methods 0.000 description 6
- 230000006870 function Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000005516 engineering process Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000007654 immersion Methods 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 238000004088 simulation Methods 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 206010034719 Personality change Diseases 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 230000016776 visual perception Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- Software Systems (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The disclosure relates to a control interaction method, a control interaction device and an electronic device as storage media, wherein the method is applied to an augmented reality XR terminal device, and the method comprises the following steps: displaying at least one control in the augmented reality space; determining a target control in the at least one control; displaying a visualized hot area corresponding to the target control; and responding to the triggering operation of the thermal area corresponding to the target control, and executing the control operation corresponding to the target control. The selection/confirmation operation of the hot area is equal to the selection/confirmation operation of the control corresponding to the hot area, so that the difficulty in triggering the control can be reduced by reasonably setting the hot area, and the aim of reducing the false touch probability is fulfilled.
Description
Technical Field
The disclosure relates to the technical field of augmented reality, and in particular relates to a control interaction method, a control interaction device and electronic equipment as well as storage media.
Background
Extended Reality (XR for short) refers to that a virtual environment capable of man-machine interaction is created by combining Reality with virtual through a computer, which is also collectively called as multiple technologies such as AR, VR, MR and the like. By integrating the visual interaction technologies of the three, the method brings the 'immersion' of seamless transition between the virtual world and the real world for the experienter.
When a user uses an augmented reality device, human-machine interaction is typically required by triggering controls within the virtual reality space. However, the current interaction mode of the user and the control is unreasonable in design, and the user cannot be assisted to complete the interaction with the control quickly.
Disclosure of Invention
In order to solve the technical problems described above or at least partially solve the technical problems described above, the disclosure provides a control interaction method, a control interaction device, an electronic device and a storage medium.
In a first aspect, the present disclosure provides a control interaction method, where the method is applied to an augmented reality XR terminal device, the method includes:
displaying at least one control in the augmented reality space;
determining a target control in the at least one control;
displaying a visualized hot area corresponding to the target control;
and responding to the triggering operation of the thermal area corresponding to the target control, and executing the control operation corresponding to the target control.
In a second aspect, the present disclosure also provides a control interaction device,
the apparatus is applied to an augmented reality XR terminal device, the apparatus comprising:
the first display module is used for displaying at least one control in the augmented reality space;
the determining module is used for determining a target control in the at least one control;
the second display module is used for displaying the visualized hot area corresponding to the target control;
and the execution module is used for responding to the triggering operation of the thermal area corresponding to the target control and executing the control operation corresponding to the target control.
In a third aspect, the present disclosure also provides an electronic device, including:
one or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the control interaction method as described above.
In a fourth aspect, the present disclosure also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a control interaction method as described above.
Compared with the prior art, the technical scheme provided by the embodiment of the disclosure has the following advantages:
the technical scheme provided by the embodiment of the disclosure is that a target control is determined in at least one control; displaying a visualized hot area corresponding to the target control; and responding to the triggering operation of the hot area corresponding to the target control, and executing the control operation corresponding to the target control, wherein the triggering operation of the hot area is equivalent to the triggering operation of the control corresponding to the hot area, so that the difficulty in triggering the control can be reduced by reasonably setting the hot area, further the aim of reducing the false touch probability and assisting a user to quickly complete the interaction with the control is fulfilled.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.
In order to more clearly illustrate the embodiments of the present disclosure or the solutions in the prior art, the drawings that are required for the description of the embodiments or the prior art will be briefly described below, and it will be obvious to those skilled in the art that other drawings can be obtained from these drawings without inventive effort.
Fig. 1 is a block diagram of an augmented reality terminal device according to an embodiment of the present disclosure;
fig. 2 is a schematic diagram of an application scenario of a head mounted display according to an embodiment of the disclosure;
FIG. 3 is a flowchart of a control interaction method provided in an embodiment of the present disclosure;
fig. 4 is a schematic diagram of a correspondence relationship between a hot area and a control according to an embodiment of the disclosure;
fig. 5 is a schematic diagram of a correspondence relationship between another hot area and a control according to an embodiment of the disclosure;
FIG. 6 is a flowchart of another control interaction method provided by an embodiment of the present disclosure;
fig. 7 and 8 are schematic diagrams of correspondence between two other hot areas and controls provided in an embodiment of the present disclosure;
FIG. 9 is a schematic structural diagram of a control interaction device in an embodiment of the disclosure;
fig. 10 is a schematic structural diagram of an electronic device in an embodiment of the disclosure.
Detailed Description
In order that the above objects, features and advantages of the present disclosure may be more clearly understood, a further description of aspects of the present disclosure will be provided below. It should be noted that, without conflict, the embodiments of the present disclosure and features in the embodiments may be combined with each other.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced otherwise than as described herein; it will be apparent that the embodiments in the specification are only some, but not all, embodiments of the disclosure.
Extended Reality (XR for short) refers to that a virtual environment capable of man-machine interaction is created by combining Reality with virtual through a computer, which is also collectively called as multiple technologies such as AR, VR, MR and the like. By integrating the visual interaction technologies of the three, the method brings the 'immersion' of seamless transition between the virtual world and the real world for the experienter.
The augmented reality terminal device is a terminal capable of realizing an augmented reality effect, and is generally provided in the form of glasses, a head mounted display (Head Mount Display, HMD), or a contact lens for realizing visual perception and other forms of perception, but the form of realizing the virtual reality device is not limited to this, and may be further miniaturized or enlarged as needed.
The augmented reality terminal device may create a virtual scene. A virtual scene is a virtual scene that an application program displays (or provides) when running on an electronic device. The virtual scene may be a simulation environment for the real world, a semi-simulation and semi-fictional virtual scene, or a pure fictional virtual scene. The virtual scene may be any one of a two-dimensional virtual scene, a 2.5-dimensional virtual scene or a three-dimensional virtual scene, and the dimension of the virtual scene is not limited in the embodiment of the present application.
Fig. 1 is a block diagram of an augmented reality terminal device according to an embodiment of the present disclosure. Referring to fig. 1, the augmented reality terminal device is illustratively a head mounted display (Head Mount Display, HMD). The main functional modules thereof can include, but are not limited to, the following components: 1) Detection (module): detecting operation commands of a user by using various sensors, and acting on a virtual environment, such as continuously updating images displayed on a display screen along with the sight of the user, so as to realize interaction between the user and the virtual and scene, for example, continuously updating real content based on the detected rotation direction of the head of the user; 2) Feedback (module): receiving data from the sensor, providing real-time feedback to the user; 3) A sensor: on one hand, an operation command from a user is accepted and acted on the virtual environment; on the other hand, the result generated after the operation is provided for the user in various feedback forms; 4) And the control module is used for: controlling the sensors and various input/output devices, including obtaining user data (e.g., motion, speech) and outputting sensory data, such as images, vibrations, temperature, sounds, etc., to effect the user, virtual environment, and the real world; 5) Modeling module: constructing a three-dimensional model of a virtual environment may also include various feedback mechanisms such as sound, touch, etc. in the three-dimensional model.
In the virtual reality scenario, the user selecting the target object may be implemented by a controller, which may be a handle, and the user selects the target object by operating a key of the handle. Of course, in other embodiments, the target object in the augmented reality terminal device may be controlled using gestures or voice instead of using the controller. Wherein the target object is an object in the virtual scene that interacts, under the control of a user or a robot program (e.g., an artificial intelligence based robot program). For example, the target object is a control.
Head mounted displays are relatively light, ergonomically comfortable, and provide high resolution content with low latency. Fig. 2 is a schematic diagram of an application scenario of a head mounted display according to an embodiment of the present disclosure. Referring to fig. 2, a sensor (e.g., a nine-axis sensor) for detecting an attitude of the virtual reality device is provided in the virtual reality device, and is used for detecting an attitude change of the virtual reality device in real time, if the user wears the virtual reality device, when the head attitude of the user changes, the real-time attitude of the head is transmitted to the processor, so as to calculate a gaze point of the user in the virtual environment, and an image in a user gaze range (i.e., a virtual field of view) in a three-dimensional model of the virtual environment is calculated according to the gaze point, and is displayed on the display screen, so that a person looks like looking in the real environment.
Currently, triggering of the control (including selection and/or confirmation) is accomplished by means of a controller that emits a laser or by means of direct touching by hand. The manner in which the laser is emitted by the controller is unstable and not intuitive. The direct touch mode by hand is extremely easy to touch by mistake. Therefore, the two modes cannot assist the user to quickly complete the interaction with the control.
For this, fig. 3 is a flowchart of a control interaction method provided in an embodiment of the present disclosure, where the method is applied to an augmented reality XR terminal device. As shown in fig. 3, the method specifically may include:
s110, displaying at least one control in the augmented reality space.
In the present application, the control may be a two-dimensional control or a three-dimensional image, which is not limited in this application.
S120, determining a target control in at least one control.
Optionally, the target control is a control that the user intends to trigger.
If the target control is a control which is intended to be triggered by the user, the specific implementation method of the step is various, and the disclosure is not limited to the specific implementation method. The specific implementation method of the step comprises the following steps of: detecting intention of each user to judge whether the condition is satisfied; and if the user intention judging condition is met, taking the control corresponding to the user intention judging condition as a target control.
Further, in practice, the user intention judgment conditions corresponding to different controls may be the same or different, which is not limited in this application.
In practice, since the user may trigger the control in multiple trigger modes (such as a trigger mode through the controller, a trigger mode through a gesture, a trigger mode through voice, etc.), different user intention judgment conditions can be set for the same control according to different trigger modes.
Further, corresponding to the same trigger manner of the same control, it may be set that the user intention judgment condition includes one or more sub judgment conditions.
In one embodiment, for triggering a control in a triggering manner of the controller, the user intention information is a distance between the laser emitted by the controller and the control, and the user intention judging condition is that the distance between the laser emitted by the controller and the control is smaller than a first set distance threshold. If the user intention information meets the user intention judging condition, determining that the user intention triggers the control, and determining the control as a target control.
In another embodiment, for triggering a control in a gesture triggering manner, the user intention information is a distance between a user hand and the control, and the user intention judging condition is that the distance between the user hand and the control is smaller than a second set distance threshold. If the user intention information meets the user intention judging condition, determining that the user intention triggers the control, and determining the control as a target control.
In another embodiment, for triggering a control in a gesture triggering manner, the user intention information is a distance between a user gaze point and the control, and the user intention judgment condition is that the distance between the user gaze point and the control is smaller than a third set distance threshold. If the user intention information meets the user intention judging condition, determining that the user intention triggers the control, and determining the control as a target control.
In another embodiment, for triggering a control in a voice triggering manner, the user intention information is a distance between a user gaze point and the control, and the user intention judgment condition is that the distance between the user gaze point and the control is smaller than a fourth set distance threshold. If the user intention information meets the user intention judging condition, determining that the user intention triggers the control, and determining the control as a target control.
S130, displaying the visualized hot area corresponding to the target control.
In one embodiment, the size of the hot zone is greater than the size of the target control.
The hot area refers to an area capable of representing a control, and a triggering operation (including a selecting/confirming operation) of a user on the hot area is equivalent to a selecting/confirming operation on a control corresponding to the hot area.
A visualized hot region means that the hot region is visible to the user. In other words, the user can visually perceive the location, boundary, of the hot region in the augmented reality space.
And S140, responding to the triggering operation of the hot area corresponding to the target control, and executing the control operation corresponding to the target control.
If the user triggers the hot area through the controller, the triggering operation of the hot area means that after the controller enters the hot area, a key on the controller is pressed. If the user triggers the hot area by gesture, the triggering operation of the hot area means that the user completes the designated action (such as clicking action) after the user's hand enters the hot area. If the user triggers the hot area in a voice mode, the triggering operation of the hot area means that the user describes the identification information and the triggering action name of the hot area in a voice mode. The identification information of the hot area refers to information capable of uniquely determining the hot area that the user wants to trigger from among the plurality of hot areas, and illustratively includes, but is not limited to, a color, a shape, a position, a number, a name of the hot area, and a color, a shape, a position, a number, a name, etc. of a target control corresponding to the hot area. Trigger action names include, but are not limited to, open, enter, point, click, select, and the like.
For example, if the function of the control a is to open the video a in the video player, when the user triggers the hot area corresponding to the control a, the video a is opened in the video player.
According to the technical scheme, the target control is determined in at least one control; displaying a visualized hot area corresponding to the target control; and responding to the triggering operation of the hot area corresponding to the target control, and executing the control operation corresponding to the target control, wherein the triggering operation of the hot area is equivalent to the triggering operation of the control corresponding to the hot area, so that the difficulty in triggering the control can be reduced by reasonably setting the hot area, further the aim of reducing the false touch probability and assisting a user to quickly complete the interaction with the control is fulfilled.
In one embodiment, the hot zone and its corresponding control at least partially coincide. The setting can be convenient for a user to quickly identify the corresponding relation between the hot area and the control, and the probability of false touch is reduced.
In the augmented reality space, controls are largely divided into two categories: one is a 2D control and one is a 3D control.
The thermal area corresponding to the 2D control may be a planar graphic or a stereoscopic graphic, which is not limited in this application. If the thermal area is a plane graph, the thermal area and the corresponding control are at least partially overlapped, which means that the thermal area and the corresponding control are positioned on the same plane, and the outline graph of the thermal area and the outline graph of the corresponding control are at least partially overlapped. If the thermal area is a three-dimensional graph, the thermal area is at least partially overlapped with the corresponding control, namely the plane of the control passes through the thermal area corresponding to the thermal area, the thermal area is sectioned by the plane of the control, and the contour graph of the obtained section is at least partially overlapped with the contour graph of the control. Fig. 4 is a schematic diagram illustrating a correspondence relationship between a hot area and a control according to an embodiment of the disclosure. In fig. 4, the control is a button with text information of 5. Referring to fig. 4, if the control is a 2D control, the thermal region is a column, and one bottom surface of the thermal region is in the plane of the control and at least partially coincides with the region occupied by the control.
The thermal area corresponding to the 3D control may be a planar graphic or a stereoscopic graphic, which is not limited in this application. If the thermal area is a plane graph, the thermal area and the corresponding control are at least partially overlapped, namely the plane of the thermal area passes through the control corresponding to the thermal area, and the control is cut by the plane of the thermal area, so that the contour graph of the cut section is at least partially overlapped with the contour graph of the thermal area. If the thermal area is a three-dimensional graph, the thermal area and the corresponding control are at least partially overlapped, which means that in the space, the intersection part of the thermal area and the corresponding control is in a three-dimensional structure. Fig. 5 is a schematic diagram illustrating a correspondence relationship between a hot area and a control according to another embodiment of the disclosure. Referring to fig. 5, if the control is a 3D control, the hot zone is in the shape of a cube or sphere, and the control is located within the hot zone.
Further, the hot zone is a solid or planar pattern, and the target control is located in the hot zone. By the arrangement, the user can quickly determine the corresponding relation between the hot area and the control, and the user is further assisted to quickly complete the interaction with the control.
Fig. 6 is a flowchart of another control interaction method provided in an embodiment of the present disclosure. Fig. 6 is a specific example of fig. 3. Referring to fig. 6, the control interaction method includes:
s210, displaying at least one control in the augmented reality space.
S220, determining a target control in at least one control.
And S230, determining the size and the position of the thermal area corresponding to the target control based on a preset rule.
The preset rule is information preset by a developer and used for determining the position and the boundary of the hot area in the augmented reality space.
Optionally, the preset rule may specify whether the hot area corresponding to the control is a planar graph or a stereoscopic graph, a relative position of the hot area and the control, and a size of a boundary line of the hot area.
Optionally, different types of controls may also be set to correspond to different preset rules.
And S240, rendering the color and/or the outline of the thermal region based on the size and the position of the thermal region.
The purpose of rendering the color and/or outline of the thermal region is to allow the user to visually perceive the location, boundary, of the thermal region in the augmented reality space.
In one embodiment, if the visualized hot region is a hot region after rendering the color, the color of the hot region is different from the background color of the augmented reality space and the color of the control; and/or if the visualized hot area is the hot area after the outline is rendered, the color of the outline of the hot area is different from the background color of the augmented reality space and the color of the control. Such an arrangement may help the user visually perceive the location, boundary, of the hot zone in the augmented reality space.
S250, displaying the rendered hot area.
And S260, responding to the triggering operation of the hot area corresponding to the target control, and executing the control operation corresponding to the target control.
According to the technical scheme, the size and the position of the thermal area corresponding to the target control are determined based on the preset rule; rendering the color and/or outline of the thermal region based on the size and location of the thermal region; the rendered hot area is displayed, so that a user can visually and intuitively perceive the position and the boundary of the hot area in the augmented reality space, and the hot area can be conveniently and accurately and quickly triggered subsequently.
Based on the above technical solutions, optionally, in response to the selected instruction of the hot area, information for prompting that the hot area is selected is displayed.
If the user triggers the controller, the instruction selected by the hot area refers to the generated instruction when the controller does not press the key on the controller after entering the hot area. If the user is triggered by a gesture, the instruction that the hot area is selected refers to an instruction generated when a specified action (such as a clicking action) is not completed after the user's hand enters the hot area. If the user triggers the hot area through voice, the triggering operation of the hot area refers to an instruction generated when the user speaks the identification information describing the hot area and does not speak the triggering action name. In other words, when the user performs the triggering operation on the hot area corresponding to the target control, two instructions are generated, one is the instruction of which the hot area is selected (i.e. the selection instruction), and the other is the instruction of which the hot area is triggered (i.e. the confirmation instruction). The instructions for which the hot zone is selected are generated prior to the instructions for which the hot zone is triggered. Subsequently, in response to the instruction that the hot area is selected, information for prompting that the hot area is selected is displayed, wherein the information is used for indicating whether a user can expect the purpose if the current position is kept motionless, a button on the controller is pressed, or a specified action (such as a clicking action) is completed, or a trigger action name is spoken, and which control is triggered is further clarified. In response to an instruction that a hot zone is triggered, a control operation associated with a control corresponding to the triggered hot zone is performed.
In one embodiment, in response to an instruction that a hot region is selected, presenting information that prompts the hot region to be selected includes: the color of the thermal region and/or the color of the thermal region contour line is changed in response to the instruction that the thermal region is selected. After the controller or the hand of the user enters the hot area or after the user commands the identification information of the hot area through voice, the color of the hot area and/or the color of the contour line of the hot area are changed, and the user can intuitively determine which control is currently selected through the color change, so that the aim of reducing the difficulty of triggering the control is further fulfilled.
In another embodiment, in response to an instruction that a hot region is selected, information is presented that prompts the hot region to be selected, including: and responding to the selected instruction of the hot area, and displaying the dynamic effect of the preset image entering the hot area. The preset image may specifically be a controller graphic, a user hand graphic, or other graphic (such as an arrow graphic, a circle image, etc.) specified in advance. The user can intuitively determine which control is currently selected through the dynamic effect of entering the hot area through the controller graph, the user hand graph or other graphs (such as arrow graphs, circle images and the like) appointed in advance, so that the aim of reducing the difficulty of triggering the control is further achieved. Fig. 7 and 8 are schematic diagrams of correspondence between two other hot areas and controls provided in an embodiment of the disclosure. Fig. 7 corresponds to fig. 4, and fig. 8 corresponds to fig. 5. Comparing fig. 4 and 7, or comparing fig. 5 and 8, a user can be assisted in quickly determining whether the controller enters the hot zone by displaying a picture of the controller graphic entering the hot zone.
It should be noted that, for simplicity of description, the foregoing method embodiments are all described as a series of acts, but it should be understood by those skilled in the art that the present invention is not limited by the order of acts described, as some steps may be performed in other orders or concurrently in accordance with the present invention. Further, those skilled in the art will also appreciate that the embodiments described in the specification are all preferred embodiments, and that the acts and modules referred to are not necessarily required for the present invention.
Fig. 9 is a schematic structural diagram of a control interaction device in an embodiment of the disclosure. The device is applied to an extended reality XR terminal device. Referring to fig. 9, the control interaction device specifically includes:
a first display module 310, configured to display at least one control in the augmented reality space;
a determining module 320, configured to determine a target control among the at least one control;
a second display module 330, configured to display a visualized hot area corresponding to the target control;
and the execution module 340 is configured to execute a control operation corresponding to the target control in response to a triggering operation on the thermal area corresponding to the target control.
Further, the size of the thermal area is greater than the size of the target control.
Further, the thermal region and the target control corresponding thereto at least partially coincide.
Further, the hot area is a solid figure, and the target control is located in the hot area.
Further, the second display module 330 is configured to:
determining the size and the position of a thermal area corresponding to the target control based on a preset rule;
rendering a color and/or contour of the thermal region based on the size and location of the thermal region;
the rendered hot region is shown.
Further, if the visualized hot area is a hot area after rendering the color, the color of the hot area is different from the background color of the augmented reality space and the color of the control; and/or the number of the groups of groups,
if the visualized hot area is the hot area after the outline is rendered, the color of the outline of the hot area is different from the background color of the augmented reality space and the color of the control.
Further, the display device also comprises a third display module for:
and responding to the selected instruction of the hot area, and displaying information for prompting the selected hot area.
Further, the display device also comprises a third display module for:
in response to the selected instruction of the thermal region, changing the color of the thermal region and/or the color of the thermal region contour.
Further, the display device also comprises a third display module for:
and responding to the selected instruction of the hot area, and displaying the dynamic effect of entering the hot area by a preset graph.
The control interaction device provided by the embodiment of the present disclosure may perform the steps of the control interaction method provided by the embodiment of the present disclosure, and have the same or corresponding beneficial effects, which are not described herein again.
Fig. 10 is a schematic structural diagram of an electronic device in an embodiment of the disclosure. Referring now in particular to fig. 10, a schematic diagram of an electronic device 1000 suitable for use in implementing embodiments of the present disclosure is shown. The electronic device 1000 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), wearable electronic devices, and the like, and fixed terminals such as digital TVs, desktop computers, smart home devices, and the like. The electronic device shown in fig. 10 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.
As shown in fig. 10, an electronic device 1000 may include a processing means (e.g., a central processor, a graphics processor, etc.) 1001 that may perform various suitable actions and processes to implement the control interaction method of the embodiments as described in the present disclosure according to a program stored in a Read Only Memory (ROM) 1002 or a program loaded from a storage means 1008 into a Random Access Memory (RAM) 1003. In the RAM 1003, various programs and information necessary for the operation of the electronic apparatus 1000 are also stored. The processing device 1001, the ROM 1002, and the RAM 1003 are connected to each other by a bus 1004. An input/output (I/O) interface 1005 is also connected to bus 1004.
In general, the following devices may be connected to the I/O interface 1005: input devices 1006 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 1007 including, for example, a Liquid Crystal Display (LCD), speaker, vibrator, etc.; storage 1008 including, for example, magnetic tape, hard disk, etc.; and communication means 1009. The communication means 1009 may allow the electronic device 1000 to communicate wirelessly or by wire with other devices to exchange information. While fig. 10 shows an electronic device 1000 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program containing program code for performing the method shown in the flowcharts, thereby implementing the control interaction method as described above. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 1009, or installed from the storage device 1008, or installed from the ROM 1002. The above-described functions defined in the method of the embodiment of the present disclosure are performed when the computer program is executed by the processing device 1001.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include an information signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with digital information communication (e.g., a communication network) in any form or medium. Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
displaying at least one control in the augmented reality space;
determining a target control in the at least one control;
displaying a visualized hot area corresponding to the target control;
and responding to the triggering operation of the thermal area corresponding to the target control, and executing the control operation corresponding to the target control.
Alternatively, the electronic device may perform other steps described in the above embodiments when the above one or more programs are executed by the electronic device.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device comprising:
one or more processors;
a memory for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement any of the control interaction methods as provided by the present disclosure.
According to one or more embodiments of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements a control interaction method as any of the present disclosure provides.
The disclosed embodiments also provide a computer program product comprising a computer program or instructions which, when executed by a processor, implement a control interaction method as described above.
It should be noted that in this document, relational terms such as "first" and "second" and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
The foregoing is merely a specific embodiment of the disclosure to enable one skilled in the art to understand or practice the disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the disclosure. Thus, the present disclosure is not intended to be limited to the embodiments shown and described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (12)
1. A control interaction method, characterized in that the method is applied to an augmented reality XR terminal device, the method comprising:
displaying at least one control in the augmented reality space;
determining a target control in the at least one control;
displaying a visualized hot area corresponding to the target control;
and responding to the triggering operation of the thermal area corresponding to the target control, and executing the control operation corresponding to the target control.
2. The method of claim 1, wherein a size of the hot zone is greater than a size of the target control.
3. The method of claim 1, wherein the hot zone and the target control corresponding thereto at least partially coincide.
4. The method of claim 3, wherein the hot zone is a solid or planar graphic and the target control is located within the hot zone.
5. The method of claim 1, wherein the exposing the visualized hot area corresponding to the target control comprises:
determining the size and the position of a thermal area corresponding to the target control based on a preset rule;
rendering a color and/or contour of the thermal region based on the size and location of the thermal region;
the rendered hot region is shown.
6. The method of claim 5, wherein if the visualized hot area is a hot area after rendering the color, the color of the hot area is different from the background color of the augmented reality space and the color of the control; and/or the number of the groups of groups,
if the visualized hot area is the hot area after the outline is rendered, the color of the outline of the hot area is different from the background color of the augmented reality space and the color of the control.
7. The method as recited in claim 1, further comprising:
and responding to the selected instruction of the hot area, and displaying information for prompting the selected hot area.
8. The method of claim 7, wherein the presenting information prompting the hot zone to be selected in response to the instruction that the hot zone is selected comprises:
in response to the selected instruction of the thermal region, changing the color of the thermal region and/or the color of the thermal region contour.
9. The method of claim 7, wherein responsive to the instruction that the hot zone is selected, presenting information prompting that the hot zone is selected comprises:
and responding to the selected instruction of the hot area, and displaying the dynamic effect of entering the hot area by a preset graph.
10. A control interaction device, wherein the device is applied to an augmented reality XR terminal device, the device comprising:
the first display module is used for displaying at least one control in the augmented reality space;
the determining module is used for determining a target control in the at least one control;
the second display module is used for displaying the visualized hot area corresponding to the target control;
and the execution module is used for responding to the triggering operation of the thermal area corresponding to the target control and executing the control operation corresponding to the target control.
11. An electronic device, the electronic device comprising:
one or more processors;
a storage means for storing one or more programs;
the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9.
12. A computer readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210680232.0A CN117270732A (en) | 2022-06-15 | 2022-06-15 | Control interaction method and device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210680232.0A CN117270732A (en) | 2022-06-15 | 2022-06-15 | Control interaction method and device, electronic equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117270732A true CN117270732A (en) | 2023-12-22 |
Family
ID=89216573
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210680232.0A Pending CN117270732A (en) | 2022-06-15 | 2022-06-15 | Control interaction method and device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117270732A (en) |
-
2022
- 2022-06-15 CN CN202210680232.0A patent/CN117270732A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10606609B2 (en) | Context-based discovery of applications | |
| US10950205B2 (en) | Electronic device, augmented reality device for providing augmented reality service, and method of operating same | |
| EP3665559B1 (en) | Electronic device and method for executing function using input interface displayed via at least portion of content | |
| KR102499354B1 (en) | Electronic apparatus for providing second content associated with first content displayed through display according to motion of external object, and operating method thereof | |
| US10331340B2 (en) | Device and method for receiving character input through the same | |
| CN110215686A (en) | Display control method and device, storage medium and electronic equipment in scene of game | |
| US10416761B2 (en) | Zoom effect in gaze tracking interface | |
| CN117270732A (en) | Control interaction method and device, electronic equipment and storage medium | |
| KR20200066962A (en) | Electronic device and method for providing content based on the motion of the user | |
| CN116610312A (en) | Interface layout method, apparatus, device, readable storage medium, and program product | |
| US20240028130A1 (en) | Object movement control method, apparatus, and device | |
| US20240103625A1 (en) | Interaction method and apparatus, electronic device, storage medium, and computer program product | |
| CN117784919A (en) | Virtual input device display method and device, electronic device and storage medium | |
| CN117631810A (en) | Operation processing method, device, equipment and medium based on virtual reality space | |
| CN117075770A (en) | Interaction control method and device based on augmented reality, electronic equipment and storage medium | |
| CN115981544A (en) | Interaction method and device based on augmented reality, electronic equipment and storage medium | |
| CN117636528A (en) | Voting processing method, device, equipment and medium based on virtual reality space | |
| CN117994284A (en) | Collision detection method, collision detection device, electronic equipment and storage medium | |
| CN117742555A (en) | Control interaction method, device, equipment and medium | |
| CN117641025A (en) | Model display method, device, equipment and medium based on virtual reality space | |
| CN117762280A (en) | Man-machine interaction method, device and equipment | |
| CN114397961A (en) | Head-mounted display device control method, head-mounted display device assembly, and medium | |
| CN117640919A (en) | Picture display method, device, equipment and medium based on virtual reality space | |
| CN117632391A (en) | Application control method, device, equipment and medium based on virtual reality space | |
| CN117631817A (en) | Measurement method, measurement device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |