WO2023246312A1

WO2023246312A1 - Interface interaction method, interface interaction apparatus, and computer storage medium

Info

Publication number: WO2023246312A1
Application number: PCT/CN2023/091527
Authority: WO
Inventors: 辛孟怡; 赵静; 崔诚伟
Original assignee: 京东方科技集团股份有限公司; 京东方智慧物联科技有限公司
Priority date: 2022-06-23
Filing date: 2023-04-28
Publication date: 2023-12-28
Also published as: CN115097995A

Abstract

The present application relates to the technical field of interaction, and discloses an interface interaction method, an interface interaction apparatus, and a computer storage medium. The method comprises: displaying a first interface by means of a display screen; displaying an image corresponding to a human body posture of at least one user at a corresponding position in the first interface; and in response to the overlapping time of the image corresponding to the human body posture and a target first control among a plurality of first controls being greater than a preset value, displaying a second interface corresponding to the target first control by means of the display screen. According to the present application, the image corresponding to the human body posture is displayed at the corresponding position in the first interface, and then, when the overlapping time of the image corresponding to the human body posture and the control is greater than the preset value, the second interface corresponding to the control is displayed. In this way, the user does not need to perform an operation, interaction between the user and an interface interaction apparatus is implemented, so that the problem in the related art that an interface interaction method is complex is solved, and the effect of simplifying the interface interaction method is achieved.

Description

Interface interaction method, interface interaction device and computer storage medium

This application claims priority to the Chinese patent application with application number 202210726631.6 and the invention name "Interface interaction method, interface interaction device and computer storage medium" submitted on June 23, 2022, the entire content of which is incorporated into this application by reference. middle.

Technical field

The present application relates to the field of interaction technology, and in particular to an interface interaction method, an interface interaction device and a computer storage medium.

Background technique

Interface interaction refers to the interaction between the user and the interface displayed on the display screen of the interface interaction device. The user can obtain information through the interface, or input information to the interface interaction device through the interface.

In an interface interaction method, multiple controls are displayed through the display screen in the interface interaction device. Each control can have a corresponding information column. When the user passes by the interface interaction device, he can click on the multiple controls displayed on the display screen. As a control in the screen, the interface interaction device can control the display screen to display the information column corresponding to the clicked control.

However, the interaction method in the above interface interaction method is relatively complex.

Contents of the invention

Embodiments of the present application provide an interface interaction method, an interface interaction device, and a computer storage medium. The technical solutions are as follows:

According to one aspect of the embodiments of the present application, an interface interaction method is provided. The method is applied to an interface interaction device. The interface interaction device includes a display screen and a human gesture recognition component. The method includes:

A first interface is displayed through the display screen, and the first interface has a plurality of first controls;

In response to the human body posture recognition component detecting the human body posture of at least one user in the preset area, based on the relative position of the at least one user and the display screen, display the at least one user at a corresponding position in the first interface. An image corresponding to a user's human body posture;

In response to the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value, a second interface corresponding to the target first control is displayed through the display screen.

Optionally, the preset area is located in the area directly facing the display screen,

Displaying an image corresponding to the human body posture of the at least one user at a corresponding position in the first interface based on the relative position of the user and the display screen includes:

Obtain the horizontal position of the at least one user in the preset area through the human gesture recognition component;

At a corresponding position in the horizontal direction in the first interface, an image corresponding to the human body posture of the at least one user is displayed.

Optionally, the human body posture recognition component detects the human body postures of at least two users in the preset area,

In response to the overlap time of the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value, the second image corresponding to the target first control is displayed on the display screen. interface, including:

In response to the fact that among the images corresponding to the human body postures of the at least two users, there are at least two images corresponding to the human body postures of the target user whose overlap time with the plurality of first controls is greater than a preset value, it is determined that the at least two images corresponding to the human body postures of the target user have an overlap time greater than a preset value. The first target user who is closest to the display screen among the target users;

Through the display screen, a second interface corresponding to the target first control overlapping with the first image is displayed, and the first image is an image corresponding to the human body posture of the first target user.

Optionally, displaying an image corresponding to the human body posture of the at least one user at a corresponding position in the first interface based on the relative position of the at least one user and the display screen includes:

Obtain the relative distance and relative position between the at least two users and the display screen respectively;

Based on the relative distance and relative position between the at least two users and the display screen, an image corresponding to the human body posture of the at least one user is displayed at the corresponding position in the first interface. The human body posture of the user The size of the corresponding image is inversely related to the relative distance between the user and the display screen.

Optionally, the second interface includes a folding control located at an edge of the second interface, and the folding control corresponds to at least two third controls,

After the second interface corresponding to the target first control is displayed on the display screen, the method further includes:

Obtain the hand information of the target user among the at least one user through the human gesture recognition component interest;

Display an image corresponding to the hand information in the second interface based on the hand information;

In response to the hand information, an operation on the folding control is issued, and at least two third controls corresponding to the folding control are displayed in the second interface.

Optionally, after displaying at least two third controls corresponding to the folding control in the second interface, the method further includes:

In response to the hand information, an operation for the third control is issued, a second interface corresponding to the third control is displayed through the display screen, and at least two third controls corresponding to the folding control are stopped from being displayed.

Optionally, displaying the second interface corresponding to the target first control through the display screen includes:

Determine the folding position of the folding control in the second interface based on the relative position of the target user and the display screen;

The second interface corresponding to the target first control is displayed through the display screen, and the folding control is displayed at the folding position of the second interface.

Optionally, the folding control is in a strip shape and is located at the edge of the second interface.

Optionally, the image corresponding to the human body posture is a humanoid image corresponding to the human body posture.

Optionally, the second interface includes a plurality of second controls, the plurality of second controls are arranged in a horizontal direction in the second interface, and the target first control is displayed through the display screen The corresponding second interface,

Based on the position of the target user among the at least one user, display on the second interface a plurality of second sub-controls corresponding to the second control directly opposite the position of the target user among the plurality of second controls, The plurality of second sub-controls are arranged along a second direction at the position where the second control is located, and the second direction intersects the horizontal direction.

Optionally, based on the position of the target user among the at least one user, display on the second interface a plurality of second controls corresponding to the second control directly opposite the position of the target user among the plurality of second controls. A second sub-control, including:

Determine the target second control facing the position of the target user among the at least one user;

From the position of the target second control, move to the top and bottom of the target second control respectively. The plurality of second sub-controls pop up.

Optionally, the interface interaction device also includes a speaker,

A plurality of gesture control information corresponding to the second interface is played through the speaker.

According to another aspect of the embodiment of the present application, an interface interaction device is provided. The interface interaction device includes a control component, a display screen, and a human posture recognition component. The control component is electrically connected to the display screen and the human posture recognition component respectively. connection, the control components include:

A first display module, used to control the display screen for a first interface, with a plurality of first controls on the first interface;

The second display module is configured to respond to the human posture recognition component detecting the human posture of at least one user in the preset area, based on the relative position of the at least one user and the display screen, in the first interface The corresponding position displays an image corresponding to the human body posture of the at least one user;

A third display module configured to display the target first control through the display screen in response to the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value. The second interface corresponding to the control.

According to another aspect of the embodiment of the present application, an interface interaction device is provided. The interface interaction device includes a processor and a memory. The memory stores at least one instruction, at least a program, a code set or an instruction set, and the At least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the above-mentioned interface interaction method.

According to another aspect of the embodiments of the present application, a computer storage medium is provided. The computer storage medium stores at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program , the code set or instruction set is loaded and executed by the processor to implement the above-mentioned interface interaction method.

According to another aspect of the embodiments of the present application, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various optional implementations.

The beneficial effects brought by the technical solutions provided by the embodiments of this application at least include:

By displaying a plurality of first controls on the first interface of the display screen, and when the human posture recognition component detects the user's human posture in the preset area, based on the user's position, the corresponding human posture is displayed at the corresponding position in the first interface. image, and then when the overlap time between the image corresponding to the human posture and a control is greater than the preset value, the second interface corresponding to the control is displayed. In this way, the interaction between the user and the interface interaction device is realized without the user's operation, the interface interaction method is simplified, the problem of the relatively complicated interface interaction method in the related technology is solved, and the effect of simplifying the interface interaction method is achieved. The user can interact with the interface interaction device without clicking on the interface interaction device, which simplifies the interface interaction method and improves the user experience.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 is a schematic structural diagram of an interface interaction device provided by an embodiment of the present application;

Figure 2 is a flow chart of an interface interaction method according to an embodiment of the present application;

Figure 3 is a flow chart of an interface interaction method according to an embodiment of the present application;

Figure 4 is a schematic diagram of a first interface in an embodiment of the present application;

Figure 5 is a schematic diagram of the positional relationship between a user and a display screen in an embodiment of the present application;

Figure 6 is a schematic diagram of the positional relationship between another user and the display screen in the embodiment of the present application;

Figure 7 is a schematic diagram of the positional relationship between a user and a display screen in an embodiment of the present application;

Figure 8 is a schematic diagram of a second interface provided by an embodiment of the present application;

Figure 9 is a schematic diagram of a second interface in an embodiment of the present application;

Figure 10 is a schematic diagram of another second interface in the embodiment of the present application;

Figure 11 is a schematic diagram of another second interface in the embodiment of the present application;

Figure 12 is a schematic diagram of another user and display screen provided by an embodiment of the present application;

Figure 13 is a schematic diagram of a third interface in the embodiment of the present application;

Figure 14 is a block diagram of an interface interaction device provided by an embodiment of the present application.

Through the above-mentioned drawings, clear embodiments of the present application have been shown, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present application's concepts in any way, but are intended to illustrate the application's concepts for those skilled in the art with reference to specific embodiments.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

Interface interaction devices can be various smart devices including display screens, such as smartphones, tablets, laptops, vertical advertising machines, electronic picture frames, and outdoor electronic screens. At present, as various interface interaction devices are widely used in all aspects of life, interface interaction has become an important way of interaction. Through interface interaction, the user can obtain various information from the interface interaction device. In contrast, the interface interaction device can also display various information to the user. For example, it can display various preset information to the user.

However, in current interface interaction methods, the user is usually required to actively operate the interface interaction device. For example, the user is required to actively click on a control displayed on the display screen of the interface interaction device, so that the interface interaction device can respond to the control clicked by the user. To display the corresponding information to achieve the effect of displaying information to users. Although this method can display information to the user and realize the function of interface interaction, the interaction process is relatively complicated (complex here may mean that it is complicated for the user). The user needs to actively stay in front of the interface interaction device and actively click on the controls. Only then can interaction be achieved. This greatly reduces the user's enthusiasm for interface interaction, reduces the possibility of the interface interaction device being used, and further reduces the possibility that the information provided by the interface interaction device is known to the user. For example, if the information to be provided by the interface interaction device is advertising information, this method will reduce the possibility that the advertising information will be seen by the user and reduce the advertising effect.

Figure 1 is a schematic structural diagram of an interface interaction device provided by an embodiment of the present application. The interface interaction device 10 may include a display screen 11 and a human gesture recognition component 12. The human body posture recognition component 12 can be installed on the display screen 11, or can be independently installed outside the display screen 11. Figure 1 shows a structure in which the human body posture recognition component 12 is installed on the display screen 11, but the embodiment of the present application does not No restrictions are imposed. When the human gesture recognition component 12 is installed on the display screen 11 , it can be located on the upper edge or the lower edge of the display screen 11 .

The display screen 11 may have an image display function, and the display screen 11 may emit light in the facing direction, so that users located in the preset area q in the facing direction can see the image displayed on the display screen 11 . In addition, the display screen 11 can also be a touch screen to enhance user experience by enriching interface interaction.

The human body posture recognition component 12 can be used to obtain the user's human body posture related information in the preset area q. Data, such as posture, location, and gesture information. The human posture recognition component 12 can obtain data related to human posture through various methods, such as obtaining data related to human posture through infrared sensors, etc., which are not limited in the embodiments of the present application. The size of the preset area q can be determined by the detection capability of the human gesture recognition component 12 and the preset detection area. For example, if the detection capability of the human gesture recognition component 12 is strong, the size of the preset area q can be larger, and the preset detection area can be less than or equal to the maximum area that the detection capability of the human gesture recognition component 12 can cover.

In addition, the interface interaction device 10 may also include a control component 13, which may be electrically connected to the display screen 11 and the human gesture recognition component 12 respectively. The control component 13 can be integrated into the display screen 11 , or the control component 13 can be independently provided outside the display screen 11 and connected to the display screen 11 in a wired or wireless manner.

The control component 13 can be used to control the display screen 11 and the human gesture recognition component 12 . For example, the image displayed on the display screen 11 can be controlled, or the data related to human posture collected by the human posture recognition component 12 can be obtained.

In addition, the interface interaction device 10 may also include a speaker 14. The speaker 14 may be electrically connected to the control component 13 for emitting sound under the control of the control component 13. The speaker 14 may be combined with the display screen 11, or the speaker 14 may be installed in the display screen 11. 14 can be independently arranged outside the display screen 11 .

The interface interaction device in the embodiment of the present application can be located in various areas, for example, it can be located in a bank lobby, a school, a car sales hall, an agricultural product sales point, a sales department, etc. Located in different areas, it can have different functions. For example, when it is located in a bank hall, it can be used to introduce financial-related information to users, such as deposit business, relevant regulations, financial services, and various financial products. It is located in a car sales hall. At the same time, it can be used to introduce the appearance, configuration, and selling price of the car to users, as well as provide simulated test drive functions, etc.

FIG. 2 is a flow chart of an interface interaction method according to an embodiment of the present application. This embodiment illustrates an example in which the interface interaction method is applied to the interface interaction device shown in FIG. 1 . The interface interaction method may include the following steps:

Step 201: Display a first interface through a display screen, with a plurality of first controls on the first interface.

Step 202: In response to the human body posture recognition component detecting the human body posture of at least one user in the preset area, based on the relative position of the at least one user and the display screen, display the human body posture corresponding to the at least one user at the corresponding position in the first interface. image.

Step 203: In response to the image corresponding to the human body posture and the target first control among the plurality of first controls, If the overlap time is greater than the preset value, the second interface corresponding to the target first control is displayed on the display screen.

To sum up, the interface interaction method provided by the embodiment of the present application displays a plurality of first controls on the first interface of the display screen, and when the human posture recognition component detects the user's human posture in the preset area, based on the user's position, the image corresponding to the human posture is displayed at the corresponding position in the first interface, and then when the overlap time between the image corresponding to the human posture and a control is greater than the preset value, the second interface corresponding to the control is displayed. In this way, the interaction between the user and the interface interaction device is realized without the user's operation, the interface interaction method is simplified, the problem of the relatively complicated interface interaction method in the related technology is solved, and the effect of simplifying the interface interaction method is achieved. The user can interact with the interface interaction device without clicking on the interface interaction device, which simplifies the interface interaction method and improves the user experience.

FIG. 3 is a flow chart of an interface interaction method according to an embodiment of the present application. This embodiment illustrates an example in which the interface interaction method is applied to the control component in the interface interaction device shown in FIG. 1 . The interface interaction method may include the following steps:

Step 301: Display a first interface through a display screen, with a plurality of first controls on the first interface.

The control component can control the display screen to display a first interface through the display screen, the first interface having a plurality of first controls.

The controls involved in the embodiments of the present application, such as the first control, the second control and the third control, may refer to icons displayed in the interface of the display screen. Each control may correspond to a control signal, and the control may be displayed on the user's screen. It is triggered by gesture control and touch control.

After the control component obtains the control through the human gesture recognition component and is triggered, the display screen can be controlled based on the control signal. For example, the display screen can be controlled to display corresponding content, or the speaker can be controlled to emit corresponding sounds.

Step 302: Detect the preset area through the human posture recognition component.

In this embodiment of the present application, the control component can continuously detect the preset area through the human gesture recognition component. Specifically, it can be to detect whether there is a user in the preset area, as well as the user's position, posture, gesture and other information.

In the embodiment of the present application, the control component can continuously detect the preset area through the human gesture recognition component after the interface interaction device is started, or it can also detect the preset area through the human gesture recognition component after it is started and after receiving a specified signal. Continuously detect the preset area, or, after startup, and after reaching the preset conditions (the conditions can be set in advance, such as a specified time after startup, or a specified moment after startup), and then through human posture recognition The component continuously checks the preset area.

Step 303: In response to the human body posture recognition component detecting the human body posture of at least one user in the preset area, based on the relative position of the at least one user and the display screen, display the human body posture corresponding to the at least one user at the corresponding position in the first interface. image.

When the human posture recognition component detects the human posture of at least one user in the preset area, it indicates that at least one user is in the preset area. At this time, the control component can display an image corresponding to the human body posture of at least one user at a corresponding position in the first interface based on the relative position of the at least one user and the display screen, which can enhance the appeal to users in the preset area. , which can then increase the probability that the user stays in the preset area and interacts with the interface interaction device, thus improving the interface interaction device.

In an exemplary embodiment, as shown in Figure 4, Figure 4 is a schematic diagram of a first interface in an embodiment of the present application, wherein a plurality of first controls k1 are displayed in the first interface m1, and an image t1 corresponding to the human body posture. The image t1 corresponding to the human body posture is a humanoid image corresponding to the human body posture. The posture of the humanoid image is basically the same as the posture of the user in the preset area. If the user looks at the display screen, he will see a little person on the display screen whose movements are consistent with his own. This can greatly improve the user's understanding of the interface interaction device. interest, thereby increasing the user's willingness to interact with the interface interactive device, thereby increasing the possibility that the information to be displayed by the interface interactive device will be known to the user. Optionally, the humanoid image corresponding to the human body posture can have multiple display forms, for example, it can be a humanoid outline, and the outline can be filled with particles, or it can be a stick figure-shaped humanoid image.

In an exemplary embodiment, step 303 may include:

1) Obtain the horizontal position of at least one user in the preset area through the human posture recognition component.

The user usually moves in the horizontal direction relative to the interface interaction device, and the control component can obtain the horizontal position of at least one user in the preset area through the human gesture recognition component.

2) Display at least one image corresponding to the human body posture of the user at the corresponding position in the horizontal direction in the first interface.

The corresponding position may be the position of the orthographic projection on the display screen when the user is located directly in front of the display screen, for example, as shown in Figure 5. Figure 5 is a schematic diagram of the positional relationship between a user and the display screen in an embodiment of the present application. . If the user 51 is located directly in front of the display screen 11 , the image t1 corresponding to the user's human body posture may be located at the orthographic projection position of the user 51 on the display screen 11 .

In addition, the corresponding position may also refer to the corresponding position of the user in the preset area. For example, the position in the preset area and the position in the display screen may be mapped in the horizontal direction, so that the position in the preset area Each position has a corresponding position in the display screen. For example, as shown in FIG. 6 , FIG. 6 is a schematic diagram of the positional relationship between another user and the display screen in an embodiment of the present application. The position of the user 51 in the preset area q may correspond to the opposite position on the display screen 11, and the image t1 corresponding to the human body posture of the user 51 may be located at the opposite position. This achieves the effect that the image corresponding to the user's human body posture will move following the user's movement, and the user experience is better.

In addition, there may be multiple users in the preset area, and the human posture recognition component may also be able to recognize the human postures of at least two users. In this case, in an exemplary embodiment, step 303 may also include:

1) Obtain the relative distance and relative position between at least two users and the display screen respectively.

The control component can respectively obtain the relative distance and relative position between at least two users and the display screen through the human gesture recognition component. The relative distance may refer to the minimum distance between the user and the display screen, or the vertical distance between the user and the plane where the display screen is located. For the relative position, reference can be made to the content shown in Figure 5 above, which will not be described again here.

2) Based on the relative distance and relative position between at least two users and the display screen, an image corresponding to the human body posture of at least one user is displayed at the corresponding position in the first interface.

Among them, the size of the image corresponding to the user's human body posture is negatively related to the relative distance between the user and the display screen. That is to say, the smaller the distance between the user and the display screen, the larger the size of the image corresponding to the user's human body posture will be. This makes it easier for the user to find his or her corresponding image on the display screen, and increases the convenience of interface interaction methods. Interactivity and fun also increase user experience.

Exemplarily, as shown in Figure 7, Figure 7 is a schematic diagram of the positional relationship between a user and the display screen in an embodiment of the present application. If the distance between user y1 and the display screen is relatively close, then the image corresponding to the human body posture of user y1 The size of y11 is larger and the distance between the user y2 and the display screen is farther, so the size of the image y12 corresponding to the human body posture of the user y2 is smaller.

Of course, the image corresponding to the user's human body posture can also move in the vertical direction in the first interface. For example, when the user moves in the direction away from the display screen, the image corresponding to the user's human body posture can be correspondingly moved in the first interface. Move in the vertical upward direction. When the user moves in the direction close to the display screen, the image corresponding to the user's human body posture can correspondingly move in the vertical downward direction in the first interface, which is consistent with the above horizontal direction. Combined with mobile, the image corresponding to the user's human body posture in the first interface can present a mirrored display method, that is, the user is displayed in the first interface in a mirrored manner, which improves the user's interactive experience.

When the housing control in Figure 7 is triggered, the control component can control the display screen to display financial home housing. House sand table, and digitally display real estate market value, transaction volume and other data in the interface. In the financial home residential sandbox, the housing financial island residential model can be displayed, and the AI smart housekeeper synchronously broadcasts the welcome speech, prompting customers to click on any property through gesture recognition for a detailed experience. For different properties, the real-time dynamic display of the property's attention rankings , monthly trading volume and average price data.

In addition, when the user selects different house types through gesture recognition, the interior scene of the house can be displayed on the display screen, and the AR (augmented reality) real-scene roaming function in the house is provided to the user. The user can move up, left, right and down. Watch and wait to experience the living room scene, and at the same time, you can check out the house purchase plan.

The control component can also play house-related information through the speakers, such as detailed introductions to residential properties and apartment types.

Step 304: In response to the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value, display a second interface corresponding to the target first control on the display screen.

After the image corresponding to the user's human body posture is displayed on the display screen, the image will move as the user moves, and the user can then move to move the image corresponding to the human body posture to a position that overlaps with a certain first control. When the image corresponding to the user's human body posture overlaps with a certain first control, the first control can change in a specified form, for example, it can emit light, become larger, have a collision effect, etc. This is not limited in the embodiments of the present application. .

It can be seen from the above discussion that the human posture recognition component may detect that there are human body postures of at least two users in the preset area. At this time, there may also be overlap times between images corresponding to the human body postures of multiple target users and multiple first controls. is greater than the preset value. In this case, step 304 may include:

1) In response to the fact that among the images corresponding to the human body postures of at least two users, there are at least two target users whose overlapping time with the plurality of first controls is greater than the preset value, determine that among the at least two target users The first target user closest to the display screen.

Among the images corresponding to the human body postures of at least two users, there are at least two images corresponding to the human body postures of the target user. When the overlap time between the images corresponding to the human body postures of at least two target users and multiple first controls is greater than the preset value, it indicates that multiple users want to interact with the interface. The device interacts, or there may be at least one user who wants to interact with the interface interaction device, and at least one user happens to stay in the preset area due to other reasons. At this time, the control component can determine at least two target users. Among the first target users who are closest to the display screen, the control component can determine the first target user who is closest to the display screen among multiple users through the human gesture recognition component.

In the embodiment of the present application, the default value can be set in advance, and can be set to 2 seconds to 5 seconds, for example 2 seconds, 3 seconds and 4 seconds etc.

2) Display the second interface corresponding to the target first control overlapping the first image through the display screen.

The nearest first target user is more likely to be a user who actually wants to interact with the interface interaction device. Then, the control component can interact with the first target user, and display the second interface corresponding to the target first control overlapping the first image through the display screen. In this way, on the one hand, confusion in the display on the display screen can be avoided, and on the other hand, the probability of the interface interaction device mistakenly interacting with users who only pass through the preset area can be reduced, that is, the probability of invalid interaction can be reduced.

In addition, please refer to FIG. 8 , which is a schematic diagram of a second interface provided by an embodiment of the present application. The second interface m2 includes a plurality of second controls k2 and a folding control z located on the second interface m2. The folding control z corresponds to at least two third controls. When the folding control z is triggered, the third control can be displayed in the second interface m2.

Optionally, the folding control z is in a strip shape and is located at the edge of the second interface m2. This can avoid the folding control from blocking and affecting other content on the second interface.

In this embodiment of the present application, a method of displaying folding controls may include:

1) Based on the relative position of the target user and the display screen, determine the folding position of the folding control in the second interface.

The control component can determine the folding position of the folding control in the second interface by referring to the above-mentioned method of determining the position of the image corresponding to the human body posture in the first interface. For example, the folding control can be placed in a position facing the target user in the second interface. And it can follow the movement of the target user's position, so that the user can easily trigger the folding control at any time in each position.

2) Display the second interface corresponding to the target first control through the display screen, and display the folding control at the folding position of the second interface.

The control component can display the second interface corresponding to the target first control through the display screen, and display the folding control at the folding position of the second interface. For example, please refer to FIG. 9 , which is a schematic diagram of a second interface in an embodiment of the present application, in which the folding control z is located at a position facing the target user 91 in the second interface m2 and can follow the target user 91 Mobile to facilitate target users 91 operations. In Figure 9, multiple second controls k2 are arranged in the horizontal direction in the second interface m2. Of course, the multiple second controls k2 can also be arranged in the second interface m2 in other ways. The embodiment of the present application is This is not restricted.

Step 305: Play multiple gesture control information corresponding to the second interface through the speaker.

After displaying the second interface, the control component can play multiple gesture control information corresponding to the second interface through the speaker. This can facilitate the user to know the gesture control method in the second interface, where the gesture Control information can include:

Select the control in a cutting manner; extend your left arm to the left and rotate it to the left to control the second interface or the display content in the second interface to rotate to the left; extend your right arm to the right and rotate it to the right to control The second interface or the displayed content in the second interface rotates to the right; crossing the arms can control returning to the previous interface.

In addition, playing multiple gesture control information corresponding to the second interface can also attract users to interact with the interface interaction device. There may also be situations where the user accidentally is in the preset area and triggers the first control in the first interface. In this case, the user can be reminded by playing a sound, and the user may hear the gesture control information. The interest in interacting with the interface interaction device can enhance the ability of the interface interaction method to promote information.

Step 306: Obtain the hand information of the target user among at least one user through the human gesture recognition component.

The control component may obtain the hand information of the target user among at least one user through the human gesture recognition component when the second interface is displayed on the display screen. The hand information may include information such as the position, posture, and gesture information of the target user's hand.

In the embodiment of the present application, the target user may refer to the user determined in step 304, for whom the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls is greater than the preset value. When there are multiple When the overlap time between the image corresponding to the user's human body posture and the target first control among the plurality of first controls is greater than a preset value, the target user may be the user closest to the display screen.

Step 307: Display the image corresponding to the hand information in the second interface based on the hand information.

The control component may display the image corresponding to the hand information in the second interface based on the hand information obtained in step 306. The image may be a palm-shaped image or other shapes, which are not limited in the embodiments of the present application. To display the image corresponding to the hand information in the second interface, you can refer to the method of displaying the image corresponding to the human posture in the above steps, that is, you can display the hand information corresponding to the position of the mirror image of the user's hand in the second interface. image, and continuously tracks the position of the user's hand to continuously display the image corresponding to the hand information to facilitate the user's gesture control on the second interface.

For example, please refer to FIG. 10 , which is a schematic diagram of another second interface in an embodiment of the present application, in which the image t2 corresponding to the hand information is shown in the second interface m2. The user can perform gesture control in the second interface m2 based on the image t2.

Step 308: Issue an operation for the folding control in response to the hand information, and display at least two third controls corresponding to the folding control in the second interface.

The control component can control the display screen, first play the transition animation on the second interface, and then display at least two third controls corresponding to the folding control on the second interface. The transition animation may include the third control gradually popping up from the position of the folded control, or gradually appearing from other positions, and so on. When the number of third controls is large, it can be arranged in multiple rows; when the number of third controls is small, it can be arranged in one row.

When the control component receives an operation for the folding control, it can control the display screen so that the folding control sends out corresponding feedback, for example, it can vibrate, light up, disconnect from the triggering position of the gesture, and gradually disappear from the second interface. A sort of.

The operation on the folding control may be a cutting action performed by the user on the folding control through a gesture.

Please refer to Figure 11, which is a schematic diagram of another second interface in an embodiment of the present application, in which at least two third controls k3 corresponding to the folding control z are displayed in the second interface m2. In addition, the second interface m2 can also display other controls, such as in-car controls and out-of-car controls to display other car-related information. After the in-car controls are triggered, the interface can display the car interior for 360° viewing; after the exterior controls are triggered, the interface can display the outside of the car for 360° viewing. In addition, the interface can also display the car's performance parameters on the left and right sides, including fuel consumption per 100 kilometers, engine displacement, top speed, length, width and height, rated power, acceleration to 100 kilometers, etc.

When the test drive control in the third control k3 shown in Figure 11 is triggered by the user, the interface interaction device can provide the user with a simulated test drive. During the simulated test drive, the simulated car scene can be displayed through the display screen and the simulated car scene can be displayed through the speaker. Play test drive gesture information, such as: Please stretch your arms forward to start the new car test drive. You can use the height of your arms to control the direction. The left arm is high, to the left, and the right arm is high, to the right.

During the simulated test drive, after the car is started, the driving speed and driving distance parameters can be displayed and changed in real time. A small map is displayed in the lower right corner of the interface to mark the locations of surrounding business districts during the test drive.

In addition, 4 navigation points can also be set. When the vehicle drives to the corresponding point, a three-dimensional user interface pop-up window on the right side of the vehicle displays information about the merchant at that location, including name, poster pictures, and promotional activity information. The pop-up window disappears after being displayed for a specified period of time (such as 4 seconds), the car continues to drive, and when it reaches the next point, business information pops up on the right. The driving process can be ended without setting. Customers can switch to other modes through the menu. The appointment test drive label can be displayed at a fixed position in the lower right corner of the interface. The designated time interval (such as 4 seconds) can be flipped once to display details.

Step 309: Issue an operation for the third control in response to the hand information, display the second interface corresponding to the third control through the display screen, and stop displaying at least two third controls corresponding to the folding control.

When the user issues an operation for the third control corresponding to the folding control through a gesture, the control component can display the second interface corresponding to the third control through the display screen, and stop displaying at least the second interface corresponding to the folding control. Two third controls. When the display of at least two third controls corresponding to the folding control is stopped, a preset transition animation may be played. For example, the at least two third controls may gradually shrink toward the position of the folding control and disappear at the position of the stacked control.

Step 310: Based on the position of the target user among the at least one users, display a plurality of second sub-controls corresponding to the second control directly opposite the position of the target user among the plurality of second controls on the second interface.

In an exemplary embodiment, the control component may display, on the second interface, a plurality of second sub-controls corresponding to the second control directly opposite the target user's position among the plurality of second controls based on the target user's position. That is, when the user is in front of a certain second control, the control component can control the display screen to display the second sub-control, so as to display more information to the user and improve the information interaction efficiency of the interface interaction method.

In an exemplary embodiment, step 310 may include:

1) Determine the target second control facing the target user's position among at least one user.

The control component may determine, through the human gesture recognition component, the target second control facing the position of the target user among the at least one user.

2) From the location of the target second control, pop up multiple second sub-controls above and below the target second control.

This display method can facilitate the user to view the second sub-control corresponding to the second control, and also facilitate the user to further control the second sub-control, thereby improving the user experience.

Please refer to FIG. 12 , which is a schematic diagram of another user and a display screen provided by an embodiment of the present application. Among them, in the second interface m2, a plurality of second sub-controls ss1 are arranged along the second direction f2 at the position of the second control k2, and the second direction f2 intersects the horizontal direction f1. FIG. 12 shows a situation where the second direction f2 and the horizontal direction f1 intersect perpendicularly, but this is not limited. The width of the second sub-control ss1 can be the same as the width of the second control to block the second control.

When displaying multiple second sub-controls ss1, the target second control can first display the transition animation and then display the multiple second sub-controls ss1. The transition animation may be rotation, gradient, etc., which is not limited in the embodiments of the present application.

The second interface m2 shown in Figure 10 can also be displayed on the display screen as the initial page in the interface interaction method provided by the embodiment of the present application. In this way, when the user passes by, different information can be automatically displayed according to the user's position. Detailed information improves the ability to display information to users.

When the green and financial controls are triggered in the first interface shown in Figure 4, the control component can control the display screen to display green financial products and services. For example, as shown in Figure 13, Figure 13 is a schematic diagram of a third interface in an embodiment of the present application, in which the third interface m3 is displayed on the display screen, and the scene of the third interface The technology tree 121 can be included in the three-dimensional space, and there can be suspended particles in the three-dimensional space. There may be multiple floating windows c1 on both sides of the interface m3.

Among them, a plurality of suspended bubbles 1211 are provided on the technology tree 121, and the theme concepts can be displayed in the suspended bubbles 1211, including: green credit, green bonds, green funds, green insurance, derivatives, etc.

Users can select these bubbles through cutting gestures. After the bubble is selected, it can fly to the upper right floating window c1. The bubble will appear in the upper right floating window c1 with a burst animation effect, and an introduction to the theme concept of the currently selected bubble will be displayed. After the bubbles on the technology tree disappear for more than a specified number (such as 3), bubbles of other themes and concepts will be automatically added to the disappearing positions.

In addition, when the third interface m3 is displayed, the control component can control the display screen 11 to display a humanoid image corresponding to the user's human body posture in the third interface m3, and the control component can also play a voice through the speaker: Please make a specified gesture ( (such as extending your right hand with your palm upward, etc.) to check changes in your carbon sink.

When the user reaches out his hand with his palm facing upward, at the position of the palm of the humanoid image, the particles in the three-dimensional scene can come together and become a ball, and then burst, popping up a three-dimensional user interface window, displaying theme concept words, such as: garbage Classification, energy saving and emission reduction (the content is divided into two categories: positive and negative feedback); the positive feedback font in the three-dimensional user interface is green and the negative feedback font is red. Among them, the positive feedback can be: water drops fall on the carbon sink balance p1 in the lower right, and the bar chart p2 increases and displays animation; negative feedback can be: water drops fall on the carbon sink balance p1 in the lower right, and the bar chart p2 decreases and displays animation. At the same time, the carbon sink balance p1 tilts left or right according to the number of positive feedback and negative feedback words.

In addition, a data dashboard b1 can be set on the left side of the third interface m3, which can be used to display real-time data of carbon trading platforms in various regions and cities.

For example, data dashboard b1 can display the carbon price trend, carbon trading volume, carbon trading amount, and market share of these cities in various cities (such as Beijing, Fujian, Guangdong, Hubei, Shanghai, Shenzhen, Tianjin, and Chongqing).

Figure 14 is a block diagram of an interface interaction device provided by an embodiment of the present application. The interface interaction device includes a control component 1410, a display screen 1420, and a human gesture recognition component 1430. The control component 1410 is connected to the display screen 1420 and the human gesture recognition component 1430 respectively. Electrically connected, the control component 1410 includes:

The first display module 1411 is used to control the display screen for the first interface, which has a plurality of first controls;

The second display module 1412 is configured to respond to the human posture recognition component detecting the human posture of at least one user in the preset area, and display the at least one user at the corresponding position in the first interface based on the relative position of the at least one user to the display screen. The image corresponding to the human body posture;

The third display module 1413 is configured to display the second interface corresponding to the target first control through the display screen in response to the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value.

To sum up, the interface interaction device provided by the embodiment of the present application displays a plurality of first controls on the first interface of the display screen, and when the human body posture recognition component detects the user's human body posture in the preset area, based on the user's position, the image corresponding to the human posture is displayed at the corresponding position in the first interface, and then when the overlap time between the image corresponding to the human posture and a control is greater than the preset value, the second interface corresponding to the control is displayed. In this way, the interaction between the user and the interface interaction device is realized without the user's operation, the interface interaction method is simplified, the problem of the relatively complicated interface interaction method in the related technology is solved, and the effect of simplifying the interface interaction method is achieved. The user can interact with the interface interaction device without clicking on the interface interaction device, which simplifies the interface interaction method and improves the user experience.

Embodiments of the present application also provide an interface interaction device. The interface interaction device includes a processor and a memory. The memory stores at least one instruction, at least a program, a code set, or an instruction set. At least one instruction, at least a program, a code set, or an instruction set. The instruction set is loaded and executed by the processor to implement the above-mentioned interface interaction method.

Embodiments of the present application also provide a computer storage medium. The computer storage medium stores at least one instruction, at least a program, a code set or an instruction set. The at least one instruction, at least a program, code set or instruction set is loaded by the processor and Execute to implement the interface interaction method as above.

Embodiments of the present application also provide a computer program product or computer program. The computer program The product or computer program includes computer instructions stored on a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above various optional implementations.

The term "at least one of A and B" in this application is only an association relationship describing associated objects, indicating that three relationships can exist. For example, at least one of A and B can mean: A exists alone, and at the same time There are three situations: A and B, and B alone. In the same way, "at least one of A, B and C" means that seven relationships can exist, which can mean: A exists alone, B exists alone, C exists alone, A and B exist at the same time, A and C exist at the same time, and both exist at the same time. C and B, seven situations A, B and C exist at the same time. In the same way, "at least one of A, B, C and D" means that fifteen relationships can exist, which can mean: A exists alone, B exists alone, C exists alone, D exists alone, A and B exist simultaneously, and at the same time A and C exist, A and D exist simultaneously, C and B exist simultaneously, D and B exist simultaneously, C and D exist simultaneously, A, B and C exist simultaneously, A, B and D exist simultaneously, A and C exist simultaneously and D, B, C and D exist at the same time, and A, B, C and D exist at the same time, these are fifteen situations.

In this application, the terms "first", "second" and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance. The term "plurality" refers to two or more than two, unless expressly limited otherwise.

In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or can be integrated into another system, or some features can be ignored, or not implemented. On the other hand, the coupling or direct coupling or communication connection between each other shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

Those of ordinary skill in the art can understand that all or part of the steps to implement the above embodiments can be achieved by It can be completed by hardware, or it can be completed by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk.

The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection of the present application. within the range.

Claims

An interface interaction method, characterized in that the method is applied to an interface interaction device, the interface interaction device includes a display screen and a human gesture recognition component, the method includes:

A first interface is displayed through the display screen, and the first interface has a plurality of first controls;

In response to the human body posture recognition component detecting the human body posture of at least one user in the preset area, based on the relative position of the at least one user and the display screen, display the at least one user at a corresponding position in the first interface. An image corresponding to a user's human body posture;

In response to the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value, a second interface corresponding to the target first control is displayed through the display screen.
The method according to claim 1, characterized in that the preset area is located in an area directly facing the display screen,

Displaying an image corresponding to the human body posture of the at least one user at a corresponding position in the first interface based on the relative position of the at least one user and the display screen includes:

Obtain the horizontal position of the at least one user in the preset area through the human gesture recognition component;

At a corresponding position in the horizontal direction in the first interface, an image corresponding to the human body posture of the at least one user is displayed.
The method according to claim 1, characterized in that the human body posture recognition component detects the human body postures of at least two users in the preset area,

In response to the overlap time of the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value, the second image corresponding to the target first control is displayed on the display screen. interface, including:

In response to the fact that among the images corresponding to the human body postures of the at least two users, there are at least two images corresponding to the human body postures of the target user whose overlap time with the plurality of first controls is greater than a preset value, it is determined that the at least two images corresponding to the human body postures of the target user have an overlap time greater than a preset value. The first target user who is closest to the display screen among the target users;

Through the display screen, a second interface corresponding to the target first control overlapping with the first image is displayed, and the first image is an image corresponding to the human body posture of the first target user.
The method according to claim 3, characterized in that, based on the relative position of the at least one user and the display screen, the corresponding position of the human body posture of the at least one user in the first interface is displayed. images including:

Obtain the relative distance and relative position between the at least two users and the display screen respectively;

Based on the relative distance and relative position between the at least two users and the display screen, an image corresponding to the human body posture of the at least one user is displayed at the corresponding position in the first interface. The human body posture of the user The size of the corresponding image is inversely related to the relative distance between the user and the display screen.
The method of claim 1, wherein the second interface includes a folding control located at an edge of the second interface, and the folding control corresponds to at least two third controls,

After the second interface corresponding to the target first control is displayed on the display screen, the method further includes:

Obtain the hand information of the target user among the at least one user through the human gesture recognition component;

Display an image corresponding to the hand information in the second interface based on the hand information;

In response to the hand information, an operation on the folding control is issued, and at least two third controls corresponding to the folding control are displayed in the second interface.
The method according to claim 5, characterized in that, after displaying at least two third controls corresponding to the folding control in the second interface, the method further includes:

In response to the hand information, an operation for the third control is issued, a second interface corresponding to the third control is displayed through the display screen, and at least two third controls corresponding to the folding control are stopped from being displayed.
The method of claim 5, wherein displaying the second interface corresponding to the target first control through the display screen includes:

Determine the folding position of the folding control in the second interface based on the relative position of the target user and the display screen;

The second interface corresponding to the target first control is displayed through the display screen. In the second interface The folding control is displayed in the folding position.
The method according to claim 5, wherein the folding control is in a strip shape and is located at the edge of the second interface.
The method according to any one of claims 1 to 8, characterized in that the image corresponding to the human body posture is a humanoid image corresponding to the human body posture.
The method according to any one of claims 1 to 8, characterized in that the second interface includes a plurality of second controls, and the plurality of second controls are arranged in the horizontal direction in the second interface, so Displaying the second interface corresponding to the target first control through the display screen,

After the second interface corresponding to the target first control is displayed on the display screen, the method further includes:

Based on the position of the target user among the at least one user, display on the second interface a plurality of second sub-controls corresponding to the second control directly opposite the position of the target user among the plurality of second controls, The plurality of second sub-controls are arranged along a second direction at the position where the second control is located, and the second direction intersects the horizontal direction.
The method according to claim 10, characterized in that, based on the location of a target user among the at least one user, the second interface displays the location of the target user among the plurality of second controls. Multiple second sub-controls corresponding to the opposite second control, including:

Determine the target second control facing the position of the target user among the at least one user;

From the position where the target second control is located, pop up the plurality of second sub-controls above and below the target second control respectively.
An interface interaction device, characterized in that the interface interaction device includes a control component, a display screen and a human posture recognition component. The control component is electrically connected to the display screen and the human posture recognition component respectively. The control component includes :

A first display module, used to control the display screen for a first interface, with a plurality of first controls on the first interface;

The second display module is configured to respond to the human gesture recognition component detecting that at least one of the preset areas is A user's human body posture, based on the relative position of the at least one user and the display screen, display an image corresponding to the at least one user's human body posture at a corresponding position in the first interface;

A third display module configured to display the target first control through the display screen in response to the overlap time between the image corresponding to the human body posture and the target first control among the plurality of first controls being greater than a preset value. The second interface corresponding to the control.
An interface interaction device, characterized in that the interface interaction device includes a processor and a memory, and the memory stores at least one instruction, at least a program, a code set or an instruction set, and the at least one instruction, the at least A program, the code set or the instruction set is loaded and executed by the processor to implement the interface interaction method according to any one of claims 1 to 11.
A computer storage medium, characterized in that at least one instruction, at least one program, a code set or an instruction set is stored in the computer storage medium, and the at least one instruction, the at least one program, the code set or an instruction set is stored in the computer storage medium. The set is loaded and executed by the processor to implement the interface interaction method as described in any one of claims 1 to 11.