WO2023245373A1

WO2023245373A1 - Electronic display device and display control method and apparatus therefor, and storage medium

Info

Publication number: WO2023245373A1
Application number: PCT/CN2022/099937
Authority: WO
Inventors: 张逸帆
Original assignee: 北京小米移动软件有限公司
Priority date: 2022-06-20
Filing date: 2022-06-20
Publication date: 2023-12-28
Also published as: CN117999537A

Abstract

An electronic display device and a display control method and apparatus therefor, and a storage medium. The method comprises: in response to an instruction to enable a one-hand operation mode having been detected, generating a floating window interface according to a display interface of an electronic display device in the current display state (S11); displaying the floating window interface on an upper layer of the display interface in a target display region on a screen of the electronic display device, wherein the target display region is determined on the basis of a gesture of a user holding the electronic display device with one hand (S12); when it is detected that the user performs a first one-hand touch-control operation in the floating window interface, controlling a first target interface to respond to the first one-hand touch-control operation (S13); and according to a change of the first target interface, controlling a second target interface to synchronously change, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is the other of the display interface and the floating window interface (S14). By means of the method, it is more convenient for a user to perform a one-hand touch-control operation.

Description

Electronic display equipment and display control method and device thereof, storage medium

Technical field

The present disclosure relates to the technical field of electronic display equipment, and in particular, to an electronic display equipment, a display control method and device thereof, and a storage medium.

Background technique

With the rapid development of screen technology, large-screen electronic display devices and folding-screen electronic display devices are gradually becoming popular among the public.

In order to better watch movies, browse the web, or view documents, people are more willing to buy large-screen electronic display devices. However, large-screen electronic display devices have the problem of inconvenient touch operations.

Contents of the invention

In order to overcome the problems existing in related technologies, the present disclosure provides an electronic display device, a display control method and device thereof, and a storage medium.

According to a first aspect of an embodiment of the present disclosure, a display control method for an electronic display device is provided, the method including:

In response to detecting the instruction to start the one-handed operation mode, generating a floating window interface according to the display interface in the current display state of the electronic display device, where the floating window interface is a reduced size of the display interface;

The floating window interface is displayed on the upper layer of the display interface in a target display area on the screen of the electronic display device, wherein the target display area is determined based on the user's gesture of holding the electronic display device with one hand. ;

When it is detected that the user performs a first one-hand touch operation in the floating window interface, the first target interface is controlled to respond to the first one-hand touch operation, and the first one-hand touch operation is the user's first one-hand touch operation. Operations performed by holding the electronic display device with one hand; and,

The second target interface is controlled to change synchronously according to the change of the first target interface, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is the The other one of the display interface and the floating window interface.

Optionally, the first target interface is configured with an application program interface, and controlling the first target interface to respond to the first one-hand touch operation includes:

Determine the target application according to the first one-hand touch operation;

By calling the interface of the target application program, the first target interface is controlled to respond to the first one-hand touch operation.

Optionally, the method also includes:

When it is detected that the user performs a second one-hand touch operation in the display interface, the first target interface is controlled to respond to the second one-hand touch operation.

Optionally, if the first target interface is the display interface and the second target interface is the floating window interface, the user performs the first one-handed operation in the floating window interface. In the case of a touch operation, controlling the first target interface to respond to the first one-hand touch operation includes:

Map the first one-handed touch operation to the display interface;

The display interface is controlled to respond to the mapping result of the first one-hand touch operation.

Optionally, displaying the floating window interface on an upper layer of the display interface in the target display area on the screen of the electronic display device includes:

Obtain the user's touch data on the electronic display device;

According to the touch data, determine the gesture of the user holding the electronic display device with one hand;

Determine the operation area where the user's hand holding the electronic display device with one hand performs a one-handed touch operation on the screen of the electronic display device according to the gesture of the user holding the electronic display device with one hand;

Determine the target display area according to the operation area;

The floating window interface is displayed on an upper layer of the display interface in the target display area.

Optionally, the touch data includes a touch image, and determining the user's gesture of holding the electronic display device with one hand according to the touch data includes:

The touch image is input into the trained gesture recognition model, and a recognition result output by the gesture recognition model representing the gesture of the user holding the electronic display device with one hand is obtained.

Optionally, the electronic display device includes an electromagnetic wave energy absorption ratio sensor, the touch data includes sensor data collected by the electromagnetic wave energy absorption ratio sensor, and based on the touch data, it is determined that the user holds the device with one hand. Gestures for electronic display devices, including:

Determine the contact area between the user and the electronic display device based on the sensor data;

According to the position distribution of the contact area on the electronic display device, a gesture of the user holding the electronic display device is determined.

According to a second aspect of an embodiment of the present disclosure, a display control device for an electronic display device is provided, the device comprising:

a generating module configured to generate a floating window interface according to the display interface in the current display state of the electronic display device in response to detecting an instruction to start the one-handed operation mode, where the floating window interface is a reduced size of the display interface The final interface;

The display module is configured to display the floating window interface on an upper layer of the display interface in a target display area on the screen of the electronic display device, wherein the target display area is based on the user holding a single-hand display. The gestures of the electronic display device are determined;

a response module configured to control the first target interface to respond to the first one-hand touch operation when it is detected that the user performs a first one-hand touch operation in the floating window interface, and the third A one-hand touch operation is an operation performed by the user holding the electronic display device with one hand;

A synchronization module configured to control the second target interface to perform synchronous changes according to changes in the first target interface, wherein the first target interface is one of the display interface and the floating window interface, so The second target interface is the other one of the display interface and the floating window interface.

Optionally, the first target interface is configured with an application program interface, and the response module includes:

A first determination sub-module configured to determine a target application based on the first one-hand touch operation;

The calling module is configured to control the first target interface to respond to the first one-hand touch operation by calling the interface of the target application program.

Optionally, the device also includes:

The control module is configured to control the first target interface to respond to the second one-hand touch operation when it is detected that the user performs a second one-hand touch operation in the display interface.

Optionally, the response module is configured to:

If the first target interface is the display interface and the second target interface is the floating window interface, map the first one-hand touch operation to the display interface; control the display interface Respond to the mapping result of the first one-hand touch operation.

Optionally, the display module includes:

An acquisition submodule configured to acquire the user's touch data on the electronic display device;

The second determination sub-module is configured to determine the gesture of the user holding the electronic display device with one hand according to the touch data;

The third determination sub-module is configured to determine, based on the user's gesture of holding the electronic display device with one hand, whether the user's hand holding the electronic display device with one hand performs a one-handed gesture on the screen of the electronic display device. Operating area for touch operations;

a fourth determination sub-module configured to determine the target display area according to the operation area;

The display submodule is configured to display the floating window interface on an upper layer of the display interface in the target display area.

Optionally, the touch data includes a touch image, and the second determination sub-module is configured to:

Optionally, the electronic display device includes an electromagnetic wave energy absorption ratio sensor, the touch data includes sensor data collected by the electromagnetic wave energy absorption ratio sensor, and the second determination sub-module is configured to:

According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the display control of the electronic display device provided by the first aspect of the present disclosure is implemented. Method steps.

According to a third aspect of an embodiment of the present disclosure, an electronic display device is provided, including:

A memory on which a computer program is stored;

A processor, configured to execute the computer program in the memory to implement the steps of the display control method of the electronic display device provided by the first aspect of the present disclosure.

The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

In response to detecting the instruction to start the one-handed operation mode, the electronic display device generates a floating window interface based on the display interface in the current display state of the electronic display device. The floating window interface is a reduced size interface of the display interface. A floating window interface is displayed on an upper layer of the display interface in a target display area on the screen of the electronic display device, where the target display area is determined based on a gesture of the user holding the electronic display device with one hand. When it is detected that the user performs a first one-hand touch operation in the floating window interface, the first target interface is controlled to respond to the first one-hand touch operation, wherein the first one-hand touch operation is held by the user with one hand. Hands operating electronic display device. And, the second target interface is controlled to change synchronously according to the change of the first target interface, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is the other of the display interface and the floating window interface. By. Using this method, since the floating window interface is an interface after the display interface has been reduced in size, the user can perform a one-handed touch operation on the display interface by performing a one-handed touch operation in the floating window interface. And because the size of the floating window interface is smaller than the size of the display interface, the floating window interface is more convenient for users to perform one-handed touch operations compared to the display interface. Therefore, this method of the present disclosure solves the problem in the related art that it is inconvenient to perform touch operations with one hand due to the large screen of a large-screen electronic display device. Problems with hand touch operation.

It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and do not limit the present disclosure.

Description of the drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow chart of a display control method of an electronic display device according to an exemplary embodiment.

FIG. 2 is a block diagram of a display control device of an electronic display device according to an exemplary embodiment.

FIG. 3 is a block diagram of an electronic display device according to an exemplary embodiment.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatus and methods consistent with aspects of the disclosure as detailed in the appended claims.

It should be noted that all actions to obtain signals, information, instructions or data in this application are performed under the premise of complying with the corresponding data protection regulations and policies of the country where the location is located, and with authorization from the owner of the corresponding device.

FIG. 1 is a flow chart of a display control method of an electronic display device according to an exemplary embodiment. As shown in FIG. 1 , the display control method of the electronic display device may include the following steps.

In step S11, in response to detecting an instruction to start the one-handed operation mode, a floating window interface is generated according to the display interface in the current display state of the electronic display device, where the floating window interface is the reduced size of the display interface. .

The one-handed operation mode may refer to a mode of using one hand for touch operation, a small screen operation mode, a virtual operation interface mode, etc. The floating window interface is the interface after the display interface is reduced in size. The floating window interface and the display interface have the same display content. It should be noted that in this disclosure, the floating window interface does not belong to the display content of the display interface.

In some implementations, a reduced-size floating window interface can be obtained by obtaining the display interface and reducing the length and width of the display interface in equal proportions. In other embodiments, the size of the floating window interface may be a default value, such as 3.5 inches, 4 inches, 4.5 inches, etc. This disclosure does not place a specific limit on the size of the floating window interface.

In response to detecting an instruction to start the one-handed operation mode, a floating window interface is generated according to the display interface in the current display state of the electronic display device. Among them, the instruction to start the one-handed operation mode can be that the user clicks the screen in a preset manner, the user slides on the screen according to a preset trajectory, the user clicks a preset button, the user inputs a preset voice, or the user uses a preset gesture to hold the electronic display. Equipment and other actions.

In step S12, the floating window interface is displayed on the upper layer of the display interface in a target display area on the screen of the electronic display device, wherein the target display area is based on the user holding the electronic display with one hand. Device gestures determined.

It should be noted that since the electronic display device can display the superposition result of multiple interfaces, in some embodiments, a floating window interface can be displayed on the upper layer of the display interface in the target display area on the screen of the electronic display device. And the transparency of the floating window interface can be freely set by the user. By adjusting the transparency of the floating window interface, the degree of occlusion of the display interface by the floating window interface can be reduced.

The target display area is determined based on the user's gesture of holding the electronic display device with one hand, and the target display area is an area other than the blind area where the user performs a one-hand touch operation.

In some embodiments, the display interface in the current display state of the electronic display device may refer to the home screen interface of the electronic display device, or may refer to the superposition result of multiple interfaces other than the floating window interface in this disclosure.

In step S13, when it is detected that the user performs a first one-hand touch operation in the floating window interface, the first target interface is controlled to respond to the first one-hand touch operation. The touch operation is an operation performed by the user holding the electronic display device with one hand.

For example, the first one-handed touch operation may be an operation performed by the user holding the electronic display device with one hand. The specific operation may be using a finger (or a finger wearing an auxiliary tool such as a touch finger glove) to click on the floating window interface. Operation of function icons (such as button controls) within the application. The first one-hand touch operation may also be an operation in which the user uses a finger to resize the floating window interface. The first one-hand touch operation may also be an operation in which the user uses a finger to slide the floating window interface (such as a page turning operation). The first one-hand touch operation may also be an operation in which the user inputs text in the floating window interface. The first one-hand touch operation may also be an operation for the user to draw in the floating window interface. The first one-hand touch operation may also be an operation for the user to play video/audio in the floating window interface.

It should be noted that the first one-hand touch operation performed by the user in the floating window interface does not include the action of dragging the floating window interface to adjust the display position of the floating window interface on the screen of the electronic display device.

In step S14, the second target interface is controlled to change synchronously according to the change of the first target interface, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is controlled to change synchronously. The target interface is the other one of the display interface and the floating window interface.

When the first target interface changes in response to the first one-hand touch operation, the second target interface is controlled to change synchronously according to the change of the first target interface. That is, in the present disclosure, the first target interface and the second target interface have a mutual mapping relationship. The display content of the first target interface and the second target interface remain consistent, but the size of the displayed content may be inconsistent.

For example, the first target interface is a display interface, and the second target interface is a floating window interface. When the display interface changes in response to the first one-hand touch operation, the floating window interface is controlled to change synchronously. In this case, the display interface is configured with an application program interface, and the corresponding interface can be called according to the first one-hand touch operation. By calling the interface, the background application can be triggered to respond, causing the display interface to change accordingly.

As another example, the first target interface is a floating window interface, and the second target interface is a display interface. When the floating window interface changes in response to the first one-hand touch operation, the display interface is controlled to change synchronously. In this case, the floating window interface is configured with an application program interface, and the corresponding interface can be called according to the first one-hand touch operation. By calling the interface, the background application can be triggered to respond, causing the floating window interface to change accordingly.

Optionally, the method also includes:

The second one-handed touch operation may be a hand other than the hand of the user holding the electronic display device with one hand. For example, if the user holds the electronic display device with his right hand, then the second one-handed touch operation refers to the left hand. operations performed.

In some embodiments, the second one-hand touch operation may be the user's hand holding the electronic display device with one hand.

For example, the second one-handed touch operation may be an operation in which the user (left hand) uses a finger, a stylus, or a mouse to click on a function icon (such as a button control) of the application program in the display interface. The second one-hand touch operation may also be an operation in which the user uses a finger to zoom in on the size of the display interface. The second one-hand touch operation may also be an operation in which the user uses a finger, a stylus, or a mouse to slide (such as turning pages) to display the interface. The second one-hand touch operation may also be an operation in which the user inputs text in the display interface. The second one-hand touch operation may also be an operation for the user to draw within the display interface. The second one-hand touch operation may also be an operation by the user to play video/audio in the display interface.

In some implementations, if the user performs a touch operation in the floating window interface, the floating window interface is determined as the first target interface, and the display interface is determined as the second target interface. In this way, the display interface is a mapping interface (projection interface) of the floating window interface.

In other embodiments, if the user performs a touch operation in the display interface, the display interface is determined as the first target interface, and the floating window interface is determined as the second target interface. In this way, the floating window interface is a mapping interface (projection interface) of the display interface.

In other implementations, in order to generate a floating window interface more simply and quickly, the display interface can be defaulted to the first target interface, and the floating window interface can be defaulted to the second target interface. In this way, the floating window interface is a mapping interface (projection interface) of the display interface, and the floating window interface has the function of displaying the projection result.

Using this method, the electronic display device responds to detecting an instruction to start the one-handed operation mode and generates a floating window interface based on the display interface in the current display state of the electronic display device. The floating window interface is the reduced size of the display interface. . A floating window interface is displayed on an upper layer of the display interface in a target display area on the screen of the electronic display device, where the target display area is determined based on a gesture of the user holding the electronic display device with one hand. When it is detected that the user performs a first one-hand touch operation in the floating window interface, the first target interface is controlled to respond to the first one-hand touch operation, wherein the first one-hand touch operation is held by the user with one hand. Hands operating electronic display device. And, the second target interface is controlled to change synchronously according to the change of the first target interface, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is the other of the display interface and the floating window interface. By. Using this method, since the floating window interface is an interface after the display interface has been reduced in size, the user can perform a one-handed touch operation on the display interface by performing a one-handed touch operation in the floating window interface. And because the size of the floating window interface is smaller than the size of the display interface, the floating window interface is more convenient for users to perform one-handed touch operations compared to the display interface. Therefore, this method of the present disclosure solves the problem in the related art that it is inconvenient to perform touch operations with one hand due to the large screen of a large-screen electronic display device. Problems with hand touch operation.

Moreover, since the user can not only perform touch operations in the floating window interface, but also can perform touch operations in the display interface, this method of the present disclosure not only facilitates the user to perform one-handed touch operations in the floating window interface, but also facilitates the user to perform touch operations in the floating window interface. It also facilitates the user to use the other hand to perform touch operations within the display interface. For example, in a teaching scenario, the teacher uses his right hand to write text in the floating window interface, and the teacher uses his left hand to slide left in the display interface to move the image in the display interface to the left, thereby making it easier for the right hand to write text continuously and avoid writing. Text overlaps each other.

Determine a target application program according to the first one-hand touch operation; and control the first target interface to respond to the first one-hand touch operation by calling an interface of the target application program.

For example, assuming that the first one-hand touch operation is an operation in which the user clicks on the icon of application A in the floating window interface, it is determined based on the operation position and operation characteristics (such as click, slide, etc.) of the first one-hand touch operation. The target application is program a that starts application A. By calling the interface of program a that starts application A, it controls the first target interface to respond to the first one-hand touch operation, thereby starting application A, and by starting application A A, the first target interface can be changed accordingly, for example, the first target interface changes to the main interface displayed after entering application A.

Mapping the first one-hand touch operation to the display interface; controlling the display interface to respond to the mapping result of the first one-hand touch operation.

For example, if the first target interface is a display interface and the second target interface is a floating window interface, then when it is detected that the user performs a first one-handed touch operation in the floating window interface, the first one-handed touch operation is Control operations are mapped to the display interface. The display interface responds to the mapping result of the first one-hand touch operation. For example, assuming that the first one-handed touch operation is the user clicking on the icon of application B in the floating window interface, then the first one-handed touch operation is mapped to the display interface, and the obtained mapping result is that the user clicks on the icon in the display interface. The icon of application B, the display interface changes in response to the user's operation of clicking the icon of application B in the display interface.

Optionally, if the first target interface is a floating window interface and the second target interface is a display interface, then when it is detected that the user performs a second one-hand touch operation in the display interface, the second one-hand touch operation is Control operations are mapped to the floating window interface. The floating window interface responds to the mapping result of the second one-handed touch operation.

In some embodiments, when the electronic display device is a tablet computer, a touch-operated notebook, or a folding screen mobile phone, one way of determining the target display area of the floating window interface on the screen of the electronic display device may be :

Obtain the user's touch data on the electronic display device. According to the touch data, the user's gesture of holding the electronic display device with one hand is determined. According to the gesture of the user holding the electronic display device with one hand, the operable area for the user's hand holding the electronic display device with one hand to perform a single-handed touch operation on the screen of the electronic display device is determined. The target display area is determined based on the operable area where the user performs touch operations with one hand. For example, the operable area where the user performs touch operation with one hand is used as the display area. For another example, a part of the operable area where the user performs a touch operation with one hand is used as the display area.

The touch data may include a touch image. According to the touch data, an implementation method of determining the user's gesture of holding the electronic display device with one hand may be:

Input the touch image into the trained gesture recognition model, and obtain the recognition result output by the gesture recognition model that represents the gesture of the user holding the electronic display device with one hand.

Assume that there are m*n capacitive lattices distributed on the screen of the electronic display device. Since metal objects and human bodies close to the screen will change the capacitance value at the corresponding position, when the user comes into contact with the screen of the electronic display device, the screen Corresponds to a touch image (capacitive image) of m*n*1.

Since the user may come into contact with the screen of the electronic display device (such as the edge of the screen) when the user holds the electronic display device, m*n*1 touch points can be obtained when the user holds the electronic display device. control image. By inputting m*n*1 touch images into the trained gesture recognition model, the recognition result output by the gesture recognition model that represents the gesture of the user holding the electronic display device can be obtained.

The training method of the gesture recognition model may be to determine the touch image sample and the gesture label of the touch image sample, train the gesture recognition model based on the touch image sample and the gesture label of the touch image sample, and obtain the trained Gesture recognition model.

An example is to input m*n*1 (assumed to be 16*40*1) touch images into a neural network (such as ResNet, ResNeXt, MobileNet, or ShuffleNet, etc.), and calculate x ₁ feature values. Take MobileNet as an example: using MobileNet's convolution layer (conv) to process a 16*40*1 touch image, a convolution result of 2*5*x ₁ can be obtained. Use MobileNet's average pooling layer (avg pool) to calculate the mean of 2*5*x ₁ , and you can get the eigenvalues of 1*1*x ₁ (that is, x ₁ eigenvalues), and the eigenvalues of 1*1*x ₁ Can represent a 16*40*1 touch image. Input the 1*1*x ₁ feature value into the fully connected + softmax layer of MobileNet to get the probability vector. The probability vector represents the gesture of the user holding the electronic display device, such as holding the electronic display device with the right hand, holding the electronic display device with the left hand, gestures other than holding the electronic display device with one hand, that is, holding the electronic display device with both hands.

In another example, a touch image of m*n*1 (assumed to be 16*40*1) is segmented to obtain a first segmented image and a second segmented image, where the first segmented image is a segmented image including the touch image. The image of the surrounding edge pixels. For example, the number of pixels in the first segmented image is (16+40)*2*4=(112*4), and the number of pixels in the second segmented image is (16-2*4)*(40-2* 4)=(8*32), where 16 represents the width of the touch image, 40 represents the length of the touch image, 2 represents that the touch image has two long sides and two wide sides, and 4 represents the segmentation width of the touch image. (That is, move the first side of the touch image inward by 4 pixels, and divide it to obtain the first sub-image with a width of 4. Move the second side of the touch image inward by 4 pixels, and divide it to obtain a width of 4 The second sub-image of 4. Move the third side of the touch image inward by 4 pixels, and divide it to obtain the third sub-image with a width of 4. Move the fourth side of the touch image inward by 4 pixels. point, a fourth sub-image with a width of 4 is obtained by segmentation. The first segmented image includes a first sub-image, a second sub-image, a third sub-image, and a fourth sub-image).

Input the first segmented image into a neural network (such as ResNet, ResNeXt, MobileNet, ShuffleNet, etc.), and calculate Y ₁ feature values. Taking MobileNet as an example: Using MobileNet's convolution layer (conv) to process the first segmentation image of 112*4*1, a convolution result of 28*1*Y ₁ can be obtained. Use MobileNet's average pooling layer (avg pool) to calculate the mean of 28*1*Y ₁ , and you can get the eigenvalues of 1*1*Y ₁ (that is, Y ₁ eigenvalues), and the eigenvalues of 1*1*Y ₁ It can represent the first segmented image of 112*4*1.

Furthermore, the second segmented image is input into a neural network (such as ResNet, ResNeXt, MobileNet, ShuffleNet, etc.), and Y ₂ feature values are calculated. Taking MobileNet as an example: Using MobileNet's convolution layer (conv) to process the second segmentation image of 8*32*1, a convolution result of 2*5*Y ₂ can be obtained. Use MobileNet's average pooling layer (avg pool) to calculate the mean of 2*5*Y ₂ , and you can get the eigenvalues of 1*1*Y ₂ (that is, Y ₂ eigenvalues), and the eigenvalues of 1*1*Y ₂ It can represent the second segmented image of 8*32*1.

Concatenate (concatenate) the eigenvalues of 1*1*Y ₁ with the eigenvalues of 1*1*Y ₂ to obtain the concatenation vector of 1*1*(Y ₁ +Y ₂ ). The splicing vector of ₁ + Y ₂ ) is input into the fully connected + softmax layer of MobileNet to obtain the probability vector. The probability vector represents the gesture of the user holding the electronic display device, such as holding the electronic display device with the right hand, holding the electronic display device with the left hand, holding the electronic display device with both hands, etc.

Another example is to segment a touch image of m*n*1 (assumed to be 16*40*1) to obtain a first segmented image, where the first segmented image includes pixels around the edges of the touch image. image. For example, the number of pixels in the first divided image is (16+40)*2*4=(112*4), where 16 represents the width of the touch image, 40 represents the length of the touch image, and 2 represents the length of the touch image. Two long sides and two wide sides, 4 represents the segmentation width of the touch image (i.e., move the first edge of the touch image inward by 4 pixels, and segment it to obtain the first sub-image with a width of 4. Move the touch image Move the second side of the image inward by 4 pixels, and divide it into a second sub-image with a width of 4. Move the third side of the touch image inward by 4 pixels, and divide it into a third sub-image with a width of 4. Sub-image. Move the fourth side of the touch image inward by 4 pixels, and segment it to obtain a fourth sub-image with a width of 4. The first segmented image includes the first sub-image, the second sub-image, and the third sub-image. , and the fourth sub-image).

Input the first segmented image into a neural network (such as ResNet, ResNeXt, MobileNet, ShuffleNet, etc.), and calculate Y ₁ feature values. Taking MobileNet as an example: Using MobileNet's convolution layer (conv) to process the first segmentation image of 112*4*1, a convolution result of 28*1*Y ₁ can be obtained. Use MobileNet's average pooling layer (avg pool) to calculate the mean value of 28*1*Y ₁ , and you can get the eigenvalues of 1*1*Y ₁ (that is, Y ₁ eigenvalues), and the eigenvalues of 1*1*Y ₁ It can represent the first segmented image of 112*4*1.

Moreover, input m*n*1 (assumed to be 16*40*1) touch images into a neural network (such as ResNet, ResNeXt, MobileNet, ShuffleNet, etc.), and calculate Y ₂ feature values. Take MobileNet as an example: Using MobileNet's convolution layer (conv) to process a 16*40*1 touch image, a convolution result of 2*5*Y ₂ can be obtained. Use MobileNet's average pooling layer (avg pool) to calculate the mean of 2*5*Y ₂ , and you can get the eigenvalues of 1*1*Y ₂ (that is, Y ₂ eigenvalues), and the eigenvalues of 1*1*Y ₂ Can represent a 16*40*1 touch image.

Another example, input m*n*1 (assumed to be 18*40*1) touch images into a target detection model (such as ResNet, ResNeXt, MobileNet, or ShuffleNet, etc.), and calculate x ₂ Eigenvalues. Use the first fully connected + softmax layer to calculate the position (x, y, w, h) of each anchor box (Anchor Box) and the category probability value corresponding to each anchor box. Categories include accidental contact with fingers, accidental contact with tiger's mouth, left-hand pressing, right-hand pressing, etc. Use the second fully connected layer + softmax to calculate the total probability vector based on the position (x, y, w, h) of each anchor box and the category probability value corresponding to each anchor box. The total probability vector represents the gesture of the user holding the electronic display device, such as holding the electronic display device with the right hand, holding the electronic display device with the left hand, holding the electronic display device with both hands, etc.

Among them, the anchor box refers to multiple a priori boxes with different aspect ratios predefined by the algorithm, centered on the anchor point in the target detection algorithm.

When the electronic display device is a large-screen electronic display device such as a touch-operable television, an electronic blackboard for teaching, etc., one implementation method of determining the display area of the floating window interface on the screen of the electronic display device may be :

Obtain the user's position relative to the electronic display device, and determine an operable area that is convenient for the user to operate based on the user's position relative to the electronic display device, or determine the user's gesture of holding the electronic display device with one hand to determine an operable area that is convenient for the user to operate, according to the The operable area determines the display area of the floating window interface. For example, assuming that it is detected that the user is standing in the lower right corner of the electronic display device, the lower right corner of the screen of the electronic display device can be determined as an operable area that is convenient for the user to operate.

Optionally, the electronic display device in the present disclosure may include an electromagnetic wave energy absorption ratio sensor SAR (specific absorption rate) sensor, and the touch data may include sensor data collected by the electromagnetic wave energy absorption ratio sensor. According to the touch data, an implementation method of determining the gesture of the user holding the electronic display device may be:

The user's contact area with the electronic display device is determined based on the sensor data. According to the position distribution of the contact area on the electronic display device, the gesture of the user holding the electronic display device is determined. For example, assuming that the multiple touch areas are located on the rear right side and the front right side of the electronic display device, it can be determined that the user's gesture of holding the electronic display device is holding the electronic device with the left hand.

Among them, the SAR sensor can determine the contact status between the user and the electronic display device (such as a mobile phone) by detecting capacitance.

In some implementations, the display control method of the electronic display device may further include:

If the user's first one-hand touch operation in the floating window interface is not detected within the preset time period, the floating window interface is controlled to enter a hidden state. For example, if the floating window interface is hidden in the user-invisible area at the edge of the screen, the user can redisplay the floating window interface in the user's visible area by dragging the floating window interface hidden at the edge of the screen. For another example, the floating window interface is changed into a floating ball (suspended point) with a smaller area. By clicking on the floating ball (suspended point), the user can change the floating ball (suspended point) into a floating window interface again.

If it is recognized that the user's gesture of holding the electronic display device is holding it with both hands, the floating window interface can be closed or controlled to enter a hidden state.

If it is detected that the user changes the gesture of holding the electronic display device, the display position of the floating window interface is adjusted according to the changed gesture.

In some implementations, another implementation of determining the display area of the floating window interface on the screen of the electronic display device: determining a fixed target display area based on the user's selection/preset settings.

FIG. 2 is a block diagram of a display control device of an electronic display device according to an exemplary embodiment. Referring to Figure 2, the device 200 includes:

The generation module 210 is configured to generate a floating window interface according to the display interface in the current display state of the electronic display device in response to detecting an instruction to start the one-handed operation mode, where the floating window interface is a reduction of the display interface. The interface after size;

The display module 220 is configured to display the floating window interface on the upper layer of the display interface in a target display area on the screen of the electronic display device, wherein the target display area is based on the user holding it with one hand. The gesture of the electronic display device is determined;

The response module 230 is configured to control the first target interface to respond to the first one-hand touch operation when it is detected that the user performs a first one-hand touch operation in the floating window interface, the The first one-hand touch operation is an operation performed by the user holding the electronic display device with one hand;

The synchronization module 240 is configured to control the second target interface to perform synchronous changes according to changes in the first target interface, wherein the first target interface is one of the display interface and the floating window interface, The second target interface is the other one of the display interface and the floating window interface.

Using this device 200, the electronic display device responds to detecting an instruction to activate the one-handed operation mode and generates a floating window interface based on the display interface in the current display state of the electronic display device. The floating window interface is a reduced size of the display interface. interface. A floating window interface is displayed on an upper layer of the display interface in a target display area on the screen of the electronic display device, where the target display area is determined based on a gesture of the user holding the electronic display device with one hand. When it is detected that the user performs a first one-hand touch operation in the floating window interface, the first target interface is controlled to respond to the first one-hand touch operation, wherein the first one-hand touch operation is held by the user with one hand. Hands performing operations on electronic display devices. And, the second target interface is controlled to change synchronously according to the change of the first target interface, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is the other of the display interface and the floating window interface. By. Using this method, since the floating window interface is an interface after the display interface has been reduced in size, the user can perform a one-handed touch operation on the display interface by performing a one-handed touch operation in the floating window interface. And because the size of the floating window interface is smaller than the size of the display interface, the floating window interface is more convenient for users to perform one-handed touch operations compared to the display interface. Therefore, this method of the present disclosure solves the problem in the related art that it is inconvenient to perform touch operations with one hand due to the large screen of a large-screen electronic display device. Problems with hand touch operation.

Optionally, the first target interface is configured with an application program interface, and the response module 230 includes:

Optionally, the device 200 also includes:

Optionally, the response module 230 is configured to:

Optionally, the display module 220 includes:

Regarding the devices in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be described in detail here.

The present disclosure also provides a computer-readable storage medium on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the display control method of the electronic display device provided by the present disclosure are implemented.

FIG. 3 is a block diagram of an electronic display device 800 according to an exemplary embodiment. For example, the electronic display device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

Referring to Figure 3, electronic display device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output interface 812, sensor component 814, and communication component 816 .

Processing component 802 generally controls the overall operations of electronic display device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned display control method of the electronic display device. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

Memory 804 is configured to store various types of data to support operations at electronic display device 800 . Examples of such data include instructions for any application or method operating on electronic display device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EEPROM), Programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic or optical disk.

Power supply component 806 provides power to various components of electronic display device 800 . Power supply components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic display device 800 .

Multimedia component 808 includes a screen that provides an output interface between the electronic display device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide action. In some embodiments, multimedia component 808 includes a front-facing camera and/or a rear-facing camera. When the electronic display device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and/or the rear camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

Audio component 810 is configured to output and/or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic display device 800 is in operating modes, such as call mode, recording mode, and voice recognition mode. The received audio signal may be further stored in memory 804 or sent via communication component 816 . In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

The input/output interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, etc. These buttons may include, but are not limited to: Home button, Volume buttons, Start button, and Lock button.

Sensor component 814 includes one or more sensors for providing various aspects of status assessment for electronic display device 800 . For example, the sensor component 814 can detect the open/closed state of the electronic display device 800, the relative positioning of components, such as the display and keypad of the electronic display device 800, the sensor component 814 can also detect the electronic display device 800 or the electronic display device 800. Changes in the position of a component of the display device 800 , the presence or absence of user contact with the electronic display device 800 , the orientation or acceleration/deceleration of the electronic display device 800 and changes in the temperature of the electronic display device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

Communication component 816 is configured to facilitate wired or wireless communication between electronic display device 800 and other devices. The electronic display device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In one exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communications component 816 also includes a near field communications (NFC) module to facilitate short-range communications. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic display device 800 may be configured by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field Programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic components are implemented for executing the display control method of the above electronic display device.

In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as a memory 804 including instructions, which can be executed by the processor 820 of the electronic display device 800 to complete the above-mentioned electronic display device is also provided. Display control method. For example, the non-transitory computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a function for performing the above when executed by the programmable device. The code part of the display control method of the electronic display device.

Other embodiments of the disclosure will be readily apparent to those skilled in the art from consideration of the specification and practice of the disclosure. This application is intended to cover any variations, uses, or adaptations of the disclosure that follow the general principles of the disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the disclosure. . It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and various modifications and changes may be made without departing from the scope thereof. The scope of the disclosure is limited only by the appended claims.

Claims

A display control method for an electronic display device, characterized in that the method includes:

In response to detecting the instruction to start the one-handed operation mode, generating a floating window interface according to the display interface in the current display state of the electronic display device, where the floating window interface is a reduced size of the display interface;

The floating window interface is displayed on the upper layer of the display interface in a target display area on the screen of the electronic display device, wherein the target display area is determined based on the user's gesture of holding the electronic display device with one hand. ;

When it is detected that the user performs a first one-hand touch operation in the floating window interface, the first target interface is controlled to respond to the first one-hand touch operation, and the first one-hand touch operation is the user's first one-hand touch operation. Operations performed by holding the electronic display device with one hand; and,

The second target interface is controlled to change synchronously according to the change of the first target interface, wherein the first target interface is one of the display interface and the floating window interface, and the second target interface is the The other one of the display interface and the floating window interface.
The method of claim 1, wherein the first target interface is configured with an application program interface, and controlling the first target interface to respond to the first one-hand touch operation includes:

Determine the target application according to the first one-hand touch operation;

By calling the interface of the target application program, the first target interface is controlled to respond to the first one-hand touch operation.
The method of claim 1, further comprising:

When it is detected that the user performs a second one-hand touch operation in the display interface, the first target interface is controlled to respond to the second one-hand touch operation.
The method according to claim 1, characterized in that, if the first target interface is the display interface and the second target interface is the floating window interface, then after detecting that the user is in the When a first one-hand touch operation is performed in the floating window interface, controlling the first target interface to respond to the first one-hand touch operation includes:

Map the first one-handed touch operation to the display interface;

The display interface is controlled to respond to the mapping result of the first one-hand touch operation.
The method according to any one of claims 1 to 4, characterized in that displaying the floating window interface on the upper layer of the display interface in the target display area on the screen of the electronic display device includes:

Obtain the user's touch data on the electronic display device;

According to the touch data, determine the gesture of the user holding the electronic display device with one hand;

Determine the operation area where the user's hand holding the electronic display device with one hand performs a one-handed touch operation on the screen of the electronic display device according to the gesture of the user holding the electronic display device with one hand;

Determine the target display area according to the operation area;

The floating window interface is displayed on an upper layer of the display interface in the target display area.
The method of claim 5, wherein the touch data includes a touch image, and determining a gesture of the user holding the electronic display device with one hand based on the touch data includes:

The touch image is input into the trained gesture recognition model, and a recognition result output by the gesture recognition model representing the gesture of the user holding the electronic display device with one hand is obtained.
The method of claim 5, wherein the electronic display device includes an electromagnetic wave energy absorption ratio sensor, and the touch data includes sensor data collected by the electromagnetic wave energy absorption ratio sensor. Data to determine the gesture of the user holding the electronic display device with one hand, including:

Determine the contact area between the user and the electronic display device based on the sensor data;

According to the position distribution of the contact area on the electronic display device, a gesture of the user holding the electronic display device is determined.
A display control device for an electronic display device, characterized in that the device includes:

a generating module configured to generate a floating window interface according to the display interface in the current display state of the electronic display device in response to detecting an instruction to start the one-handed operation mode, where the floating window interface is a reduced size of the display interface The final interface;

The display module is configured to display the floating window interface on an upper layer of the display interface in a target display area on the screen of the electronic display device, wherein the target display area is based on the user holding a single-hand display. The gestures of the electronic display device are determined;

a response module configured to control the first target interface to respond to the first one-hand touch operation when it is detected that the user performs a first one-hand touch operation in the floating window interface, and the third A one-hand touch operation is an operation performed by the user holding the electronic display device with one hand;

A synchronization module configured to control the second target interface to perform synchronous changes according to changes in the first target interface, wherein the first target interface is one of the display interface and the floating window interface, so The second target interface is the other one of the display interface and the floating window interface.
A computer-readable storage medium on which a computer program is stored, characterized in that when the program is executed by a processor, the steps of the method described in any one of claims 1-7 are implemented.
An electronic display device, characterized by including:

A memory on which a computer program is stored;

A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1-7.