WO2024080593A1 - Screen display method, device, electronic apparatus and storage medium - Google Patents

Screen display method, device, electronic apparatus and storage medium Download PDF

Info

Publication number
WO2024080593A1
WO2024080593A1 PCT/KR2023/013946 KR2023013946W WO2024080593A1 WO 2024080593 A1 WO2024080593 A1 WO 2024080593A1 KR 2023013946 W KR2023013946 W KR 2023013946W WO 2024080593 A1 WO2024080593 A1 WO 2024080593A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
screen
display content
split
area
Prior art date
Application number
PCT/KR2023/013946
Other languages
French (fr)
Inventor
Zhifan QIU
Original Assignee
Samsung Electronics Co., Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Samsung Electronics Co., Ltd. filed Critical Samsung Electronics Co., Ltd.
Publication of WO2024080593A1 publication Critical patent/WO2024080593A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14Display of multiple viewports
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/38Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory with means for controlling the display position
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00Aspects of display data processing
    • G09G2340/04Changes in size, position or resolution of an image
    • G09G2340/0464Positioning
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2354/00Aspects of interface with display user

Definitions

  • the present disclosure relates to the field of computer technology, and in particular, to a screen display method, device, electronic apparatus, and storage medium.
  • the present disclosure provides a screen display method, device, electronic apparatus, and storage medium to solve at least the above problems in the related art, and may not solve any of the above problems.
  • the technical solutions of the present disclosure are as follows.
  • a screen display method includes: displaying first display content in a display area of a terminal screen; determining target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content; displaying the target display content completely in a target split-screen area after the terminal screen is split.
  • the displaying of the target display content completely in the target split-screen area after the terminal screen is split includes: in response to that the target display content cannot be completely displayed in the target split-screen area, adjusting display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
  • the determining of the target display content in the first display content includes: obtaining visual focus position information of a user in the display area of the terminal screen; determining the target display content based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
  • the visual focus position information is obtained through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
  • the preset condition includes at least one of a program to which the first display content belongs is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
  • the visual focus position is determined by one of determining the visual focus position by executing a gaze tracking algorithm utilizing a camera device associated with the terminal; receiving a selection input of the visual focus position in the display area of the terminal screen by the user; determining a center position in the display area of the terminal screen as the video focus position.
  • the determining of the target display content based on the first display content and the visual focus position information includes: acquiring page layout information of the first display content; determining component information of the area where the visual focus position is located, based on the page layout information; determining the target display content based on the component information of the area where the visual focus position is located.
  • the determining of the target display content in the first display content includes: determining a video that is currently displayed in the first display content as the target display content.
  • the method further includes obtaining area information of the target display content and area information of the target split-screen area; determining whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
  • the area information of the target split-screen area is position information of the target split-screen area
  • the area information of the target display content is position information of the target display content
  • the determining of whether the target display content can be completely displayed in the target split-screen area based on the comparison between the area information of the target display content and the area information of the target split-screen area includes: determining that the target display content cannot be completely displayed in the target split-screen area in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area.
  • the area information of the target split-screen area is a size and position information of the target split-screen area
  • the adjusting of the display content in the target split-screen area includes: determining a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
  • the determining of the split-screen adjustment strategy based on the comparison between the size of the target display content and the size of the target split-screen area and/or whether the target page supports scrolling display includes determining the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the size of the target display content is larger than the size of the target split-screen area, or the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display; determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display.
  • the adjusting of the display content in the target split-screen area includes: displaying the target display content on a target page in a form of a floating window, wherein the target page is a page where the display content in the target split-screen area is located.
  • the adjusting of the display content in the target split-screen area includes: determining a split-screen adjustment strategy based on whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
  • the determining of the split-screen adjustment strategy based on whether the target page supports scrolling display includes: determining the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the target page does not support scrolling display; determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the target page supports scrolling display.
  • a screen display device includes: a full-screen display unit configured to display first display content in a display area of a terminal screen; a target determination unit configured to determine target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content; a split-screen display unit configured to display the target display content completely in a target split-screen area after the terminal screen is split.
  • the split-screen display unit is configured to: in response to that the target display content cannot be completely displayed in the target split-screen area, adjust display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
  • the target determination unit is configured to: obtain visual focus position information of a user in the display area of the terminal screen; determine the target display content based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
  • the visual focus position information is obtained through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
  • the preset condition includes at least one of a program to which the first display content belongs is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
  • the visual focus position is determined by one of determining the visual focus position by executing a gaze tracking algorithm utilizing a camera device associated with the terminal; receiving a selection input of the visual focus position in the display area of the terminal screen by the user; determining a center position in the display area of the terminal screen as the video focus position.
  • the target determination unit is configured to acquire page layout information of the first display content; determine component information of the area where the visual focus position is located, based on the page layout information; determine the target display content based on the component information of the area where the visual focus position is located.
  • the target determination unit is configured to determine a video that is currently displayed in the first display content as the target display content.
  • the device further includes a complete display determination unit, the complete display determination unit is configured to obtain area information of the target display content and area information of the target split-screen area; determine whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
  • the area information of the target split-screen area is position information of the target split-screen area
  • the area information of the target display content is position information of the target display content
  • the complete display determination unit is configured to determine that the target display content cannot be completely displayed in the target split-screen area in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area.
  • the area information of the target split-screen area is a size and position information of the target split-screen area
  • the area information of the target display content is visual focus position information of a user in the display area of the terminal screen, wherein display content of an area where the visual focus position is located is the target display content
  • the complete display determination unit is configured to determine a complete display area in the target split-screen area based on the size and the position information of the target split-screen area; determine that the target display content cannot be completely displayed in the target split-screen area in the case that the visual focus position is outside the complete display area.
  • the split-screen display unit is configured to determine a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
  • the split-screen display unit is configured to determine the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the size of the target display content is larger than the size of the target split-screen area, or the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display; determine the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display.
  • the split-screen display unit is configured to display the target display content on a target page in a form of a floating window, wherein the target page is a page where the display content in the target split-screen area is located.
  • the split-screen display unit is configured to determine a split-screen adjustment strategy based on whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
  • the split-screen display unit is configured to determine the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the target page does not support scrolling display; determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the target page supports scrolling display.
  • an electronic apparatus including: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to execute the screen display method according to the present disclosure.
  • a computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to execute the screen display method according to the present disclosure.
  • the target split-screen area is adjusted so that the target display content can be completely displayed in the target split-screen area.
  • the target display content is the content that the user is viewing before screen splitting
  • the content viewed by the user can be completely displayed after screen splitting, and there is no impact on the content viewed by the user before and after screen splitting.
  • the target display content is the display content of the area where the visual focus position is located, which can maintain the consistency and integrity of the user's visual focus content before and after screen splitting.
  • FIG. 1 is a schematic diagram of a split-screen display in the related art.
  • FIG. 2 is a schematic diagram of another split-screen display in the related art.
  • FIG. 3 is a flow chart of a screen display method according to an exemplary embodiment.
  • FIG. 4 is a flow chart of determining target display content according to an exemplary embodiment.
  • FIG. 5 is a schematic diagram showing target display content according to an exemplary embodiment.
  • FIG. 6a, FIG. 6b and FIG. 6c are schematic diagrams showing a comparison before and after adjustment according to an exemplary embodiment.
  • FIG. 7a, FIG. 7b and FIG. 7c are another schematic diagrams showing a comparison before and after adjustment according to an exemplary embodiment.
  • FIG. 8 is a flow chart of another screen display method according to an exemplary embodiment.
  • FIG. 9 is a block diagram of a screen display device according to an exemplary embodiment.
  • FIG. 10 is a block diagram of an electronic apparatus 1000 according to an exemplary embodiment.
  • At least one of several items in the present disclosure all means including three parallel cases of “any one of the several items", “a combination of any of the several items", “all of the several items”.
  • “including at least one of A and B” includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B.
  • Another example is to "execute at least one of step 1 and step 2", which means the following three parallel cases of: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.
  • FIG. 1 is a schematic diagram of a split-screen display in the related art.
  • the content the user is viewing is a video on a page in FIG. 1.
  • the display area is changed, as shown in FIG. 1, the video the user is viewing is not fully displayed.
  • FIG. 2 is a schematic diagram of another split-screen display in the related art.
  • the content the user is viewing is part of text content on a page as shown in FIG. 2, and the user selects the split-screen display (up and down split-screens), the display area is changed, as shown in FIG. 2, the text content that the user is viewing is not displayed.
  • the present disclosure provides a screen display method, device, electronic apparatus and storage medium.
  • display content in the target split-screen area is adjusted so that the target display content can be completely displayed in the adjusted target split-screen area.
  • the target display content is content that a user is viewing before screen splitting
  • the content viewed by the user can be completely displayed after screen splitting, and there is no impact on the content viewed by the user before and after screen splitting.
  • Fig. 3 is a flow chart of a screen display method according to an exemplary embodiment.
  • first display content may be displayed in a display area of a terminal screen.
  • the first display content may be content currently displayed in the display area of the terminal screen.
  • the terminal may be, but is not limited to, a smart phone.
  • the terminal screen may be, but is not limited to, a straight screen, a curved screen, and a foldable screen, wherein, when the terminal screen is the foldable screen, performing split-screen display on the terminal screen may be, but not limited to, folding the terminal screen.
  • step 302 in response to performing split-screen display on the terminal screen, target display content in the first display content is determined, wherein the target display content is at least part of the first display content. For example, when a user folds the terminal screen, the terminal screen may be triggered to be split.
  • the present disclosure is not limited to this, and other conditions may also be used to trigger the terminal screen to be split.
  • the step of determining the target display content in the first display content may also be performed before performing split-screen display on the terminal screen.
  • the target display content may be content that the user is viewing within the display area of the terminal screen.
  • FIG. 4 is a flow chart of determining target display content according to an exemplary embodiment.
  • step 401 visual focus position information of a user in a display area of a terminal screen may be acquired.
  • the visual focus position information may include, but is not limited to, visual focus position coordinates.
  • the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal. That is, the visual focus position may be determined by executing the gaze tracking algorithm utilizing the camera device associated with the terminal.
  • the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
  • the camera device associated with the terminal may be, but not limited to, a front camera of the terminal and a camera of another terminal connected to the terminal, where the another terminal may be smart glasses (e.g., AR glasses).
  • the visual focus position information may be determined by a gaze tracking algorithm according to eye information of a user identified by the smart glasses based on collected eye images of the user.
  • the preset condition may include, but is not limited to, at least one of: a program to which the first display content belongs (a program currently being used by a user) is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
  • programs in the preset program set may be programs belonging to a white list, such as but not limited to common split-screen programs, news or social programs, and the like.
  • the fact that the terminal does not move may be determined by using a motion sensor of the terminal to detect that the terminal does not move relatively for a period of time.
  • the states in the preset state set may include, but are not limited to, the terminal plays a video in full screen, the terminal is in a power saving or low power mode, and the terminal is in a split-screen state.
  • the preset condition includes that the duration that the user watches the terminal screen exceeds the preset threshold before the split-screen display occurs, the acquired visual focus position information can be more accurate and effective.
  • the program is a program in the preset program set
  • the terminal does not move
  • the state of the terminal does not belong to a state in a preset state set
  • the user is a user authorized to collect images.
  • a front camera of the terminal is turned on to collect the user's (face) image in real time during the use of the program, and based on the user's (face) image, the user's eye information (which may include eyeballs, wrinkles, iris and other information) may be identified, and then the user's real-time visual focus position information is determined based on the user's eye information through the gaze tracking algorithm (biometric identification through the eyeballs, wrinkles, iris and other information), so that when performing split-screen display on the terminal screen, the visual focus position information determined at the current moment may be obtained.
  • the visual focus position information determined at the current moment is obtained.
  • the visual focus position information may be corrected to obtain more accurate visual focus position information.
  • the visual focus position may be determined by receiving a selection input of a visual focus position within the display area of the terminal screen by the user.
  • the visual focus position information may be acquired by displaying prompt information for prompting the user to select a visual focal point in the display area of the terminal screen, receiving the visual focal point selected by the user based on the prompt information, and determining the visual focus position information based on the visual focal point.
  • a preset position within the display area of the terminal screen may be determined as the video focus position.
  • the visual focus position information may be acquired by using preset position information in the display area of the terminal screen (e.g., center position information in the display area of the terminal screen) as the visual focus position information.
  • the target display content may be determined based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
  • the target display content may be determined by locating a component corresponding to an area where the visual focal point is located. Specifically: firstly, page layout information of the first display content may be obtained; then, component information of the area where the visual focus position is located may be determined based on the page layout information; finally, the target display content may be determined based on the component information of the area where the visual focus position is located.
  • layout tree structure of a page where the first display content is located may be obtained through an Android uiautomator command, and hierarchy information, component information, and size and position information of each component of the page where the first display content is located are determined based on the layout tree structure, and the above information is used as the page layout information. Then, based on the page layout information, the component information of the area where the visual focus position is located (including but not limited to a component name, display content of the component on the page, a size and position information of a display area of the component on the page) may be determined. Finally, the target display content may be determined based on the component information of the area where the visual focus position is located.
  • some components may be configured to display different content, for example, some components may be configured to display both still images and dynamic images, in order to determine the target display content more accurately, judgment of a dynamic update area (that is, the area where the dynamic image is displayed) may be increased.
  • the page layout information of the first display content may be obtained, and it may be determined whether the page where the first display content is located exists the dynamic update area; then, if there is a dynamic update area and the area where the visual focus position is located is the dynamic update area, it may be determined that the target display content is a dynamic image; if there is no dynamic update area, or there is a dynamic update area but the area where the visual focus position is located is not the dynamic update area, the component information of the area where the visual focus position is located may be determined based on the page layout information, and the target display content may be determined based on the component information of the area where the visual focus position is located.
  • layout tree structure of a page where the first display content is located may be obtained through an Android uiautomator command, and hierarchy information, component information, and size and position information of each component of the page where the first display content is located are determined based on the layout tree structure, the above information is used as the page layout information, and a dynamic update area is obtained through an Android SurfaceFlinger command. If the dynamic update area obtained through the Android SurfaceFlinger command is not empty, it is determined that there is a dynamic update area, and a size and position information of the dynamic update area may be obtained, otherwise it is determined that there is no dynamic update area.
  • the target display content is a dynamic image; if there is no dynamic update area, or there is a dynamic update area but the area where the visual focus position is located is not the dynamic update area, the component information (including but not limited to the component name, the display content of the component on the page, and the size and position information of the display area of the component on the page) of the area where the visual focus position is located may be determined based on the page layout information.
  • the target display content may be determined based on the component information of the area where the visual focus position is located. For example, the component name is ImageView, it may be determined that the target display content is a still image.
  • the visual focus position used may be visual focus position coordinates (X, Y), or Y value (Y-axis coordinate value) in the visual focus position coordinates (X, Y).
  • FIG. 5 is a schematic diagram of target display content according to an exemplary embodiment.
  • the visual focus position is the Y value in the visual focus position coordinates (X, Y)
  • the target display content is a video (dynamic image) in the dotted box.
  • the above visual focus tracking may also not be performed, and the currently displayed video in the display area of the terminal screen may be directly determined as the target display content.
  • the currently displayed video may be a fully displayed video or a partially displayed video in the display area of the terminal screen.
  • the target display content may be completely displayed in a target split-screen area after the terminal screen is split.
  • the terminal screen may be split into multiple split-screen areas, and the target split-screen area is one of the multiple split-screen areas.
  • Each split-screen area may display a part of the first display content.
  • the display content in the target split-screen area is adjusted so that the target display content may be completely displayed in the target split-screen area.
  • split-screen display may be performed on the terminal screen, and it is determined whether the target display content may be completely displayed in the target split-screen area.
  • area information of the target display content and area information of the target split-screen area may be obtained firstly; whether the target display content can be completely displayed in the target split-screen area may be determined based on a comparison between the area information of the target display content and the area information of the target split-screen area.
  • the target display content can be, but not limited to, images or non-images.
  • the images may include, but are not limited to, still images and dynamic images, wherein the dynamic images may be, but are not limited to, videos or gifs.
  • non-images may include, but are not limited to, text and the like.
  • the area information of the target split-screen area may be position information of the target split-screen area, and the area information of the target display content may be position information of the target display content.
  • the position information of the target split-screen area may be position information of remaining areas other than content fixedly occupying the target split-screen area (such as the navigation bar) in the target split-screen area.
  • the position information of the target split-screen area may be, but not limited to, position information of each point in the target split-screen area.
  • the position information of the target display content may be, but not limited to, position information of each point in the target display content.
  • the position information may include, but is not limited to, position coordinates.
  • the target display content in the case where there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area (that is, the position of the point in the target display content is not within the range of the target split-screen area), it is determined that the target display content cannot be completely displayed in the target split-screen area.
  • the position information of each point in the target display content can be found in the position information of the target split-screen area (that is, the position of each point in the target display content is within the range of the target split-screen area)
  • it is determined that the target display content can be completely displayed in the target split-screen area.
  • the target split-screen area may be a rectangular area, and if the target display content is an image, the target display content may also be a rectangular area.
  • the position information of the target split-screen area may be, but not limited to, position information of four vertices of the target split-screen area
  • the position information of the target display content may be, but not limited to, position information of four vertices of the target display content.
  • the position information may include, but is not limited to, position coordinates. Then, by comparing the position information of the four vertices of the target split-screen area and the position information of the four vertices of the target display content, it may be determined whether the target display content can be completely displayed in the target split-screen area.
  • the area information of the target split-screen area may also be a size and position information of the target split-screen area, and the area information of the target display content may also be a size and position information of the target display content. Based on a comparison between the size and the position information of the target display content and the size and the position information of the target split-screen area, it is determined whether the target display content can be completely displayed in the target split-screen area.
  • the area information of the target split-screen area is a size and position information of the target split-screen area
  • the area information of the target display content is visual focus position information of a user in the display area of the terminal screen (i.e., the visual focus position information obtained in the above steps)
  • the display content of the area where the visual focus position is located is the target display content.
  • a complete display area within the target split-screen area may be determined based on the size and the position information of the target split-screen area; then, in the case where the visual focus position is outside the complete display area, it is determined that the target display content cannot be completely displayed in the target split-screen area.
  • the visual focus position is located in the complete display area, it is determined that the target display content can be completely displayed in the target split-screen area.
  • the target display content is not an image
  • the display content in the target split-screen area may not be adjusted.
  • the target display content is an image or a non-image
  • the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split
  • a split-screen adjustment strategy may be determined based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and the display content in the target split-screen area is adjusted based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
  • the split-screen adjustment strategy may be determined as adjusting a current split-screen ratio and/or adjusting a current split-screen direction; if the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display, the split-screen adjustment strategy is determined as adjusting a display position of the target display content in the target split-screen area.
  • a target distance and a target direction may be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page may be scrolled based on the target distance and target direction, thereby adjusting the display position of the target display content in the target split-screen area.
  • the current split-screen ratio is 0.5:0.5
  • the current split-screen direction is up and down split-screens
  • the current split-screen ratio (for example, the split-screen ratio is adjusted to 0.7:0.3) may be adjusted and/or the split-screen direction may adjusted to left and right split-screens.
  • the target display content is an image
  • the size of the target display content is larger than the size of the target split-screen area
  • the target page supports scrolling display it may be determined whether the target page needs to be scrolled, and if the target page needs to be scrolled, the target page is scrolled and the split-screen adjustment strategy of adjusting the current split-screen ratio and/or adjusting the current split-screen direction is performed.
  • the target split-screen area may be a rectangular area, and if the target display content is an image, the target display content may also be a rectangular area.
  • a Y-axis coordinate value of a top edge of the target display content may be obtained based on the position information of the target display content firstly (the Y-axis coordinate value of the position coordinates of each point on the top edge is the same), and a Y-axis coordinate value of a top edge of the target split-screen area may be obtained based on the position information of the target split-screen area (the Y-axis coordinate value of the position coordinates of each point on the top edge is the same); then a difference between the two Y-axis coordinate values is calculated, the target direction is determined according to a sign of the difference and the position information of a coordinate origin, the target distance is determined according to a absolute value of the difference, and the target page
  • FIG. 6 is a schematic diagram showing a comparison before and after adjustment according to an exemplary embodiment
  • FIG. 6(a) is a schematic diagram before split-screen display is performed
  • the target display content is a video in the figure
  • the size of the target display content is not larger than the size of the split-screen area and the target page may be scrolled.
  • FIG. 6(b) is a schematic diagram before a split-screen adjustment strategy is performed after split-screen display is performed
  • FIG. 6(c) is a schematic diagram after a split-screen adjustment strategy is performed.
  • the target display content may also be displayed on the target page in the form of a floating window, where the target page is a page where the display content in the target split-screen area is located.
  • a split-screen adjustment strategy may be determined based on whether the target page supports scrolling display, and based on the determined split-screen adjustment strategy, the display content in the target split-screen area may be adjusted, wherein the target page is a page where the display content in the target split-screen area is located.
  • the split-screen adjustment strategy may be determined as adjusting the current split-screen ratio and/or adjusting the current split-screen direction; if the target page supports scrolling display, the split-screen adjustment strategy is determined as adjusting the display position of the target display content in the target split-screen area.
  • the target distance and the target direction may be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page may be scrolled based on the target distance and the target direction, thereby adjusting the display position of the target display content in the target split-screen area.
  • the current split-screen ratio is 0.5:0.5
  • the current split-screen direction is up and down split-screens
  • the current split-screen ratio may be adjusted (for example, the split-screen ratio is adjusted to 0.7:0.3) and/or the split-screen direction is adjusted to left and right split-screens.
  • the target page may be scrolled so that the target display content is scrolled to the center of the target split-screen area.
  • the target distance and the target direction may be determined in the following manner: in the case that the target display content is determined based on the visual focus position information, firstly, the Y value in the visual focus position coordinates (X, Y) and the Y-axis coordinate value corresponding to the center position of the target split-screen area may be obtained respectively, then the difference between the two Y-axis coordinate values is calculated, and the target direction is determined according to the sign of the difference and the position information of the coordinate origin, the target distance is determined based on the absolute value of the difference, and then the target page may be scrolled by injecting the input event based on the target distance and target direction.
  • the target distance and the target direction may also be determined in the following manner: firstly, the Y-axis coordinate value corresponding to the center position of the target display content and the Y-axis coordinate value corresponding to the center position of the target split-screen area may be obtained respectively, and then the difference between the two Y-axis coordinate values is calculated, the target direction is determined according to the sign of the difference and the position information of the coordinate origin, and the target distance is determined according to the absolute value of the difference.
  • FIG. 7 is another schematic diagram before and after adjustment according to an exemplary embodiment
  • FIG. 7(a) is a schematic diagram before split-screen display is performed, the target display content is part of text content in the figure, and the target page supports scrolling display
  • FIG. 7(b) is a schematic diagram before a split-screen adjustment strategy is performed after split-screen display is performed
  • FIG. 7(c) is a schematic diagram after a split-screen adjustment strategy is performed.
  • FIG. 8 is a flow chart of another screen display method according to an exemplary embodiment.
  • split-screen display may be performed on the terminal screen.
  • the target display content is a dynamic image
  • a target distance and a target direction may be determined based on position information of the target display content and position information of the target split-screen area, and the target page may be scrolled based on the target distance and the target direction, thereby adjusting the display position of the target display content in the target split-screen area.
  • the target display content is not a dynamic image
  • the target display content is a still image
  • the same steps as those of the target display content being a dynamic image are performed, which will not be repeated here.
  • the target display content is not a still image (it has been determined that the target display content is not a dynamic image at this time)
  • the split-screen adjustment strategy is determined as adjusting the current split-screen ratio and/or adjusting the current split-screen direction; if the target page supports scroll display, the split-screen adjustment strategy is determined as adjusting the display position of the target display content in the target split-screen area.
  • the target distance and target direction may be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page may be scrolled based on the target distance and target direction, thereby adjusting the display position of the target display content in the target split-screen area.
  • FIG. 9 is a block diagram of a screen display device according to an exemplary embodiment.
  • the screen display device 900 includes a full-screen display unit 901, a target determination unit 902 and a split-screen display unit 903.
  • the full-screen display unit 901 may display first display content in a display area of a terminal screen.
  • the first display content may be content currently displayed in the display area of the terminal screen.
  • the terminal may be, but is not limited to, a smart phone.
  • the terminal screen may be, but is not limited to, a straight screen, a curved screen, and a foldable screen, wherein, when the terminal screen is the foldable screen, performing split-screen display on the terminal screen may be, but not limited to, folding the terminal screen.
  • the target determination unit 902 may determine target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content. For example, when a user folds the terminal screen, the terminal screen may be triggered to be split.
  • the present disclosure is not limited to this, and other conditions may also be used to trigger the terminal screen to be split.
  • the target determination unit 902 may perform the step of determining the target display content in the first display content before performing split-screen display on the terminal screen.
  • the target display content may be content that the user is viewing within the display area of the terminal screen.
  • the content that the user is viewing may be determined by a user's visual focal point.
  • the target determination unit 902 may acquire visual focus position information of the user in the display area of the terminal screen.
  • the visual focus position information may include, but is not limited to, visual focus position coordinates.
  • the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal. That is, the visual focus position may be determined by executing the gaze tracking algorithm utilizing the camera device associated with the terminal.
  • the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
  • the camera device associated with the terminal may be, but not limited to, a front camera of the terminal and a camera of another terminal connected to the terminal, where the another terminal may be smart glasses (e.g., AR glasses).
  • the visual focus position information may be determined by a gaze tracking algorithm according to eye information of a user identified by the smart glasses based on collected eye images of the user.
  • the preset condition may include, but is not limited to, at least one of: a program to which the first display content belongs (a program currently being used by a user) is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
  • programs in the preset program set may be programs belonging to a white list, such as but not limited to common split-screen programs, news or social programs, and the like.
  • the fact that the terminal does not move may be determined by using a motion sensor of the terminal to detect that the terminal does not move relatively for a period of time.
  • the states in the preset state set may include, but are not limited to, the terminal plays a video in full screen, the terminal is in a power saving or low power mode, and the terminal is in a split-screen state.
  • the preset condition includes that the duration that the user watches the terminal screen exceeds the preset threshold before the split-screen display occurs, the acquired visual focus position information can be more accurate and effective.
  • the target determination unit 902 may determine the visual focus position by receiving a selection input of a visual focus position within the display area of the terminal screen by the user.
  • the visual focus position information can be acquired by displaying prompt information for prompting the user to select a visual focal point in the display area of the terminal screen, receiving the visual focal point selected by the user based on the prompt information, and determining the visual focus position information based on the visual focal point.
  • the target determination unit 902 may determine a preset position (e.g., but not limited to, a central position) within the display area of the terminal screen as the video focus position.
  • a preset position e.g., but not limited to, a central position
  • the visual focus position information may be acquired by using preset position information in the display area of the terminal screen (e.g., center position information in the display area of the terminal screen) as the visual focus position information.
  • the determining unit 901 may determine the target display content based on the first display content and the visual focus position information, where the target display content is display content in an area where the visual focus position is located.
  • the determination unit 901 may determine the target display content by locating a component corresponding to an area where the visual focus is located. Specifically: firstly, the determination unit 901 may obtain page layout information of the first display content; then, the determination unit 901 may determine component information of the area where the visual focus position is located based on the page layout information; finally, the determination unit 901 may determine the target display content based on the component information of the area where the visual focus position is located.
  • the determining unit 901 may obtain layout tree structure of a page where the first display content is located through an Android uiautomator command, and determine hierarchy information, component information, and size and position information of each component of the page where the first display content is located based on the layout tree structure, and use the above information as the page layout information. Then, the determining unit 901 may determine the component information of the area where the visual focus position is located (including but not limited to a component name, display content of the component on the page, a size and position information of a display area of the component on the page) based on the page layout information. Finally, the determining unit 901 may determine the target display content based on the component information of the area where the visual focus position is located.
  • the visual focus position may be visual focus position coordinates (X, Y), or Y value (Y-axis coordinate value) in the visual focus position coordinates (X, Y).
  • the target determination unit 902 may also directly determine a video currently displayed in the display area of the terminal screen as the target display content without performing the above visual focus tracking.
  • the currently displayed video may be a fully displayed video or a partially displayed video in the display area of the terminal screen.
  • the split-screen display unit 903 may completely display the target display content in the target split-screen area after the terminal screen is split.
  • the terminal screen can be split into multiple split-screen areas, and the target split-screen area is one of the multiple split-screen areas.
  • Each split-screen area may display a part of the first display content.
  • the split-screen display unit 903 in response to that the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split, the split-screen display unit 903 adjusts the display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
  • the split-screen display unit 903 before performing an operation, may perform split-screen display on the terminal screen, and determine whether the target display content can be completely displayed in the target split-screen area.
  • the screen display device 900 also includes a complete display determination unit (not shown), the complete display determination unit may obtain area information of the target display content and area information of the target split-screen area firstly; the complete display determination unit may then determine whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
  • the target display content can be, but not limited to, images or non-images.
  • the images may include, but are not limited to, still images and dynamic images, wherein the dynamic images may be, but are not limited to, videos or gifs.
  • non-images may include, but are not limited to, text and the like.
  • the area information of the target split-screen area may be position information of the target split-screen area
  • the area information of the target display content may be position information of the target display content
  • the position information of the target split-screen area may be, but not limited to, position information of each point in the target split-screen area.
  • the position information of the target display content may be, but not limited to, position information of each point in the target display content.
  • the position information may include, but is not limited to, position coordinates.
  • the target display content in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area (that is, the position of the point in the target display content is not within the range of the target split-screen area), it is determined that the target display content cannot be completely displayed in the target split-screen area.
  • the position information of each point in the target display content can be found in the position information of the target split-screen area (that is, the position of each point in the target display content is within the range of the target split-screen area)
  • it is determined that the target display content can be completely displayed in the target split-screen area.
  • the target split-screen area may be a rectangular area, and if the target display content is an image, the target display content may also be a rectangular area.
  • the position information of the target split-screen area may be, but not limited to, position information of four vertices of the target split-screen area
  • the position information of the target display content may be, but not limited to, position information of four vertices of the target display content.
  • the position information may include, but is not limited to, position coordinates. Then, by comparing the position information of the four vertices of the target split-screen area and the position information of the four vertices of the target display content, it can be determined whether the target display content can be completely displayed in the target split-screen area.
  • the area information of the target split-screen area can also be a size and position information of the target split-screen area
  • the area information of the target display content can also be a size and position information of the target display content. Based on a comparison between the size and the position information of the target display content and the size and the position information of the target split-screen area, it is determined whether the target display content can be completely displayed in the target split-screen area.
  • the area information of the target split-screen area is a size and position information of the target split-screen area
  • the area information of the target display content is visual focus position information of a user in the display area of the terminal screen (i.e., the visual focus position information obtained in the above steps), wherein the display content of the area where the visual focus position is located is the target display content.
  • the complete display determination unit may determine a complete display area within the target split-screen area based on the size and the position information of the target split-screen area; then, the complete display determination unit may determine, in the case where the visual focus position is outside the complete display area, that the target display content cannot be completely displayed in the target split-screen area, and determine, in the case where the visual focus position is located in the complete display area, that the target display content can be completely displayed in the target split-screen area.
  • the split-screen display unit 903 may not adjust the display content in the target split-screen area.
  • the split-screen display unit 903 may determine a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjust the display content in the target split-screen area based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
  • the split-screen display unit 903 may determine the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction; if the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display, determine the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area.
  • a target distance and a target direction can be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page can be scrolled based on the target distance and target direction, thereby adjusting the display position of the target display content in the target split-screen area.
  • the split-screen display unit 903 may also display the target display content on the target page in the form of a floating window, where the target page is a page where the display content in the target split-screen area is located.
  • the split-screen display unit 903 may determine a split-screen adjustment strategy based on whether the target page supports scrolling display, and adjust the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
  • the split-screen display unit 903 may determine the split-screen adjustment strategy as adjusting the current split-screen ratio and/or adjusting the current split-screen direction; if the target page supports scrolling display, the split-screen display unit 903 may determine the split-screen adjustment strategy as adjusting the display position of the target display content in the target split-screen area.
  • the target distance and the target direction can be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page can be scrolled based on the target distance and the target direction, thereby adjusting the display position of the target display content in the target split-screen area.
  • FIG. 10 is a block diagram of an electronic apparatus 1000 according to an exemplary embodiment.
  • the electronic apparatus 1000 includes at least one memory 1001 and at least one processor 1002, the at least one memory 1001 stores a computer-executable instruction set, when the computer-executable instruction set is executed by the at least one processor 1002, the screen display method according to the present disclosure is executed.
  • the electronic apparatus 1000 may be a PC computer, a tablet device, a personal digital assistant, a smart phone, or any other device capable of executing the above instruction set.
  • the electronic apparatus 1000 does not have to be a single electronic device, but may also be any set of devices or circuits capable of executing the above instructions (or instruction set) individually or jointly.
  • the electronic apparatus 1000 may also be a part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces locally or remotely (e.g., via wireless transmission).
  • the processor 1002 may include a central processing unit (CPU), graphics processing unit (GPU), programmable logic device, special purpose processor system, microcontroller or microprocessor.
  • the processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.
  • the processor 1002 may execute instructions or code stored in the memory 1001, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transport protocol.
  • the memory 1001 may be integrated with the processor 1002, e.g., a RAM or flash memory is arranged within an integrated circuit microprocessor or the like. Additionally, the memory 1001 may include a separate device such as an external disk drive, storage array, or any other storage device that may be used by a database system. The memory 1001 and the processor 1002 may be operatively coupled, or may communicate with each other, e.g., through I/O ports, network connections, etc., to enable the processor 1002 to read files stored in the memory.
  • the electronic apparatus 1000 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of electronic apparatus 1000 may be connected to each other via a bus and/or network.
  • a video display such as a liquid crystal display
  • a user interaction interface such as a keyboard, mouse, touch input device, etc.
  • a computer-readable storage medium storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the screen display method according to the present disclosure.
  • Examples of computer-readable storage media herein include: Read Only Memory (ROM), Random Access Programmable Read Only Memory (RAPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blue-ray or optical disk storage, Hard Disk Drive (HDD), Solid State Drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards or extremely fast digital (XD) cards), magnetic tape
  • the instructions or computer programs in the computer-readable storage medium described above may be executed in an environment deployed in a computer device.
  • the computer programs and any associated data, data files, and data structures are distributed on a networked computer system, so that the computer programs and any associated data, data files, and data structures are stored, accessed and executed through one or more processors or computers in a distributed manner.
  • the target split-screen area is adjusted so that the target display content can be completely displayed in the target split-screen area.
  • the target display content is the content that the user is viewing before screen splitting
  • the content viewed by the user can be completely displayed after screen splitting, and there is no impact on the content viewed by the user before and after screen splitting.
  • the target display content is the display content of the area where the visual focus position is located, which can maintain the consistency and integrity of the user's visual focus content before and after screen splitting.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • User Interface Of Digital Computer (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

Provided are a a screen display method, device, electronic apparatus, and storage medium. The screen display method includes displaying first display content in a display area of a terminal screen; determining target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content; displaying the target display content completely in a target split-screen area after the terminal screen is split. The disclosure enables content viewed by a user to be completely displayed after screen splitting, and ensures that there is no impact on the content viewed by the user before and after screen splitting.

Description

SCREEN DISPLAY METHOD, DEVICE, ELECTRONIC APPARATUS AND STORAGE MEDIUM
The present disclosure relates to the field of computer technology, and in particular, to a screen display method, device, electronic apparatus, and storage medium.
With the development of electronic terminals, their achievable functions are also diverse. At present, most electronic terminals can not only display page content in full screen, but also provide a function of split-screen display. However, when a user selects split-screen display, a problem that the content that the user is viewing before screen splitting does not be displayed completely may occur.
The present disclosure provides a screen display method, device, electronic apparatus, and storage medium to solve at least the above problems in the related art, and may not solve any of the above problems. The technical solutions of the present disclosure are as follows.
According to a first aspect of the embodiments of the present disclosure, a screen display method is provided, the method includes: displaying first display content in a display area of a terminal screen; determining target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content; displaying the target display content completely in a target split-screen area after the terminal screen is split.
Alternatively, the displaying of the target display content completely in the target split-screen area after the terminal screen is split includes: in response to that the target display content cannot be completely displayed in the target split-screen area, adjusting display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
Alternatively, the determining of the target display content in the first display content includes: obtaining visual focus position information of a user in the display area of the terminal screen; determining the target display content based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
Alternatively, the visual focus position information is obtained through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
Alternatively, the preset condition includes at least one of a program to which the first display content belongs is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
Alternatively, the visual focus position is determined by one of determining the visual focus position by executing a gaze tracking algorithm utilizing a camera device associated with the terminal; receiving a selection input of the visual focus position in the display area of the terminal screen by the user; determining a center position in the display area of the terminal screen as the video focus position.
Alternatively, the determining of the target display content based on the first display content and the visual focus position information includes: acquiring page layout information of the first display content; determining component information of the area where the visual focus position is located, based on the page layout information; determining the target display content based on the component information of the area where the visual focus position is located.
Alternatively, the determining of the target display content in the first display content includes: determining a video that is currently displayed in the first display content as the target display content.
Alternatively, the method further includes obtaining area information of the target display content and area information of the target split-screen area; determining whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
Alternatively, in the case that the target display content is an image, the area information of the target split-screen area is position information of the target split-screen area, and the area information of the target display content is position information of the target display content; wherein the determining of whether the target display content can be completely displayed in the target split-screen area based on the comparison between the area information of the target display content and the area information of the target split-screen area includes: determining that the target display content cannot be completely displayed in the target split-screen area in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area.
Alternatively, in the case that the target display content is not an image, the area information of the target split-screen area is a size and position information of the target split-screen area, the area information of the target display content is visual focus position information of a user in the display area of the terminal screen, wherein display content of an area where the visual focus position is located is the target display content; wherein the determining of whether the target display content can be completely displayed in the target split-screen area based on the comparison between the area information of the target display content and the area information of the target split-screen area includes: determining a complete display area in the target split-screen area based on the size and the position information of the target split-screen area; determining that the target display content cannot be completely displayed in the target split-screen area in the case that the visual focus position is outside the complete display area.
Alternatively, in the case that the target display content is an image, the adjusting of the display content in the target split-screen area includes: determining a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
Alternatively, the determining of the split-screen adjustment strategy based on the comparison between the size of the target display content and the size of the target split-screen area and/or whether the target page supports scrolling display includes determining the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the size of the target display content is larger than the size of the target split-screen area, or the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display; determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display.
Alternatively, in the case that the target display content is an image, the adjusting of the display content in the target split-screen area includes: displaying the target display content on a target page in a form of a floating window, wherein the target page is a page where the display content in the target split-screen area is located.
Alternatively, in the case that the target display content is not an image, the adjusting of the display content in the target split-screen area includes: determining a split-screen adjustment strategy based on whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
Alternatively, the determining of the split-screen adjustment strategy based on whether the target page supports scrolling display includes: determining the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the target page does not support scrolling display; determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the target page supports scrolling display.
According to a second aspect of the embodiments of the present disclosure, a screen display device is provided. The device includes: a full-screen display unit configured to display first display content in a display area of a terminal screen; a target determination unit configured to determine target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content; a split-screen display unit configured to display the target display content completely in a target split-screen area after the terminal screen is split.
Alternatively, the split-screen display unit is configured to: in response to that the target display content cannot be completely displayed in the target split-screen area, adjust display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
Alternatively, the target determination unit is configured to: obtain visual focus position information of a user in the display area of the terminal screen; determine the target display content based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
Alternatively, the visual focus position information is obtained through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
Alternatively, the preset condition includes at least one of a program to which the first display content belongs is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
Alternatively, the visual focus position is determined by one of determining the visual focus position by executing a gaze tracking algorithm utilizing a camera device associated with the terminal; receiving a selection input of the visual focus position in the display area of the terminal screen by the user; determining a center position in the display area of the terminal screen as the video focus position.
Alternatively, the target determination unit is configured to acquire page layout information of the first display content; determine component information of the area where the visual focus position is located, based on the page layout information; determine the target display content based on the component information of the area where the visual focus position is located.
Alternatively, the target determination unit is configured to determine a video that is currently displayed in the first display content as the target display content.
Alternatively, the device further includes a complete display determination unit, the complete display determination unit is configured to obtain area information of the target display content and area information of the target split-screen area; determine whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
Alternatively, in the case that the target display content is an image, the area information of the target split-screen area is position information of the target split-screen area, and the area information of the target display content is position information of the target display content; wherein the complete display determination unit is configured to determine that the target display content cannot be completely displayed in the target split-screen area in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area.
Alternatively, in the case that the target display content is not an image, the area information of the target split-screen area is a size and position information of the target split-screen area, the area information of the target display content is visual focus position information of a user in the display area of the terminal screen, wherein display content of an area where the visual focus position is located is the target display content; the complete display determination unit is configured to determine a complete display area in the target split-screen area based on the size and the position information of the target split-screen area; determine that the target display content cannot be completely displayed in the target split-screen area in the case that the visual focus position is outside the complete display area.
Alternatively, in the case that the target display content is an image, the split-screen display unit is configured to determine a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
Alternatively, the split-screen display unit is configured to determine the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the size of the target display content is larger than the size of the target split-screen area, or the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display; determine the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display.
Alternatively, in the case that the target display content is an image, the split-screen display unit is configured to display the target display content on a target page in a form of a floating window, wherein the target page is a page where the display content in the target split-screen area is located.
Alternatively, in the case that the target display content is not an image, the split-screen display unit is configured to determine a split-screen adjustment strategy based on whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
Alternatively, the split-screen display unit is configured to determine the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the target page does not support scrolling display; determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the target page supports scrolling display.
According to a third aspect of the embodiments of the present disclosure, there is provided an electronic apparatus, including: at least one processor; at least one memory storing computer-executable instructions, wherein the computer-executable instructions, when executed by the at least one processor, cause the at least one processor to execute the screen display method according to the present disclosure.
According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to execute the screen display method according to the present disclosure.
The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:
According to the screen display method, device, electronic apparatus and storage medium of the present disclosure, after split-screen display, if the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split, the target split-screen area is adjusted so that the target display content can be completely displayed in the target split-screen area. In this way, when the target display content is the content that the user is viewing before screen splitting, the content viewed by the user can be completely displayed after screen splitting, and there is no impact on the content viewed by the user before and after screen splitting.
In addition, according to the screen display method, device, electronic apparatus and storage medium of the present disclosure, the target display content is the display content of the area where the visual focus position is located, which can maintain the consistency and integrity of the user's visual focus content before and after screen splitting.
It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.
The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure and do not unduly limit the present disclosure.
FIG. 1 is a schematic diagram of a split-screen display in the related art.
FIG. 2 is a schematic diagram of another split-screen display in the related art.
FIG. 3 is a flow chart of a screen display method according to an exemplary embodiment.
FIG. 4 is a flow chart of determining target display content according to an exemplary embodiment.
FIG. 5 is a schematic diagram showing target display content according to an exemplary embodiment.
FIG. 6a, FIG. 6b and FIG. 6c are schematic diagrams showing a comparison before and after adjustment according to an exemplary embodiment.
FIG. 7a, FIG. 7b and FIG. 7c are another schematic diagrams showing a comparison before and after adjustment according to an exemplary embodiment.
FIG. 8 is a flow chart of another screen display method according to an exemplary embodiment.
FIG. 9 is a block diagram of a screen display device according to an exemplary embodiment.
FIG. 10 is a block diagram of an electronic apparatus 1000 according to an exemplary embodiment.
In order to make those skilled in the art better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
It should be noted that terms "first", "second" and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequential order. It is to be understood that data so used may be interchanged under appropriate circumstances such that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the following examples are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of devices and methods consistent with some aspects of the present disclosure as recited in the appended claims.
It should be noted here that "at least one of several items" in the present disclosure all means including three parallel cases of "any one of the several items", "a combination of any of the several items", "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is to "execute at least one of step 1 and step 2", which means the following three parallel cases of: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.
With the development of electronic terminals, their achievable functions are also diverse. At present, most electronic terminals can not only display page content in a full screen, but also provide a split-screen display function. For example, with the continuous development of flexible screens, more and more users choose to use foldable screen terminals (such as foldable screen mobile phones), and the foldable screen terminals can be folded into a vertical interactive mode, so as to realize the split-screen display function and provide convenience for the users to operate.
However, when the user selects the split-screen display, the display area of the screen will be changed, which may cause the problem that the content that the user is viewing before screen splitting is displayed incompletely or even not displayed, and the user needs to manually adjust it. For example, FIG. 1 is a schematic diagram of a split-screen display in the related art. The content the user is viewing is a video on a page in FIG. 1. When the user selects the split-screen display (up and down split-screens), the display area is changed, as shown in FIG. 1, the video the user is viewing is not fully displayed. For another example, FIG. 2 is a schematic diagram of another split-screen display in the related art. The content the user is viewing is part of text content on a page as shown in FIG. 2, and the user selects the split-screen display (up and down split-screens), the display area is changed, as shown in FIG. 2, the text content that the user is viewing is not displayed.
In order to solve the above-mentioned problems in the related art, the present disclosure provides a screen display method, device, electronic apparatus and storage medium. After split-screen display, if target display content cannot be completely displayed in a target split-screen area after a terminal screen is split, display content in the target split-screen area is adjusted so that the target display content can be completely displayed in the adjusted target split-screen area. In this way, when the target display content is content that a user is viewing before screen splitting, the content viewed by the user can be completely displayed after screen splitting, and there is no impact on the content viewed by the user before and after screen splitting.
The screen display method, device, electronic apparatus and storage medium according to the present disclosure will be described in detail with reference to FIGS. 3 to 10.
Fig. 3 is a flow chart of a screen display method according to an exemplary embodiment.
Referring to FIG. 3, in step 301, first display content may be displayed in a display area of a terminal screen. Here, the first display content may be content currently displayed in the display area of the terminal screen.
According to an exemplary embodiment of the present disclosure, the terminal may be, but is not limited to, a smart phone. The terminal screen may be, but is not limited to, a straight screen, a curved screen, and a foldable screen, wherein, when the terminal screen is the foldable screen, performing split-screen display on the terminal screen may be, but not limited to, folding the terminal screen.
In step 302, in response to performing split-screen display on the terminal screen, target display content in the first display content is determined, wherein the target display content is at least part of the first display content. For example, when a user folds the terminal screen, the terminal screen may be triggered to be split. Of course, the present disclosure is not limited to this, and other conditions may also be used to trigger the terminal screen to be split.
According to an exemplary embodiment of the present disclosure, the step of determining the target display content in the first display content may also be performed before performing split-screen display on the terminal screen.
According to an exemplary embodiment of the present disclosure, the target display content may be content that the user is viewing within the display area of the terminal screen.
As an example, the content that the user is viewing may be determined by a user's visual focal point. FIG. 4 is a flow chart of determining target display content according to an exemplary embodiment.
Referring to FIG. 4, in step 401, visual focus position information of a user in a display area of a terminal screen may be acquired. Here, the visual focus position information may include, but is not limited to, visual focus position coordinates.
According to an exemplary embodiment of the present disclosure, the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal. That is, the visual focus position may be determined by executing the gaze tracking algorithm utilizing the camera device associated with the terminal.
According to an exemplary embodiment of the present disclosure, the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied. It should be noted that the camera device associated with the terminal may be, but not limited to, a front camera of the terminal and a camera of another terminal connected to the terminal, where the another terminal may be smart glasses (e.g., AR glasses). In the case where the visual focus position information is obtained based on the smart glasses, the visual focus position information may be determined by a gaze tracking algorithm according to eye information of a user identified by the smart glasses based on collected eye images of the user.
According to an exemplary embodiment of the present disclosure, the preset condition may include, but is not limited to, at least one of: a program to which the first display content belongs (a program currently being used by a user) is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs. It should be noted that programs in the preset program set may be programs belonging to a white list, such as but not limited to common split-screen programs, news or social programs, and the like. The fact that the terminal does not move may be determined by using a motion sensor of the terminal to detect that the terminal does not move relatively for a period of time. The states in the preset state set may include, but are not limited to, the terminal plays a video in full screen, the terminal is in a power saving or low power mode, and the terminal is in a split-screen state. When the preset condition includes that the duration that the user watches the terminal screen exceeds the preset threshold before the split-screen display occurs, the acquired visual focus position information can be more accurate and effective.
As an example, considering that the user may choose to perform split-screen display on the terminal screen when using various programs, in order to improve efficiency, when the user chooses to use each program in the terminal, judgment may be made based on the following conditions: the program is a program in the preset program set, the terminal does not move, the state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images. Only when the above conditions are satisfied, a front camera of the terminal is turned on to collect the user's (face) image in real time during the use of the program, and based on the user's (face) image, the user's eye information (which may include eyeballs, wrinkles, iris and other information) may be identified, and then the user's real-time visual focus position information is determined based on the user's eye information through the gaze tracking algorithm (biometric identification through the eyeballs, wrinkles, iris and other information), so that when performing split-screen display on the terminal screen, the visual focus position information determined at the current moment may be obtained. Of course, in order to make the obtained visual focus position information more accurate, when it is determined that the user has been watched the terminal screen for more than a preset time before the current moment, the visual focus position information determined at the current moment is obtained.
As an example, after the visual focus position information is acquired, the visual focus position information may be corrected to obtain more accurate visual focus position information.
According to another exemplary embodiment of the present disclosure, the visual focus position may be determined by receiving a selection input of a visual focus position within the display area of the terminal screen by the user. For example, but not limited to, the visual focus position information may be acquired by displaying prompt information for prompting the user to select a visual focal point in the display area of the terminal screen, receiving the visual focal point selected by the user based on the prompt information, and determining the visual focus position information based on the visual focal point.
According to another exemplary embodiment of the present disclosure, a preset position (e.g., but not limited to, a central position) within the display area of the terminal screen may be determined as the video focus position. For example, but not limited to, the visual focus position information may be acquired by using preset position information in the display area of the terminal screen (e.g., center position information in the display area of the terminal screen) as the visual focus position information.
In step 402, the target display content may be determined based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
According to an exemplary embodiment of the present disclosure, the target display content may be determined by locating a component corresponding to an area where the visual focal point is located. Specifically: firstly, page layout information of the first display content may be obtained; then, component information of the area where the visual focus position is located may be determined based on the page layout information; finally, the target display content may be determined based on the component information of the area where the visual focus position is located.
As an example, when an operating system of the terminal is an Android system, firstly, layout tree structure of a page where the first display content is located may be obtained through an Android uiautomator command, and hierarchy information, component information, and size and position information of each component of the page where the first display content is located are determined based on the layout tree structure, and the above information is used as the page layout information. Then, based on the page layout information, the component information of the area where the visual focus position is located (including but not limited to a component name, display content of the component on the page, a size and position information of a display area of the component on the page) may be determined. Finally, the target display content may be determined based on the component information of the area where the visual focus position is located.
According to the exemplary embodiment of the present disclosure, since some components may be configured to display different content, for example, some components may be configured to display both still images and dynamic images, in order to determine the target display content more accurately, judgment of a dynamic update area (that is, the area where the dynamic image is displayed) may be increased. Specifically, firstly, the page layout information of the first display content may be obtained, and it may be determined whether the page where the first display content is located exists the dynamic update area; then, if there is a dynamic update area and the area where the visual focus position is located is the dynamic update area, it may be determined that the target display content is a dynamic image; if there is no dynamic update area, or there is a dynamic update area but the area where the visual focus position is located is not the dynamic update area, the component information of the area where the visual focus position is located may be determined based on the page layout information, and the target display content may be determined based on the component information of the area where the visual focus position is located.
As an example, when an operating system of the terminal is an Android, firstly, layout tree structure of a page where the first display content is located may be obtained through an Android uiautomator command, and hierarchy information, component information, and size and position information of each component of the page where the first display content is located are determined based on the layout tree structure, the above information is used as the page layout information, and a dynamic update area is obtained through an Android SurfaceFlinger command. If the dynamic update area obtained through the Android SurfaceFlinger command is not empty, it is determined that there is a dynamic update area, and a size and position information of the dynamic update area may be obtained, otherwise it is determined that there is no dynamic update area. Then, if there is a dynamic update area and the area where the visual focus position is located is the dynamic update area, it may be determined that the target display content is a dynamic image; if there is no dynamic update area, or there is a dynamic update area but the area where the visual focus position is located is not the dynamic update area, the component information (including but not limited to the component name, the display content of the component on the page, and the size and position information of the display area of the component on the page) of the area where the visual focus position is located may be determined based on the page layout information. Finally, the target display content may be determined based on the component information of the area where the visual focus position is located. For example, the component name is ImageView, it may be determined that the target display content is a still image.
As an example, in the above step of determining the component information of the area where the visual focus position is located, and the step of determining whether the area where the visual focus position is located is a dynamic update area, the visual focus position used may be visual focus position coordinates (X, Y), or Y value (Y-axis coordinate value) in the visual focus position coordinates (X, Y).
As an example, FIG. 5 is a schematic diagram of target display content according to an exemplary embodiment. Referring to FIG. 5, the visual focus position is the Y value in the visual focus position coordinates (X, Y), and the target display content is a video (dynamic image) in the dotted box.
According to an exemplary embodiment of the present disclosure, the above visual focus tracking may also not be performed, and the currently displayed video in the display area of the terminal screen may be directly determined as the target display content. Here, the currently displayed video may be a fully displayed video or a partially displayed video in the display area of the terminal screen.
Returning to FIG. 3, in step 303, the target display content may be completely displayed in a target split-screen area after the terminal screen is split. Here, the terminal screen may be split into multiple split-screen areas, and the target split-screen area is one of the multiple split-screen areas. Each split-screen area may display a part of the first display content.
According to an exemplary embodiment of the present disclosure, in response to that the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split, the display content in the target split-screen area is adjusted so that the target display content may be completely displayed in the target split-screen area.
According to an exemplary embodiment of the present disclosure, before step 303 is performed, split-screen display may be performed on the terminal screen, and it is determined whether the target display content may be completely displayed in the target split-screen area.
According to an exemplary embodiment of the present disclosure, in order to determine whether the target display content can be completely displayed in the target split-screen area, area information of the target display content and area information of the target split-screen area may be obtained firstly; whether the target display content can be completely displayed in the target split-screen area may be determined based on a comparison between the area information of the target display content and the area information of the target split-screen area.
Here, different situations may be distinguished in combination with the target display content determined in the above steps. The target display content can be, but not limited to, images or non-images. For example, the images may include, but are not limited to, still images and dynamic images, wherein the dynamic images may be, but are not limited to, videos or gifs. For example, non-images may include, but are not limited to, text and the like.
In the cast where the target display content is an image, the area information of the target split-screen area may be position information of the target split-screen area, and the area information of the target display content may be position information of the target display content. As an example, since a navigation bar of a program after split-screen display may not be affected by the split-screen, it still occupies the corresponding position of the target split-screen area, so the position information of the target split-screen area may be position information of remaining areas other than content fixedly occupying the target split-screen area (such as the navigation bar) in the target split-screen area.
As an example, in the case where the target display content is an image, since the target split-screen area is a surface, which is composed of lines, and each line is formed by connecting points, the position information of the target split-screen area may be, but not limited to, position information of each point in the target split-screen area. Similarly, the position information of the target display content may be, but not limited to, position information of each point in the target display content. Here, the position information may include, but is not limited to, position coordinates. Then, in the case where there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area (that is, the position of the point in the target display content is not within the range of the target split-screen area), it is determined that the target display content cannot be completely displayed in the target split-screen area. In the case where the position information of each point in the target display content can be found in the position information of the target split-screen area (that is, the position of each point in the target display content is within the range of the target split-screen area), it is determined that the target display content can be completely displayed in the target split-screen area.
As an example, the target split-screen area may be a rectangular area, and if the target display content is an image, the target display content may also be a rectangular area. Based on this, the position information of the target split-screen area may be, but not limited to, position information of four vertices of the target split-screen area, and the position information of the target display content may be, but not limited to, position information of four vertices of the target display content. Here, the position information may include, but is not limited to, position coordinates. Then, by comparing the position information of the four vertices of the target split-screen area and the position information of the four vertices of the target display content, it may be determined whether the target display content can be completely displayed in the target split-screen area.
In the case where the target display content is an image, the area information of the target split-screen area may also be a size and position information of the target split-screen area, and the area information of the target display content may also be a size and position information of the target display content. Based on a comparison between the size and the position information of the target display content and the size and the position information of the target split-screen area, it is determined whether the target display content can be completely displayed in the target split-screen area.
In the case where the target display content is not an image, the area information of the target split-screen area is a size and position information of the target split-screen area, and the area information of the target display content is visual focus position information of a user in the display area of the terminal screen (i.e., the visual focus position information obtained in the above steps), wherein the display content of the area where the visual focus position is located is the target display content. In this case, firstly, a complete display area within the target split-screen area may be determined based on the size and the position information of the target split-screen area; then, in the case where the visual focus position is outside the complete display area, it is determined that the target display content cannot be completely displayed in the target split-screen area. In the case where the visual focus position is located in the complete display area, it is determined that the target display content can be completely displayed in the target split-screen area.
As an example, in the case where the target display content is not an image, firstly, based on the size and the position information of the target split-screen area, it may be determined that an area within a height range [0, 4/5] of the target split-screen area is the complete display area; Then, based on the Y value in the visual focus position coordinates (X, Y), it may be determined whether the visual focus position is within the height range [0, 4/5] of the target split-screen area, and if the visual focus position is not located in the height range [0, 4/5] of the target split-screen area, it is determined that the target display content cannot be completely displayed in the target split-screen area, and if the visual focus position is within the height range [0, 4/5] of the target split-screen area, it is determined that the target display content can be completely displayed in the target split-screen area.
According to an exemplary embodiment of the present disclosure, when the target display content can be completely displayed in the target split-screen area after the terminal screen is split, the display content in the target split-screen area may not be adjusted.
In the following, for the case where the target display content is an image or a non-image, (in the case where the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split), how to adjust the display content in the target split-screen area is described in detail.
In the case where the target display content is an image, a split-screen adjustment strategy may be determined based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and the display content in the target split-screen area is adjusted based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
Specifically, in the case where the target display content is an image, if the size of the target display content is larger than the size of the target split-screen area, or if the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display, the split-screen adjustment strategy may be determined as adjusting a current split-screen ratio and/or adjusting a current split-screen direction; if the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display, the split-screen adjustment strategy is determined as adjusting a display position of the target display content in the target split-screen area. For example, a target distance and a target direction may be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page may be scrolled based on the target distance and target direction, thereby adjusting the display position of the target display content in the target split-screen area.
As an example, the current split-screen ratio is 0.5:0.5, the current split-screen direction is up and down split-screens, and when the target display content is an image, there is a situation that the target display content cannot be completely displayed in the target split-screen area, and the size of the target display content is larger than the size of the target split-screen area, the current split-screen ratio (for example, the split-screen ratio is adjusted to 0.7:0.3) may be adjusted and/or the split-screen direction may adjusted to left and right split-screens.
As an example, if the target display content is an image, the size of the target display content is larger than the size of the target split-screen area, and the target page supports scrolling display, it may be determined whether the target page needs to be scrolled, and if the target page needs to be scrolled, the target page is scrolled and the split-screen adjustment strategy of adjusting the current split-screen ratio and/or adjusting the current split-screen direction is performed.
As an example, the target split-screen area may be a rectangular area, and if the target display content is an image, the target display content may also be a rectangular area. In the case where the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display, a Y-axis coordinate value of a top edge of the target display content may be obtained based on the position information of the target display content firstly (the Y-axis coordinate value of the position coordinates of each point on the top edge is the same), and a Y-axis coordinate value of a top edge of the target split-screen area may be obtained based on the position information of the target split-screen area (the Y-axis coordinate value of the position coordinates of each point on the top edge is the same); then a difference between the two Y-axis coordinate values is calculated, the target direction is determined according to a sign of the difference and the position information of a coordinate origin, the target distance is determined according to a absolute value of the difference, and the target page is scrolled by injecting an input event based on the target distance and the target direction. In this embodiment, scrolling the target page based on the target distance and the target direction may scroll the target display content to a center position of the target split-screen area. FIG. 6 is a schematic diagram showing a comparison before and after adjustment according to an exemplary embodiment, FIG. 6(a) is a schematic diagram before split-screen display is performed, the target display content is a video in the figure, and the size of the target display content is not larger than the size of the split-screen area and the target page may be scrolled. FIG. 6(b) is a schematic diagram before a split-screen adjustment strategy is performed after split-screen display is performed, and FIG. 6(c) is a schematic diagram after a split-screen adjustment strategy is performed.
In the case where the target display content is an image, the target display content may also be displayed on the target page in the form of a floating window, where the target page is a page where the display content in the target split-screen area is located.
In the case where the target display content is not an image, a split-screen adjustment strategy may be determined based on whether the target page supports scrolling display, and based on the determined split-screen adjustment strategy, the display content in the target split-screen area may be adjusted, wherein the target page is a page where the display content in the target split-screen area is located.
Specifically, in the case where the target display content is not an image, and if the target page does not support scrolling display, the split-screen adjustment strategy may be determined as adjusting the current split-screen ratio and/or adjusting the current split-screen direction; if the target page supports scrolling display, the split-screen adjustment strategy is determined as adjusting the display position of the target display content in the target split-screen area. For example, the target distance and the target direction may be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page may be scrolled based on the target distance and the target direction, thereby adjusting the display position of the target display content in the target split-screen area.
As an example, the current split-screen ratio is 0.5:0.5, the current split-screen direction is up and down split-screens, and in the case where the target display content is not an image, there is a situation where the target display content cannot be completely displayed in the target split-screen area, and the target page does not support scrolling display, the current split-screen ratio may be adjusted (for example, the split-screen ratio is adjusted to 0.7:0.3) and/or the split-screen direction is adjusted to left and right split-screens.
As an example, if the target display content is not an image, and if the target page supports scrolling display, the target page may be scrolled so that the target display content is scrolled to the center of the target split-screen area. In this embodiment, the target distance and the target direction may be determined in the following manner: in the case that the target display content is determined based on the visual focus position information, firstly, the Y value in the visual focus position coordinates (X, Y) and the Y-axis coordinate value corresponding to the center position of the target split-screen area may be obtained respectively, then the difference between the two Y-axis coordinate values is calculated, and the target direction is determined according to the sign of the difference and the position information of the coordinate origin, the target distance is determined based on the absolute value of the difference, and then the target page may be scrolled by injecting the input event based on the target distance and target direction. In this embodiment, the target distance and the target direction may also be determined in the following manner: firstly, the Y-axis coordinate value corresponding to the center position of the target display content and the Y-axis coordinate value corresponding to the center position of the target split-screen area may be obtained respectively, and then the difference between the two Y-axis coordinate values is calculated, the target direction is determined according to the sign of the difference and the position information of the coordinate origin, and the target distance is determined according to the absolute value of the difference. FIG. 7 is another schematic diagram before and after adjustment according to an exemplary embodiment, FIG. 7(a) is a schematic diagram before split-screen display is performed, the target display content is part of text content in the figure, and the target page supports scrolling display, FIG. 7(b) is a schematic diagram before a split-screen adjustment strategy is performed after split-screen display is performed, and FIG. 7(c) is a schematic diagram after a split-screen adjustment strategy is performed.
FIG. 8 is a flow chart of another screen display method according to an exemplary embodiment.
Referring to FIG. 8, firstly, in response to performing split-screen display on a terminal screen, visual focus position information of a user in a display area of a terminal screen is obtained, and target display content is determined, wherein the target display content is display content of an area where the visual focus position is located.
Next, split-screen display may be performed on the terminal screen.
Then, if the target display content is a dynamic image, it may be determined whether the target display content can be completely displayed in a target split-screen area (after the terminal screen is split), and if it can be completely displayed, the display content in the target split-screen area is not adjusted, and if it cannot be completely displayed, the following operations are performed: when a size of the target display content is larger than a size of the target split-screen area, or when the size of the target display content is not larger than the size of the target split-screen area and the target page ( a page where the display content in the target split-screen area is located) does not support scrolling display, a split-screen adjustment strategy is determined as adjusting a current split-screen ratio and/or adjusting a current split-screen orientation; when the size of the target split-screen area is not larger than the size of the target split-screen area and the target page supports scrolling display, the split-screen adjustment strategy is determined as adjusting a display position of the target display content in the target split-screen area. For example, a target distance and a target direction may be determined based on position information of the target display content and position information of the target split-screen area, and the target page may be scrolled based on the target distance and the target direction, thereby adjusting the display position of the target display content in the target split-screen area.
In the case where the target display content is not a dynamic image, it may be determined whether the target display content is a still image.
Next, in the case where the target display content is a still image, the same steps as those of the target display content being a dynamic image are performed, which will not be repeated here.
In the case where the target display content is not a still image (it has been determined that the target display content is not a dynamic image at this time), it may be determined whether the visual focus position is within a height range [0, 4/5] of the target split-screen area, if the visual focus position is within the height range [0, 4/5] of the target split-screen area, the display content in the target split-screen area may not be adjusted. If the visual focus position is outside the height range [0, 4/5] of the target split-screen area, the following operations are performed: if the target page does not support scrolling display, the split-screen adjustment strategy is determined as adjusting the current split-screen ratio and/or adjusting the current split-screen direction; if the target page supports scroll display, the split-screen adjustment strategy is determined as adjusting the display position of the target display content in the target split-screen area. For example, the target distance and target direction may be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page may be scrolled based on the target distance and target direction, thereby adjusting the display position of the target display content in the target split-screen area.
FIG. 9 is a block diagram of a screen display device according to an exemplary embodiment. Referring to FIG. 9, the screen display device 900 includes a full-screen display unit 901, a target determination unit 902 and a split-screen display unit 903.
The full-screen display unit 901 may display first display content in a display area of a terminal screen. Here, the first display content may be content currently displayed in the display area of the terminal screen.
According to an exemplary embodiment of the present disclosure, the terminal may be, but is not limited to, a smart phone. The terminal screen may be, but is not limited to, a straight screen, a curved screen, and a foldable screen, wherein, when the terminal screen is the foldable screen, performing split-screen display on the terminal screen may be, but not limited to, folding the terminal screen.
The target determination unit 902 may determine target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content. For example, when a user folds the terminal screen, the terminal screen may be triggered to be split. Of course, the present disclosure is not limited to this, and other conditions may also be used to trigger the terminal screen to be split.
According to an exemplary embodiment of the present disclosure, the target determination unit 902 may perform the step of determining the target display content in the first display content before performing split-screen display on the terminal screen.
According to an exemplary embodiment of the present disclosure, the target display content may be content that the user is viewing within the display area of the terminal screen.
As an example, the content that the user is viewing may be determined by a user's visual focal point.
Firstly, the target determination unit 902 may acquire visual focus position information of the user in the display area of the terminal screen. Here, the visual focus position information may include, but is not limited to, visual focus position coordinates.
According to an exemplary embodiment of the present disclosure, the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal. That is, the visual focus position may be determined by executing the gaze tracking algorithm utilizing the camera device associated with the terminal.
According to an exemplary embodiment of the present disclosure, the visual focus position information may be acquired through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied. It should be noted that the camera device associated with the terminal may be, but not limited to, a front camera of the terminal and a camera of another terminal connected to the terminal, where the another terminal may be smart glasses (e.g., AR glasses). In the case where the visual focus position information is obtained based on the smart glasses, the visual focus position information may be determined by a gaze tracking algorithm according to eye information of a user identified by the smart glasses based on collected eye images of the user.
According to an exemplary embodiment of the present disclosure, the preset condition may include, but is not limited to, at least one of: a program to which the first display content belongs (a program currently being used by a user) is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs. It should be noted that programs in the preset program set may be programs belonging to a white list, such as but not limited to common split-screen programs, news or social programs, and the like. The fact that the terminal does not move may be determined by using a motion sensor of the terminal to detect that the terminal does not move relatively for a period of time. The states in the preset state set may include, but are not limited to, the terminal plays a video in full screen, the terminal is in a power saving or low power mode, and the terminal is in a split-screen state. When the preset condition includes that the duration that the user watches the terminal screen exceeds the preset threshold before the split-screen display occurs, the acquired visual focus position information can be more accurate and effective.
According to another exemplary embodiment of the present disclosure, the target determination unit 902 may determine the visual focus position by receiving a selection input of a visual focus position within the display area of the terminal screen by the user. For example, but not limited to, the visual focus position information can be acquired by displaying prompt information for prompting the user to select a visual focal point in the display area of the terminal screen, receiving the visual focal point selected by the user based on the prompt information, and determining the visual focus position information based on the visual focal point.
According to another exemplary embodiment of the present disclosure, the target determination unit 902 may determine a preset position (e.g., but not limited to, a central position) within the display area of the terminal screen as the video focus position. For example, but not limited to, the visual focus position information may be acquired by using preset position information in the display area of the terminal screen (e.g., center position information in the display area of the terminal screen) as the visual focus position information.
Next, the determining unit 901 may determine the target display content based on the first display content and the visual focus position information, where the target display content is display content in an area where the visual focus position is located.
According to an exemplary embodiment of the present disclosure, the determination unit 901 may determine the target display content by locating a component corresponding to an area where the visual focus is located. Specifically: firstly, the determination unit 901 may obtain page layout information of the first display content; then, the determination unit 901 may determine component information of the area where the visual focus position is located based on the page layout information; finally, the determination unit 901 may determine the target display content based on the component information of the area where the visual focus position is located.
As an example, when an operating system of the terminal is an Android system, firstly, the determining unit 901 may obtain layout tree structure of a page where the first display content is located through an Android uiautomator command, and determine hierarchy information, component information, and size and position information of each component of the page where the first display content is located based on the layout tree structure, and use the above information as the page layout information. Then, the determining unit 901 may determine the component information of the area where the visual focus position is located (including but not limited to a component name, display content of the component on the page, a size and position information of a display area of the component on the page) based on the page layout information. Finally, the determining unit 901 may determine the target display content based on the component information of the area where the visual focus position is located.
As an example, the visual focus position may be visual focus position coordinates (X, Y), or Y value (Y-axis coordinate value) in the visual focus position coordinates (X, Y).
According to an exemplary embodiment of the present disclosure, the target determination unit 902 may also directly determine a video currently displayed in the display area of the terminal screen as the target display content without performing the above visual focus tracking. Here, the currently displayed video may be a fully displayed video or a partially displayed video in the display area of the terminal screen.
Returning to FIG. 9, the split-screen display unit 903 may completely display the target display content in the target split-screen area after the terminal screen is split. Here, the terminal screen can be split into multiple split-screen areas, and the target split-screen area is one of the multiple split-screen areas. Each split-screen area may display a part of the first display content.
According to an exemplary embodiment of the present disclosure, in response to that the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split, the split-screen display unit 903 adjusts the display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
According to an exemplary embodiment of the present disclosure, the split-screen display unit 903, before performing an operation, may perform split-screen display on the terminal screen, and determine whether the target display content can be completely displayed in the target split-screen area. Based on this, the screen display device 900 also includes a complete display determination unit (not shown), the complete display determination unit may obtain area information of the target display content and area information of the target split-screen area firstly; the complete display determination unit may then determine whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
Here, different situations can be distinguished in combination with the target display content determined in the above steps. The target display content can be, but not limited to, images or non-images. For example, the images may include, but are not limited to, still images and dynamic images, wherein the dynamic images may be, but are not limited to, videos or gifs. For example, non-images may include, but are not limited to, text and the like.
In the case where the target display content is an image, the area information of the target split-screen area may be position information of the target split-screen area, and the area information of the target display content may be position information of the target display content.
As an example, in the case where the target display content is an image, since the target split-screen area is a surface, which is composed of lines, and each line is formed by connecting points, the position information of the target split-screen area may be, but not limited to, position information of each point in the target split-screen area. Similarly, the position information of the target display content may be, but not limited to, position information of each point in the target display content. Here, the position information may include, but is not limited to, position coordinates. Then, in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area (that is, the position of the point in the target display content is not within the range of the target split-screen area), it is determined that the target display content cannot be completely displayed in the target split-screen area. In the case where the position information of each point in the target display content can be found in the position information of the target split-screen area (that is, the position of each point in the target display content is within the range of the target split-screen area), it is determined that the target display content can be completely displayed in the target split-screen area.
As an example, the target split-screen area may be a rectangular area, and if the target display content is an image, the target display content may also be a rectangular area. Based on this, the position information of the target split-screen area may be, but not limited to, position information of four vertices of the target split-screen area, and the position information of the target display content may be, but not limited to, position information of four vertices of the target display content. Here, the position information may include, but is not limited to, position coordinates. Then, by comparing the position information of the four vertices of the target split-screen area and the position information of the four vertices of the target display content, it can be determined whether the target display content can be completely displayed in the target split-screen area.
In the case where the target display content is an image, the area information of the target split-screen area can also be a size and position information of the target split-screen area, and the area information of the target display content can also be a size and position information of the target display content. Based on a comparison between the size and the position information of the target display content and the size and the position information of the target split-screen area, it is determined whether the target display content can be completely displayed in the target split-screen area.
In the case where the target display content is not an image, the area information of the target split-screen area is a size and position information of the target split-screen area, and the area information of the target display content is visual focus position information of a user in the display area of the terminal screen (i.e., the visual focus position information obtained in the above steps), wherein the display content of the area where the visual focus position is located is the target display content. In this case, first, the complete display determination unit may determine a complete display area within the target split-screen area based on the size and the position information of the target split-screen area; then, the complete display determination unit may determine, in the case where the visual focus position is outside the complete display area, that the target display content cannot be completely displayed in the target split-screen area, and determine, in the case where the visual focus position is located in the complete display area, that the target display content can be completely displayed in the target split-screen area.
According to an exemplary embodiment of the present disclosure, in the case where the target display content can be completely displayed in the target split-screen area after the terminal screen is split, the split-screen display unit 903 may not adjust the display content in the target split-screen area.
In the case where the target display content is an image, the split-screen display unit 903 may determine a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjust the display content in the target split-screen area based on the determined split-screen adjustment strategy, where the target page is a page where the display content in the target split-screen area is located.
Specifically, in the case where the target display content is an image, if the size of the target display content is larger than the size of the target split-screen area, or if the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display, the split-screen display unit 903 may determine the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction; if the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display, determine the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area. For example, a target distance and a target direction can be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page can be scrolled based on the target distance and target direction, thereby adjusting the display position of the target display content in the target split-screen area.
In the case where the target display content is an image, the split-screen display unit 903 may also display the target display content on the target page in the form of a floating window, where the target page is a page where the display content in the target split-screen area is located.
In the case where the target display content is not an image, the split-screen display unit 903 may determine a split-screen adjustment strategy based on whether the target page supports scrolling display, and adjust the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
Specifically, in the case where the target display content is not an image, if the target page does not support scrolling display, the split-screen display unit 903 may determine the split-screen adjustment strategy as adjusting the current split-screen ratio and/or adjusting the current split-screen direction; if the target page supports scrolling display, the split-screen display unit 903 may determine the split-screen adjustment strategy as adjusting the display position of the target display content in the target split-screen area. For example, the target distance and the target direction can be determined based on the position information of the target display content and the position information of the target split-screen area, and the target page can be scrolled based on the target distance and the target direction, thereby adjusting the display position of the target display content in the target split-screen area.
FIG. 10 is a block diagram of an electronic apparatus 1000 according to an exemplary embodiment.
Referring to FIG. 10, the electronic apparatus 1000 includes at least one memory 1001 and at least one processor 1002, the at least one memory 1001 stores a computer-executable instruction set, when the computer-executable instruction set is executed by the at least one processor 1002, the screen display method according to the present disclosure is executed.
As an example, the electronic apparatus 1000 may be a PC computer, a tablet device, a personal digital assistant, a smart phone, or any other device capable of executing the above instruction set. Here, the electronic apparatus 1000 does not have to be a single electronic device, but may also be any set of devices or circuits capable of executing the above instructions (or instruction set) individually or jointly. The electronic apparatus 1000 may also be a part of an integrated control system or system manager, or may be configured as a portable electronic device that interfaces locally or remotely (e.g., via wireless transmission).
In electronic apparatus 1000, the processor 1002 may include a central processing unit (CPU), graphics processing unit (GPU), programmable logic device, special purpose processor system, microcontroller or microprocessor. By way of example and not limitation, the processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, and the like.
The processor 1002 may execute instructions or code stored in the memory 1001, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transport protocol.
The memory 1001 may be integrated with the processor 1002, e.g., a RAM or flash memory is arranged within an integrated circuit microprocessor or the like. Additionally, the memory 1001 may include a separate device such as an external disk drive, storage array, or any other storage device that may be used by a database system. The memory 1001 and the processor 1002 may be operatively coupled, or may communicate with each other, e.g., through I/O ports, network connections, etc., to enable the processor 1002 to read files stored in the memory.
In addition, the electronic apparatus 1000 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of electronic apparatus 1000 may be connected to each other via a bus and/or network.
According to an exemplary embodiment of the present disclosure, there may also be provided a computer-readable storage medium storing instructions, wherein the instructions, when executed by at least one processor, cause the at least one processor to perform the screen display method according to the present disclosure. Examples of computer-readable storage media herein include: Read Only Memory (ROM), Random Access Programmable Read Only Memory (RAPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blue-ray or optical disk storage, Hard Disk Drive (HDD), Solid State Drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards or extremely fast digital (XD) cards), magnetic tapes, floppy disks, magneto-optical data storage devices, optical data storage devices, hard disks, solid state disks, and any other devices that are configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer programs. The instructions or computer programs in the computer-readable storage medium described above may be executed in an environment deployed in a computer device. In addition, in one example, the computer programs and any associated data, data files, and data structures are distributed on a networked computer system, so that the computer programs and any associated data, data files, and data structures are stored, accessed and executed through one or more processors or computers in a distributed manner.
According to the screen display method, device, electronic apparatus and storage medium of the present disclosure, after split-screen display, if the target display content cannot be completely displayed in the target split-screen area after the terminal screen is split, the target split-screen area is adjusted so that the target display content can be completely displayed in the target split-screen area. In this way, when the target display content is the content that the user is viewing before screen splitting, the content viewed by the user can be completely displayed after screen splitting, and there is no impact on the content viewed by the user before and after screen splitting.
In addition, according to the screen display method, device, electronic apparatus and storage medium of the present disclosure, the target display content is the display content of the area where the visual focus position is located, which can maintain the consistency and integrity of the user's visual focus content before and after screen splitting.
Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principle of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure. The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.
It should be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (15)

  1. A screen display method, comprising:
    displaying first display content in a display area of a terminal screen;
    determining target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content;
    displaying the target display content completely in a target split-screen area after the terminal screen is split.
  2. The screen display method according to claim 1, wherein the displaying of the target display content completely in the target split-screen area after the terminal screen is split comprises:
    in response to that the target display content cannot be completely displayed in the target split-screen area, adjusting display content in the target split-screen area so that the target display content can be completely displayed in the target split-screen area.
  3. The screen display method according to claim 1, wherein the determining of the target display content in the first display content comprises:
    obtaining visual focus position information of a user in the display area of the terminal screen;
    determining the target display content based on the first display content and the visual focus position information, wherein the target display content is display content in an area where the visual focus position is located.
  4. The screen display method according to claim 3, wherein the visual focus position information is obtained through a gaze tracking algorithm by using a camera device associated with the terminal in the case that a preset condition is satisfied.
  5. The screen display method according to claim 4, wherein the preset condition comprises at least one of:
    a program to which the first display content belongs is a program in a preset program set, the terminal does not move, a state of the terminal does not belong to a state in a preset state set, the user is a user authorized to collect images, a duration that the user watches the terminal screen exceeds a preset threshold before the split-screen display occurs.
  6. The screen display method of claim 3, wherein the visual focus position is determined by one of:
    determining the visual focus position by executing a gaze tracking algorithm utilizing a camera device associated with the terminal;
    receiving a selection input of the visual focus position in the display area of the terminal screen by the user;
    determining a center position in the display area of the terminal screen as the video focus position.
  7. The screen display method according to claim 3, wherein the determining of the target display content based on the first display content and the visual focus position information comprises:
    acquiring page layout information of the first display content;
    determining component information of the area where the visual focus position is located, based on the page layout information;
    determining the target display content based on the component information of the area where the visual focus position is located.
  8. The screen display method according to claim 1, wherein the determining of the target display content in the first display content comprises:
    determining a video that is currently displayed in the first display content as the target display content.
  9. The screen display method of claim 2, further comprising:
    obtaining area information of the target display content and area information of the target split-screen area;
    determining whether the target display content can be completely displayed in the target split-screen area, based on a comparison between the area information of the target display content and the area information of the target split-screen area.
  10. The screen display method according to claim 9, wherein in the case that the target display content is an image, the area information of the target split-screen area is position information of the target split-screen area, and the area information of the target display content is position information of the target display content;
    wherein the determining of whether the target display content can be completely displayed in the target split-screen area based on the comparison between the area information of the target display content and the area information of the target split-screen area comprises:
    determining that the target display content cannot be completely displayed in the target split-screen area in the case that there is a point, in the target display content, of which position information is different from that of each point in the target split-screen area.
  11. The screen display method according to claim 9, wherein in the case that the target display content is not an image, the area information of the target split-screen area is a size and position information of the target split-screen area, the area information of the target display content is visual focus position information of a user in the display area of the terminal screen, wherein display content of an area where the visual focus position is located is the target display content;
    wherein the determining of whether the target display content can be completely displayed in the target split-screen area based on the comparison between the area information of the target display content and the area information of the target split-screen area comprises:
    determining a complete display area in the target split-screen area based on the size and the position information of the target split-screen area;
    determining that the target display content cannot be completely displayed in the target split-screen area in the case that the visual focus position is outside the complete display area.
  12. The screen display method according to claim 1, wherein in the case that the target display content is an image, the adjusting of the display content in the target split-screen area comprises:
    determining a split-screen adjustment strategy based on a comparison between a size of the target display content and a size of the target split-screen area and/or whether a target page supports scrolling display, and adjusting the display content in the target split-screen area based on the determined split-screen adjustment strategy, wherein the target page is a page where the display content in the target split-screen area is located.
  13. The screen display method according to claim 12, wherein the determining of the split-screen adjustment strategy based on the comparison between the size of the target display content and the size of the target split-screen area and/or whether the target page supports scrolling display comprises:
    determining the split-screen adjustment strategy as adjusting a current split-screen ratio and/or adjusting a current split-screen direction in the case that the size of the target display content is larger than the size of the target split-screen area, or the size of the target display content is not larger than the size of the target split-screen area and the target page does not support scrolling display;
    determining the split-screen adjustment strategy as adjusting a display position of the target display content in the target split-screen area in the case that the size of the target display content is not larger than the size of the target split-screen area and the target page supports scrolling display.
  14. The screen display method according to claim 2, wherein in the case that the target display content is an image, the adjusting of the display content in the target split-screen area comprises:
    displaying the target display content on a target page in a form of a floating window, wherein the target page is a page where the display content in the target split-screen area is located.
  15. A screen display device, comprising:
    a full-screen display unit configured to display first display content in a display area of a terminal screen;
    a target determination unit configured to determine target display content in the first display content in response to performing split-screen display on the terminal screen, wherein the target display content is at least part of the first display content;
    a split-screen display unit configured to display the target display content completely in a target split-screen area after the terminal screen is split.
PCT/KR2023/013946 2022-10-11 2023-09-15 Screen display method, device, electronic apparatus and storage medium WO2024080593A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211242474.8A CN117908812A (en) 2022-10-11 2022-10-11 Screen display method and device, electronic equipment and storage medium
CN202211242474.8 2022-10-11

Publications (1)

Publication Number Publication Date
WO2024080593A1 true WO2024080593A1 (en) 2024-04-18

Family

ID=90669540

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2023/013946 WO2024080593A1 (en) 2022-10-11 2023-09-15 Screen display method, device, electronic apparatus and storage medium

Country Status (2)

Country Link
CN (1) CN117908812A (en)
WO (1) WO2024080593A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009289051A (en) * 2008-05-29 2009-12-10 Sony Corp Web page display device and web page display method
US20140189566A1 (en) * 2012-12-31 2014-07-03 Lg Electronics Inc. Method and an apparatus for processing at least two screens
US20160363994A1 (en) * 2015-06-12 2016-12-15 Panasonic Intellectual Property Corporation Of America Display control method for highlighting display element focused by user
US20220035513A1 (en) * 2020-07-28 2022-02-03 Lg Electronics Inc. Mobile terminal and control method therefor
WO2022092652A1 (en) * 2020-10-26 2022-05-05 삼성전자 주식회사 Electronic device providing user interface

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009289051A (en) * 2008-05-29 2009-12-10 Sony Corp Web page display device and web page display method
US20140189566A1 (en) * 2012-12-31 2014-07-03 Lg Electronics Inc. Method and an apparatus for processing at least two screens
US20160363994A1 (en) * 2015-06-12 2016-12-15 Panasonic Intellectual Property Corporation Of America Display control method for highlighting display element focused by user
US20220035513A1 (en) * 2020-07-28 2022-02-03 Lg Electronics Inc. Mobile terminal and control method therefor
WO2022092652A1 (en) * 2020-10-26 2022-05-05 삼성전자 주식회사 Electronic device providing user interface

Also Published As

Publication number Publication date
CN117908812A (en) 2024-04-19

Similar Documents

Publication Publication Date Title
WO2011078540A2 (en) Mobile device and related control method for external output depending on user interaction based on image sensing module
WO2014204047A1 (en) Foldable display device and method of controlling therefor
WO2015030321A1 (en) Portable device and method of controlling therefor
WO2014104472A1 (en) Method and apparatus for dual display
WO2017119664A1 (en) Display apparatus and control methods thereof
WO2014107006A1 (en) Display apparatus and control method thereof
WO2020130648A1 (en) Electronic device for adaptively altering information display area and operation method thereof
WO2018008991A1 (en) Display device and method for image processing
WO2021015505A1 (en) Foldable electronic device and photographing method using multiple cameras in foldable electronic device
WO2018155824A1 (en) Display apparatus and control method thereof
WO2017142143A1 (en) Method and apparatus for providing summary information of a video
WO2015182811A1 (en) Apparatus and method for providing user interface
WO2015084034A1 (en) Method and apparatus for displaying images
WO2019139367A1 (en) Display device and method for touch interface
WO2020213834A1 (en) Electronic device for displaying execution screens of plurality of applications and operation method thereof
WO2017160028A1 (en) Object management and visualization using a computing device
WO2020013651A1 (en) Electronic device and method for transmitting content of electronic device
WO2021080290A1 (en) Electronic apparatus and control method thereof
WO2021137507A1 (en) Display apparatus and controlling method thereof
EP3753237A1 (en) Electronic device and control method thereof
WO2024080593A1 (en) Screen display method, device, electronic apparatus and storage medium
WO2020241973A1 (en) Display apparatus and control method thereof
WO2019132563A1 (en) Image panning method
WO2020027417A1 (en) Electronic device and method for providing virtual input tool
WO2022124822A1 (en) Image morphing adaption method and apparatus