CN111241385A - Information processing method, information processing apparatus, computer system, and medium - Google Patents

Information processing method, information processing apparatus, computer system, and medium Download PDF

Info

Publication number
CN111241385A
CN111241385A CN201811443393.8A CN201811443393A CN111241385A CN 111241385 A CN111241385 A CN 111241385A CN 201811443393 A CN201811443393 A CN 201811443393A CN 111241385 A CN111241385 A CN 111241385A
Authority
CN
China
Prior art keywords
user
attention
coordinate
coordinate range
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201811443393.8A
Other languages
Chinese (zh)
Inventor
孟宪胜
陈宏勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Original Assignee
Beijing Jingdong Century Trading Co Ltd
Beijing Jingdong Shangke Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Jingdong Century Trading Co Ltd, Beijing Jingdong Shangke Information Technology Co Ltd filed Critical Beijing Jingdong Century Trading Co Ltd
Priority to CN201811443393.8A priority Critical patent/CN111241385A/en
Publication of CN111241385A publication Critical patent/CN111241385A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00—Commerce
    • G06Q30/06—Buying, selling or leasing transactions
    • G06Q30/0601—Electronic shopping [e-shopping]
    • G06Q30/0631—Item recommendations

Landscapes

  • Business, Economics & Management (AREA)
  • Accounting & Taxation (AREA)
  • Finance (AREA)
  • Development Economics (AREA)
  • Economics (AREA)
  • Marketing (AREA)
  • Strategic Management (AREA)
  • Physics & Mathematics (AREA)
  • General Business, Economics & Management (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The present disclosure provides an information processing method, an apparatus, a computer system, and a medium, the information processing method including: acquiring the coordinate and time information of a visual focus of a user on a display screen; determining the coordinate range of a visual attention area of a user and corresponding attention duration based on the coordinate and time information of the visual focus on the display screen; and determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration which are pre-labeled in the current display interface.

Description

Information processing method, information processing apparatus, computer system, and medium
Technical Field
The present disclosure relates to the field of internet technologies, and in particular, to an information processing method, apparatus, computer system, and medium.
Background
The information recommendation technology can effectively help users to find needed information from massive information, and is more and more concerned by society, such as commodity recommendation, news recommendation, picture recommendation, social information recommendation and the like. Taking commodity recommendation as an example for explanation, at present, a e-commerce platform recommends commodities for a user, and generally performs big data analysis on a background server based on shopping history of the user on a website and browsed commodities, analyzes commodities which are favorite or possibly needed by the user, and recommends the commodities to the user.
In the course of implementing the disclosed concept, the inventors found that there are at least the following problems in the prior art: when a user browses an information list, the user pays attention to the duration and the like of certain or some types of specific information, and a network platform cannot easily acquire related information in a statistical manner; in addition, when a user browses an information page, the user may leave a computer or mobile equipment in the period, and the network platform cannot sense the behavior of the user and the information such as the duration of the behavior; in addition, the user can pay attention to one or more detailed information introductions on the information page, and the network platform cannot capture the user behavior and the corresponding information.
Disclosure of Invention
In view of the above, the present disclosure provides an information processing method, apparatus, computer system, and medium that determine attention content and attention degree of a user based on user visual information.
One aspect of the present disclosure provides an information processing method, which may include the operations of: firstly, obtaining the coordinate and time information of a visual focus of a user on a display screen, then determining the coordinate range of a visual attention area of the user and the corresponding attention duration based on the coordinate and time information of the visual focus on the display screen, and then determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration which are pre-marked in a current display interface. Therefore, the display content and the attention degree concerned by the user can be determined according to the visual focus of the user, and the preference and the required information of the user can be determined conveniently according to the concerned content of the user.
According to the embodiment of the disclosure, in one aspect, the method may be applied to a client, and accordingly, the client has a camera, and the obtaining of the coordinate and time information of the visual focus of the user on the display screen may include: firstly, the camera is used for collecting the eyeball movement and focusing information of a user, and then the coordinates of the visual focus of the user on the display screen and the corresponding time are obtained according to the eyeball movement and focusing information of the user. On the other hand, the method can be applied to a server, and the server acquires the coordinate and time information of the visual focus of the user on the display screen by receiving first information sent by the client, wherein the first information comprises the coordinate and time information of the visual focus of the user on the display screen. That is, the operations of the method provided by the present disclosure may be performed on one terminal, or may be performed on two or more terminals, or may be performed on a server, for example, a first terminal collects a video of a user through a camera, performs video processing on a second terminal or the server to determine information such as a visual focus of the user, and other subsequent operations may also be performed in one or more of the first terminal, the second terminal, and the server.
According to an embodiment of the present disclosure, the determining the coordinate range of the visual attention area of the user and the corresponding attention duration based on the coordinate of the visual focus on the display screen and the time information may include the following operations: firstly, the coordinate and time information of the visual focus of the user on the display screen are formatted to obtain a plurality of binary groups, wherein the binary groups comprise the coordinate and the corresponding time, then, the two tuples are sorted based on time, and then, coordinate ranges corresponding to two different adjacent coordinates and the maximum time difference corresponding to the two different coordinates are obtained, if at least two coordinate ranges are adjacent, a union set is taken for adjacent coordinate ranges to obtain a union set coordinate range and a corresponding maximum time difference, then the coordinate ranges corresponding to two adjacent different coordinates and the maximum time differences corresponding to two different coordinates are respectively used as the coordinate range of a visual attention area of a user and a corresponding attention duration, and/or taking the union set coordinate range and the corresponding maximum time difference as the coordinate range of the visual attention area of the user and the corresponding attention duration respectively. Therefore, the attention area of the user can be determined according to the visual focus of the user at different moments, and information such as the attention duration and the like of the two mutually nested areas can be obtained in a union mode, for example, the hardware parameter display area of a computer and the memory parameter display area therein.
According to an embodiment of the present disclosure, determining the attention duration of each display content by the user based on the coordinate range of each display content, the coordinate range of the visual attention area, and the corresponding attention duration, which are pre-labeled in the current display interface, may include the following operations: firstly, obtaining each first rectangle formed by the coordinate range of each display content marked in the current display interface, and obtaining each second rectangle formed by the coordinate range of the visual attention area, wherein the coordinate range of each display content marked in advance is a result of marking the coordinate range of each display content respectively aiming at different display sizes and/or resolutions, then obtaining the overlapping rate of each first rectangle and each second rectangle, then if the overlapping rate of the first rectangle and the second rectangle is larger than a set threshold value, determining the display content corresponding to the first rectangle concerned by the user, and then determining the attention duration of the user to each display content based on the attention duration of the visual attention area corresponding to the second rectangle of which the overlapping rate of the first rectangle is larger than the set threshold value. Therefore, the method and the device can simply and conveniently determine the display content and the attention duration which are concerned by the user, can be applied to various display sizes and/or resolutions, and can ensure the accuracy of the acquired information.
According to an embodiment of the present disclosure, the method may further include the operations of: firstly, after acquiring coordinates and time information of visual focuses of a user on a display screen, acquiring attention duration of the user at coordinate positions of the visual focuses on the display screen, then removing the coordinates and time information of the visual focuses with the attention duration smaller than a removal duration threshold value on the display screen to obtain the coordinates and time information of effective visual focuses on the display screen, and correspondingly, determining a coordinate range of a visual attention area of the user and a corresponding attention duration based on the coordinates and time information of the visual focuses on the display screen comprises determining the coordinate range of the visual attention area of the user and the corresponding attention duration based on the coordinates and time information of the effective visual focuses on the display screen. Because the content corresponding to the position scanned by the visual focus of the user is not the content concerned by the user, the information can be removed, the amount of invalid data is greatly reduced, the consumption of computing resources is reduced, and the computing speed is improved.
According to an embodiment of the present disclosure, the method may further include the operations of: firstly, after determining a coordinate range of a visual attention area of a user and a corresponding attention duration, filtering an invalid coordinate range and a corresponding attention duration based on a preset valid maximum area, where the invalid coordinate range is a coordinate range in which an area included in the coordinate range is larger than the valid maximum area, and correspondingly, determining the attention duration of the user for each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration pre-labeled in a current display interface may include: and determining the attention duration of the user to each display content based on the coordinate range of each display content pre-labeled in the current display interface, the coordinate range of the filtered visual attention area and the corresponding attention duration. Because the display content is definitely in the display interface or a certain partial area of the display interface, when the area of the determined visual attention area is larger than the preset maximum effective area (such as the size of the display panel, the preset area and the like), the coordinate range is an invalid coordinate range, and the corresponding attention duration is also invalid data, so that filtering can be performed, thereby further removing the invalid data, reducing the consumption of computing resources, and being beneficial to improving the computing speed.
According to an embodiment of the present disclosure, when the method is applied to a client, the method may further include the following operations: after the attention duration of each display content of the user is determined, the attention duration of the user to each display content is sent to a server, then recommendation information sent by the server is received, wherein the recommendation information is at least recommendation information determined based on the attention duration of the user to each display content, and the recommendation information is displayed to the user. According to the embodiment, the favorite or required commodities of the user and the like can be determined according to the attention degree of the user to the display content, so that the recommendation information can be displayed by the client conveniently.
According to an embodiment of the present disclosure, when the method is applied to a client, the method may further include the following operations: after determining the attention duration of the user to each display content, determining user preference information at least based on the attention duration of the user to each display content, then storing the user preference information, determining recommendation information at least based on the user preference information and a commodity information database, and displaying the recommendation information to the user.
According to an embodiment of the present disclosure, the method is applied to a server, and in an embodiment, the method may further include the following operations: firstly, after determining the attention duration of each display content by a user, determining user preference information at least based on the attention duration of each display content by the user, then storing the user preference information, determining recommendation information at least based on the user preference information and a commodity information database, and then sending the recommendation information to a client so as to facilitate the client to display the recommendation information. In another embodiment, the method may further include the following operations: firstly, after the attention duration of each display content of a user is determined, user preference information is determined at least based on the attention duration of each display content of the user, and then the user preference information is sent to a client so that the client can determine recommendation information according to the user preference information to display. In another embodiment, the method may further include the following operations: after the attention duration of the user to each display content is determined, the attention duration of the user to each display content is sent to the client, so that the client determines subsequent related operations according to the attention duration of the user to each display content.
Another aspect of the present disclosure provides an information processing apparatus, which may include a first obtaining module, a first determining module, and a second determining module, where the first obtaining module is configured to obtain coordinates and time information of a visual focus of a user on a display screen, the first determining module is configured to determine a coordinate range of a visual attention area of the user and a corresponding attention duration based on the coordinates and the time information of the visual focus on the display screen, and the second determining module is configured to determine an attention duration of the user for each display content based on a coordinate range of each display content pre-labeled in a current display interface, the coordinate range of the visual attention area, and the corresponding attention duration.
According to an embodiment of the disclosure, in an embodiment, when the apparatus is applied to a client, the client has a camera, and the first obtaining module may include a collecting unit and a first obtaining unit, where the collecting unit is configured to collect eyeball movement and focusing information of a user through the camera, and the first obtaining unit is configured to obtain coordinates of a visual focus of the user on a display screen and corresponding time according to the eyeball movement and focusing information of the user. In another embodiment, the apparatus is applicable to a server, and the first obtaining module may include a first receiving unit, where the first receiving unit is configured to receive first information, where the first information includes coordinates of a visual focus of a user on a display screen and time information.
According to an embodiment of the present disclosure, the first determining module may include a formatting unit, a sorting unit, a time difference obtaining unit, a combining unit, and a first determining unit, where the formatting unit is configured to format coordinates and time information of a visual focus of the user on a display screen to obtain a plurality of binary groups, where the binary groups include coordinates and corresponding times, the sorting unit is configured to sort the binary groups based on the times, the time difference obtaining unit is configured to obtain coordinate ranges corresponding to two adjacent different coordinates and maximum time differences corresponding to the two different coordinates, the combining unit is configured to, if at least two coordinate ranges are adjacent, merge the adjacent coordinate ranges to obtain a merged coordinate range and a corresponding maximum time difference, and the first determining unit is configured to respectively use the coordinate ranges corresponding to the two adjacent different coordinates and the maximum time differences corresponding to the two different coordinates as the visual focus of the user And/or respectively taking the coordinate range of the union set and the corresponding maximum time difference as the coordinate range of the visual attention area of the user and the corresponding attention duration.
According to an embodiment of the present disclosure, the second determining module may include a second obtaining unit, an overlapping rate calculating unit, a second determining unit and a third determining unit, wherein the second obtaining unit is configured to obtain each first rectangle formed by a coordinate range of each display content marked in the current display interface and obtain each second rectangle formed by a coordinate range of the visual attention area, the coordinate range of each display content marked in advance is a result of marking the coordinate range of each display content for different display sizes and/or resolutions, the overlapping rate calculating unit is configured to obtain an overlapping rate of each first rectangle and each second rectangle, and the second determining unit is configured to determine the display content corresponding to the first rectangle that the user pays attention to if the overlapping rate of the first rectangle and the second rectangle is greater than a set threshold value, the third determining unit is configured to determine a duration of attention of the user to each display content, based on the duration of attention of the visual attention area corresponding to the second rectangle, where the overlapping rate of the first rectangle is greater than the set threshold.
According to the embodiment of the disclosure, the apparatus may further include a second obtaining module and a removing module, where the second obtaining module is configured to obtain a duration of attention of the user at the coordinate position of each visual focus on the display screen after obtaining the coordinate and time information of the visual focus of the user on the display screen, the removing module is configured to remove the coordinate and time information of the visual focus on the display screen, where the duration of attention is smaller than a threshold of the removal duration, to obtain the coordinate and time information of the effective visual focus on the display screen, and correspondingly, the first determining module is specifically configured to determine the coordinate range of the visual attention area of the user and the corresponding duration of attention based on the coordinate and time information of the effective visual focus on the display screen.
According to an embodiment of the present disclosure, the apparatus may further include a filtering module, where the filtering module is configured to, after determining a coordinate range and a corresponding attention duration of a visual attention area of the user, filter an invalid coordinate range and a corresponding attention duration based on a preset valid maximum area, where the invalid coordinate range is a coordinate range in which an area included in the coordinate range is larger than the valid maximum area, and correspondingly, the second determining module is specifically configured to, based on a coordinate range of each display content pre-labeled in a current display interface, a coordinate range of the filtered visual attention area, and a corresponding attention duration, determine an attention duration of the user to each display content.
According to an embodiment of the disclosure, the device is suitable for a client, and may further include a first sending module, a receiving module, and a first display module, where the first sending module is configured to send the duration of attention of the user to each display content to a server after determining the duration of attention of the user to each display content, the receiving module is configured to receive recommendation information sent by the server, the recommendation information is recommendation information determined based on at least the duration of attention of the user to each display content, and the first display module is configured to display the recommendation information to the user.
According to an embodiment of the present disclosure, the apparatus is adapted to a client, and the apparatus may further include a third determining module, a fourth determining module, and a second displaying module, where the third determining module is configured to determine, after determining a duration of attention of a user to each display content, user preference information based on at least the duration of attention of the user to each display content, the fourth determining module is configured to store the user preference information and determine recommendation information based on at least the user preference information and a product information database, and the second displaying module is configured to display the recommendation information to the user.
According to an embodiment of the disclosure, the apparatus is applied to a server, and in an implementation manner, the apparatus may further include a fifth determining module, a sixth determining module, and a second sending module, where the fifth determining module is configured to determine, after determining a duration of attention of a user to each display content, user preference information based on at least the duration of attention of the user to each display content, the sixth determining module is configured to store the user preference information, and determine recommendation information based on at least the user preference information and a product information database, and the second sending module is configured to send the recommendation information to a client. In another embodiment, the apparatus may further include a seventh determining module and a third sending module, where the seventh determining module is configured to determine, after determining the attention duration of the user to each display content, user preference information based on at least the attention duration of the user to each display content, and the third sending module is configured to send the user preference information to the client. In another embodiment, the apparatus may further include a fourth sending module, configured to send the duration of attention of the user to each display content to the client after determining the duration of attention of the user to each display content.
Another aspect of the disclosure provides a computer system comprising one or more processors and a storage device for storing executable instructions that, when executed by the processors, implement a method as described above.
Another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions for implementing the method as described above when executed.
Another aspect of the disclosure provides a computer program comprising computer executable instructions for implementing the method as described above when executed.
According to the embodiment of the disclosure, the problems that the e-commerce platform is not easy to acquire the commodity information focused on by the user when the user browses the commodity list in a statistical manner, the user cannot know whether the user browses the display interface or not, and the related information such as one or more items of information of the commodity focused on by the user is not easy to acquire can be at least partially solved, and therefore the technical effects of determining the focused content and the focused degree of the user based on the visual information of the user and further determining the preference and the demand of the user can be realized.
According to the embodiment of the disclosure, the information such as the attention duration of the user to the two attention areas nested with each other can be respectively obtained, so that the attention information of the user can be further refined to determine the real requirement of the user.
According to the embodiment of the disclosure, a large amount of invalid data can be removed in the calculation process, the consumption of calculation resources is reduced, and the calculation speed is improved.
According to the embodiment of the disclosure, each operation can be performed on the same execution main body, or can be performed on different execution main bodies, for example, for devices with relatively weak computing power such as a desktop, google glasses, virtual reality head-mounted devices, and the like, it is only necessary to collect the eye video of the user, send the eye video of the user to the server for processing, and receive and display the recommendation information sent by the server, so that the application range is wide.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent from the following description of embodiments of the present disclosure with reference to the accompanying drawings, in which:
FIG. 1A schematically illustrates an application scenario of an information processing method, apparatus, and computer system and medium according to embodiments of the disclosure;
FIG. 1B schematically shows a system architecture diagram suitable for implementing an information processing method according to an embodiment of the present disclosure;
FIG. 2A schematically illustrates a flow chart of an information processing method according to an embodiment of the present disclosure;
FIG. 2B schematically illustrates a flow chart of a method of determining a coordinate range and a corresponding length of attention of a visual area of interest of a user according to an embodiment of the present disclosure;
FIG. 2C schematically illustrates a schematic diagram of a coordinate range of a visual area of interest, in accordance with an embodiment of the present disclosure;
FIG. 2D schematically illustrates a diagram of coordinate ranges of pre-labeled display content, in accordance with an embodiment of the present disclosure;
FIG. 2E schematically illustrates a flow chart of a method of determining a length of time of interest to a user for each display content in accordance with an embodiment of the present disclosure;
FIG. 3 schematically shows a flow chart of an information processing method according to another embodiment of the present disclosure;
fig. 4 schematically shows a block diagram of an information processing apparatus according to an embodiment of the present disclosure; and
FIG. 5 schematically shows a block diagram of a computer system suitable for implementing the information processing method according to an embodiment of the present disclosure.
Detailed Description
Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood that the description is illustrative only and is not intended to limit the scope of the present disclosure. In the following detailed description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the embodiments of the disclosure. It may be evident, however, that one or more embodiments may be practiced without these specific details. Moreover, in the following description, descriptions of well-known structures and techniques are omitted so as to not unnecessarily obscure the concepts of the present disclosure.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The terms "comprises," "comprising," and the like, as used herein, specify the presence of stated features, steps, operations, and/or components, but do not preclude the presence or addition of one or more other features, steps, operations, or components.
All terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It is noted that the terms used herein should be interpreted as having a meaning that is consistent with the context of this specification and should not be interpreted in an idealized or overly formal sense.
Where a convention analogous to "at least one of A, B and C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B and C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.). Where a convention analogous to "A, B or at least one of C, etc." is used, in general such a construction is intended in the sense one having skill in the art would understand the convention (e.g., "a system having at least one of A, B or C" would include but not be limited to systems that have a alone, B alone, C alone, a and B together, a and C together, B and C together, and/or A, B, C together, etc.).
The embodiment of the disclosure provides an information processing method, an information processing device, a computer system and a medium. The technical scheme provided by the disclosure is applicable to any one or more of the following scenes: commodity recommendation, news recommendation, picture recommendation, social information recommendation and the like. The method includes a visual attention area determination process and a display content attention information determination process. In the process of determining the visual attention area, firstly, the coordinate and time information of the visual focus of the user on the display screen are acquired, such as which coordinate position of the display is concerned at that moment, and then the coordinate range of the visual attention area of the user and the corresponding attention duration are determined based on the coordinate and time information of the visual focus on the display screen, such as how long the attention is paid to which area range of the display. After the visual attention area is determined, a display content attention information determining process is carried out, the attention duration of the user to each display content is determined based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration which are labeled in advance in the current display interface, and the display content and the attention duration corresponding to the visual attention area are determined according to the display content corresponding to each display area of the current display which is labeled in advance.
Fig. 1A schematically illustrates an application scenario of an information processing method, apparatus, and computer system and medium according to an embodiment of the present disclosure.
As shown in fig. 1A, taking a product display interface as an example for explanation, a display interface of the display may include a product photo, model information, inventory information, price information, payment method information, color information, hardware parameter information, and the like, a user focuses on observing the color information when browsing the current interface, and other information is simply browsed. In the prior art, the information of the commodity browsed by the user can be counted, but the actual attention degree (for example, how long the user actually watches the commodity, during which there may be a possibility that the user leaves the display interface for a period of time and then returns to continue browsing the information) and the more specific information concerned of the commodity are not convenient to count, and the information can actually represent the preference and the demand of the user more accurately. The method and the device for determining the attention of the user to the display content in the display interface based on the actual visual focus of the user help an e-commerce platform to explore the preference and the demand of the user, such as the user in FIG. 1A pays more attention to the color of a mobile phone.
Fig. 1B schematically illustrates a system architecture 100 suitable for implementing an information processing method according to an embodiment of the present disclosure. It should be noted that fig. 1B is only an example of a system architecture to which the embodiments of the present disclosure may be applied to help those skilled in the art understand the technical content of the present disclosure, and does not mean that the embodiments of the present disclosure may not be applied to other devices, systems, environments or scenarios.
As shown in fig. 1B, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, to name a few.
The user may use the terminal devices 101, 102, 103 to interact with the server 105 via the network 104 to receive or send messages or the like. The terminal devices 101, 102, 103 have image sensors for capturing video containing images of the user's eyes, on which various communication client applications may be installed, such as shopping-like applications, web browser applications, search-like applications, instant messaging tools, mailbox clients, social platform software, etc. (by way of example only).
The terminal devices 101, 102, 103 may be various electronic devices having a camera and a display screen and supporting web browsing, including but not limited to smart phones, tablets, laptop portable computers, desktop computers, internet televisions, virtual reality devices, and so on.
The server 105 may be a server that provides various services, such as a background management server (for example only) that processes video sent by the terminal devices 101, 102, 103 to determine display content that is of interest to the user, and also provides support for websites browsed by the user using the terminal devices 101, 102, 103. The background management server may analyze and perform other processing on the received data such as the user request, and feed back a processing result (e.g., a webpage, information, or data obtained or generated according to the user request) to the terminal device.
It should be noted that the information processing method provided by the embodiment of the present disclosure may be executed by the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the information processing apparatus provided by the embodiment of the present disclosure may also be disposed in the terminal devices 101, 102, 103 and/or the server 105. The information processing method provided by the embodiment of the present disclosure may also be executed by a server or a server cluster that is different from the server 105 and is capable of communicating with the terminal devices 101, 102, 103 and/or the server 105. Accordingly, the information processing apparatus provided in the embodiment of the present disclosure may also be provided in a server or a server cluster different from the server 105 and capable of communicating with the terminal devices 101, 102, 103 and/or the server 105.
It should be understood that the number of terminal devices, networks, and servers are merely illustrative. There may be any number of terminal devices, networks, and servers, as desired for implementation.
The technical scheme provided by the disclosure is applicable to any one or more of the following scenes: commodity recommendation, news recommendation, picture recommendation, social information recommendation and the like. In order to better understand the technical solution of the present disclosure, the following description takes an example of determining the commodity information (which may belong to a scene of commodity recommendation) focused by the user as an example.
Fig. 2A schematically shows a flow chart of an information processing method according to an embodiment of the present disclosure.
As shown in fig. 2A, the method includes operations S201 to S203.
In operation S201, coordinates of a visual focus of a user on a display screen and time information are acquired.
In this embodiment, the operation may be performed at the client, at the server, or at the client and the server, respectively. For example, when executed at a client, the client may have a camera, and when a user logs in a website of an e-commerce platform and browses a commodity, the client acquires a video and processes the video to determine coordinates and time information of a visual focus of the user on a display screen, specifically, eyeball movement and focusing information of the user may be acquired by the camera (for example, positioning pupils of both eyes to determine focusing information), and then coordinates and corresponding time of the visual focus of the user on the display screen are acquired according to the eyeball movement and focusing information, and the focusing information may be acquired by analyzing the pupils of the user. In addition, other prior art techniques can be used to process the video to determine the coordinate and time information of the user's visual focus on the display screen, which is not limited herein.
For example, obtaining the coordinates of the user's visual focus on the display screen may be obtained by: firstly, shooting a human eye image, carrying out binarization processing on the human eye image, then filtering by using a Gaussian kernel function to remove noise in the human eye image, then acquiring a human eye pupil center from the human eye image, and determining the coordinate of a visual focus on a display screen at least based on the two pupil centers.
For another example, the client has a camera and a near-infrared light source, and when determining the coordinates of the visual focus of the user on the display screen, the near-infrared light source can be controlled to be turned on, the camera is controlled to shoot the face of the user in real time, and for each frame of shot image, the computer obtains the pupil center and the corneal reflection point of the light source through an image processing technology, so as to obtain a two-dimensional vector from the pupil center to the corneal reflection point. The vector can be used to calculate the two-dimensional coordinates for viewing the display screen at that time by the user's correction equation. In the present embodiment, the correction equation is obtained as follows: a plurality of points, such as 9 points, appear on the screen in sequence, and the user needs to annotate in sequenceFrom these 9 points, for each point, the system knows the two-dimensional coordinates S ═ x (for that point) on the screens，ys) And the pupil center to cornea reflection vector P of the user (x)p，yp) The transformation of P and S can be obtained by equations (1) and (2).
xp＝a0+a1xs+a2ys+a3xsys+a4xs 2+a5ys 2(1)
yp＝a6+a7xs+a8ys+a9xsys+a10xs 2+a11ys 2(2)
Wherein, aiIs a parameter term, each calibration point may give a set of equations as shown above, 9 calibration points may give 9 sets of equations, parameter aiThe solution can be obtained by a least square method. However, it should be understood that the claimed subject matter is not limited to the above examples, and that other coordinate acquisition techniques of visual focus on a display screen are also within the scope of the present disclosure.
It should be noted that the size, resolution, and the like of the display of different clients may be different, and therefore, when processing the video, some information of the client, such as the brand, model, size, resolution, and the like of the client, may need to be utilized, so as to determine which coordinate of the display the visual focus of the user is located.
For another example, the method is applied to a server, and the server acquires the coordinate and the time information of the visual focus of the user on the display screen by receiving first information sent by the client, where the first information includes the coordinate and the time information of the visual focus of the user on the display screen. For another example, when the client is executed at the server, the client may obtain video information sent by the client, and obtain some client information of the client, such as information of the brand, model, display screen size, resolution, and the like of the client (the client information may be related information obtained by the server when the user logs in a website of the e-commerce platform, or may be information that is required by the server to the client when the server needs the related information, which is not limited herein), and then process the video information to obtain the coordinate and time information of the visual focus of the user on the display screen.
Then, in operation S202, a coordinate range of a visual attention area of the user and a corresponding attention duration are determined based on the coordinate of the visual focus on the display screen and time information.
In this embodiment, the coordinate range of the visual attention area and the corresponding attention duration may be determined based on the coordinate of the formatted visual focus on the display screen and the time information, for example, after the coordinates of the visual focus on the display screen are sorted by time, the maximum time interval between two adjacent different coordinates is taken as the visual attention area and the corresponding attention duration corresponding to the two coordinates. If the time of the coordinates (x1, y1) and (x2, y2) after time sorting is t1 and t2, respectively, the attention duration of the visual attention area { (x1, y1), (x2, y2) } is t2-t 1.
Fig. 2B schematically illustrates a flowchart of a method of determining a coordinate range of a visual attention area of a user and a corresponding attention duration according to an embodiment of the present disclosure.
As shown in fig. 2B, the determining the coordinate range of the visual attention area of the user and the corresponding attention duration based on the coordinate of the visual focus on the display screen and the time information may include the following operations.
In operation S2021, the coordinate and time information of the visual focus of the user on the display screen are formatted to obtain a plurality of binary groups, where each binary group includes a coordinate and a corresponding time.
For example, the coordinate and time information of the user's visual focus on the display screen is formatted into a binary form similar to the following ({ x1, y1}, t11) form. Where { x1, y1} is the coordinate of the visual focus on the display screen, and t11 is the time information corresponding to { x1, y1 }.
In operation S2022, the duplets are sorted based on time.
For example, the following form can be formed:
Figure BDA0001885175750000151
in operation S2023, a coordinate range corresponding to two adjacent different coordinates, which may refer to two adjacent coordinates with different x and y values, and a maximum time difference corresponding to the two different coordinates, are obtained.
For example, the above binary ({ x2, y2}, t22) is compared with the binary ({ x1, y1}, t11) to obtain a coordinate range ({ x1, y1}, { x2, y2}) of a visual attention area, and the attention duration is (t22-t 11).
In operation S2024, if at least two coordinate ranges are adjacent, a union set is taken for the adjacent coordinate ranges to obtain a union set coordinate range and a corresponding maximum time difference.
For example, a coordinate range ({ x1, y1}, { x2, y2}) is adjacent to a coordinate range ({ x2, y2}, { x3, y3}), a union coordinate range ({ x1, y1}, { x3, y3}) and the corresponding duration of interest (t31-t11) can be obtained. It should be noted that if the two coordinate ranges are not adjacent, the operation is not required.
In operation S2025, the coordinate ranges corresponding to the two adjacent different coordinates and the maximum time differences corresponding to the two different coordinates are respectively used as the coordinate range of the visual attention area of the user and the corresponding attention duration, and/or the union coordinate range and the corresponding maximum time difference are respectively used as the coordinate range of the visual attention area of the user and the corresponding attention duration.
For example, coordinate ranges ({ x1, y1}, { x2, y2}), and union coordinate ranges ({ x1, y1}, { x3, y3}) formed by combination are respectively taken as coordinate ranges of the visual attention area of the user, and the attention duration is (t22-t11) and (t31-t11), respectively.
By the operation, when two visual attention areas nested with each other exist, each visual attention area and the attention duration can be acquired respectively.
FIG. 2C schematically shows a schematic diagram of a coordinate range of a visual area of interest according to an embodiment of the present disclosure.
As shown in fig. 2C, this region includes 3 visual attention regions: the system comprises a commodity introduction area, a detailed content area of the commodity introduction and a multifunctional information area, wherein the multifunctional information area is one sub-area in the detailed content area of the commodity introduction. Because the detailed content area and the multifunctional information area introduced by the commodity are only display contents, the prior art is not easy to acquire the attention information of the user to the area, the user attention area with the granularity can be acquired, the detailed content area of the commodity introduction with the nesting relation can be acquired simultaneously, and the display content corresponding to the user attention area can be acquired based on the user attention area conveniently in the follow-up process.
In operation S203, the attention duration of the user to each display content is determined based on the coordinate range of each display content, the coordinate range of the visual attention area, and the corresponding attention duration, which are pre-labeled in the current display interface.
The coordinate range of each display content labeled in advance may be an area which is calibrated in advance in the display interface for the display content by the e-commerce platform and is desired to acquire the attention information, wherein because the sizes, resolutions, and the like of the display screens of different terminal devices may be different, labeling needs to be performed for different sizes, resolutions, and the like of the display screens. It should be noted that, when the client displays the web page of the e-commerce platform, it is usually necessary to adaptively adjust the display mode of the display content according to the size and resolution of the display screen, so that a certain labeled coordinate range (e.g., a standard labeled coordinate range or a labeled coordinate range of a specific electronic device) can be adaptively adjusted by using an adaptive adjustment method of the display mode, such as zooming, translating, line changing and the like according to a certain proportion, and details are not described herein.
FIG. 2D schematically illustrates a diagram of coordinate ranges of pre-labeled display content, in accordance with an embodiment of the present disclosure.
As shown in fig. 2D, the display interface has 13 pre-calibrated coordinate ranges (areas corresponding to the boxes), and the display contents corresponding to these coordinate ranges are the user preference information or the requirement information that the e-commerce platform or the manufacturer wishes to know, and each pre-calibrated coordinate range can be as follows:
for example, the coordinate recording mode of the page information item:
introduction of commodities: sp ({ x1, y1}, { x2, y2})
Description of the goods-details: sp1 ({ x11, y11}, { x21, y21})
Merchandise introduction-details-multi-functional information: sp2 ({ x112, y111}, { x212, y211})
Fig. 2E schematically illustrates a flowchart of a method of determining a user's attention duration for each display content according to an embodiment of the present disclosure.
As shown in fig. 2E, the determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration, which are pre-labeled in the current display interface, may include the following operations.
In operation S2031, first rectangles formed by coordinate ranges of the display contents marked in the current display interface are obtained, and second rectangles formed by coordinate ranges of the visual attention area are obtained, where the coordinate ranges of the pre-marked display contents are the result of marking the coordinate ranges of the display contents respectively for different display sizes and/or resolutions.
In operation S2032, an overlap ratio of the first rectangles and the second rectangles is acquired.
In operation S2033, if the overlap ratio of the first rectangle and the second rectangle is greater than the set threshold, it is determined that the user focuses on the display content corresponding to the first rectangle.
For example, it can be implemented using the following codes:
Figure BDA0001885175750000171
Figure BDA0001885175750000181
in operation S2034, a user' S attention duration for each display content is determined based on an attention duration of a visual attention area corresponding to a second rectangle whose overlap ratio of the first rectangle is greater than a set threshold.
Referring to fig. 2D, the display content focused by the user can be obtained by the above operations: the commodity introduction, the detailed contents of the commodity introduction and the multifunctional information can also acquire respective attention duration, wherein the multifunctional information is probably the most attention content of the user.
Fig. 3 schematically shows a flow chart of an information processing method according to another embodiment of the present disclosure.
As shown in fig. 3, the information processing method may further include operations S301 to S302.
In operation S301, after coordinates of the visual focus of the user on the display screen and time information are acquired, a length of time of attention of the user at a coordinate position of each visual focus on the display screen is acquired.
In operation S302, coordinates and time information of the visual focus on the display screen, of which the attention duration is less than the removal duration threshold, are removed, and coordinates and time information of the effective visual focus on the display screen are obtained;
specifically, invalid information is eliminated according to the time length of the user in one visual focus. The removal duration threshold value can be determined according to the actual use effect, such as the removal duration threshold value is slightly smaller than the duration corresponding to the normal reading speed of a person. Because the content corresponding to the position scanned by the visual focus of the user is not the content concerned by the user, the information can be removed, the amount of invalid data is greatly reduced, the consumption of computing resources is reduced, and the computing speed is improved.
Accordingly, the determining the coordinate range and the corresponding attention duration of the visual attention area of the user based on the coordinate and the time information of the visual focus on the display screen comprises determining the coordinate range and the corresponding attention duration of the visual attention area of the user based on the coordinate and the time information of the effective visual focus on the display screen.
In another embodiment, the information processing method may further include operation S303.
In operation S303, after determining a coordinate range of a visual attention area of a user and a corresponding attention duration, filtering an invalid coordinate range and a corresponding attention duration based on a preset valid maximum area, where the invalid coordinate range is a coordinate range in which an area included in the coordinate range is greater than the valid maximum area.
In this embodiment, the preset effective maximum area may be determined according to a coordinate range of each display content labeled in advance, and if the coordinate range of the visual attention area is much larger than the coordinate range of each display content labeled in advance, it may be indicated that the coordinate range of the visual attention area is invalid; in addition, the coordinate range of the visual attention area may also be determined according to the attribute information of the client, such as the size of the display screen of the client, and if the coordinate range of the visual attention area is larger than the size of the display screen of the client, the coordinate range of the visual attention area is indicated to be invalid.
Correspondingly, the determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration, which are pre-labeled in the current display interface, includes: and determining the attention duration of the user to each display content based on the coordinate range of each display content pre-labeled in the current display interface, the coordinate range of the filtered visual attention area and the corresponding attention duration.
In the method for improving the display content of the embodiment of the disclosure, it is considered that the display content is always in the display interface or a certain partial area of the display interface, and when the area of the determined visual attention area is larger than the preset maximum effective area (for example, the size of the display panel, the preset area, and the like), it indicates that the coordinate range is an invalid coordinate range, and the corresponding attention duration is also invalid data, so that filtering can be performed, so that invalid data can be further removed, consumption of computing resources is reduced, and the method is helpful for improving the computing speed.
In another embodiment, when the method is applied to a client, the method may further include the following operations in order to determine the user's taste using big data analysis and to present the relevant recommendation information.
For example, first, after determining the attention duration of the user to each display content, the attention duration of the user to each display content is sent to the server.
And then, receiving recommendation information sent by the server, wherein the recommendation information is determined at least based on the attention duration of the user to each display content. For example, as shown in fig. 2C and 2D, the server may determine that the user is interested in multiple functions, such as a copier having a printing/copying/scanning function, according to the length of time the user focuses on each display content, and may determine, in conjunction with the data of the goods information database, information of a certain number of copiers having a printing/copying/scanning function to send to the client.
Then, the recommendation information is presented to the user.
In another embodiment, when the method is applied to a client, the method may further include recommending information for the user, for example, when the client has its own database and is highly computing, the information may be recommended to the user through a local database.
For example, first, after determining the length of time of user's attention to each display content, user preference information is determined based on at least the length of time of user's attention to each display content.
Then, the user preference information is stored, and recommendation information is determined based on at least the user preference information and the commodity information database.
Then, the recommendation information is presented to the user.
For example, tourist attraction information that the user pays attention to includes: beach, coast, palm tree, etc., then the appropriate sights may be determined in the local database and presented to the user.
In another embodiment, the method is applied to a server, and in one embodiment, the method further includes: after determining the attention duration of the user to each display content, determining user preference information at least based on the attention duration of the user to each display content, storing the user preference information, determining recommendation information at least based on the user preference information and a commodity information database, and sending the recommendation information to a client.
In another embodiment, the method further comprises: after the attention duration of the user to each display content is determined, determining user preference information at least based on the attention duration of the user to each display content, and sending the user preference information to a client. The method and the system are also applicable to the server, and after determining the user preference information according to the attention duration of the user to each display content, the server sends the user preference information to the client so that the client can execute the relevant operation according to the user preference information.
In another embodiment, the method further comprises: and after determining the attention duration of the user to each display content, sending the attention duration of the user to each display content to the client. The implementation mode is suitable for the situation that the client side has a database of the client side, and information which can be recommended to the user can be determined according to the attention duration of the user to each display content.
Correspondingly, the invention also provides an information processing device.
Fig. 4 schematically shows a block diagram of an information processing apparatus according to an embodiment of the present disclosure.
As shown in fig. 4, the apparatus 400 may include a first obtaining module 410, a first determining module 420, and a second determining module 430, where the first obtaining module 410 is configured to obtain coordinates and time information of a visual focus of a user on a display screen, the first determining module 420 is configured to determine a coordinate range of a visual attention area of the user and a corresponding attention duration based on the coordinates and the time information of the visual focus on the display screen, and the second determining module 430 is configured to determine an attention duration of the user for each display content based on the coordinate range of each display content pre-labeled in a current display interface, the coordinate range of the visual attention area, and the corresponding attention duration.
For example, in an embodiment mode, when the apparatus 400 is applied to a client, the client has a camera, and the first obtaining module 410 may include a collecting unit and a first obtaining unit, where the collecting unit is configured to collect the eye movement and focus information of the user through the camera, and the first obtaining unit is configured to obtain coordinates of the visual focus of the user on the display screen and corresponding time according to the eye movement and focus information of the user. In another embodiment, the apparatus is suitable for a server, and the first obtaining module 410 may include a first receiving unit, configured to receive first information, where the first information includes coordinates of a visual focus of a user on a display screen and time information.
Specifically, the first determining module 420 may include a formatting unit, a sorting unit, a time difference obtaining unit, a combining unit, and a first determining unit, where the formatting unit is configured to format coordinates and time information of a visual focus of the user on a display screen to obtain a plurality of binary groups, each of the binary groups includes coordinates and corresponding time, the sorting unit is configured to sort the binary groups based on time, the time difference obtaining unit is configured to obtain a coordinate range corresponding to two adjacent different coordinates and a maximum time difference corresponding to the two different coordinates, the combining unit is configured to, if at least two coordinate ranges are adjacent, merge the adjacent coordinate ranges to obtain a merged coordinate range and a corresponding maximum time difference, and the first determining unit is configured to respectively use the coordinate range corresponding to the two adjacent different coordinates and the maximum time difference corresponding to the two different coordinates as a visual attention area of the user And/or respectively taking the union set coordinate range and the corresponding maximum time difference as the coordinate range of the visual attention area of the user and the corresponding attention duration.
Specifically, the second determining module 430 may include a second obtaining unit, an overlap ratio calculating unit, a second determining unit, and a third determining unit, where the second obtaining unit is configured to obtain each first rectangle formed by a coordinate range of each display content labeled in the current display interface, and obtain each second rectangle formed by a coordinate range of the visual attention area, where the coordinate range of each display content labeled in advance is a result of labeling the coordinate range of each display content for different display sizes and/or resolutions, the overlap ratio calculating unit is configured to obtain an overlap ratio of each first rectangle and each second rectangle, and the second determining unit is configured to determine that the user pays attention to the display content corresponding to the first rectangle if the overlap ratio of the first rectangle and the second rectangle is greater than a set threshold value, the third determining unit is configured to determine a duration of attention of the user to each display content, based on the duration of attention of the visual attention area corresponding to the second rectangle, where the overlapping rate of the first rectangle is greater than the set threshold.
In addition, the apparatus 400 may further include a second obtaining module 440 and a removing module 450, where the second obtaining module 440 is configured to obtain a length of attention of the user at the coordinate position of each visual focus on the display screen after obtaining the coordinate and time information of the visual focus of the user on the display screen, the removing module 450 is configured to remove the coordinate and time information of the visual focus on the display screen, where the length of attention is smaller than a threshold of the removing length of time, to obtain the coordinate and time information of the effective visual focus on the display screen, and accordingly, the first determining module 420 is specifically configured to determine the coordinate range of the visual attention area of the user and the corresponding length of attention based on the coordinate and time information of the effective visual focus on the display screen.
In addition, the apparatus 400 may further include a filtering module 460, where the filtering module 460 is configured to, after determining the coordinate range of the visual attention area of the user and the corresponding attention duration, filter an invalid coordinate range and the corresponding attention duration based on a preset valid maximum area, where the invalid coordinate range is a coordinate range in which an area included in the coordinate range is greater than the valid maximum area, and correspondingly, the second determining module 430 is specifically configured to determine the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the filtered visual attention area and the corresponding attention duration, which are pre-labeled in the current display interface.
In an embodiment, the apparatus 400 is applicable to a client, and the apparatus 400 may further include a first sending module, a receiving module, and a first presentation module, where the first sending module is configured to send the duration of attention of the user to each display content to a server after determining the duration of attention of the user to each display content, the receiving module is configured to receive recommendation information sent by the server, the recommendation information is recommendation information determined based on at least the duration of attention of the user to each display content, and the first presentation module is configured to present the recommendation information to the user.
In another embodiment, the apparatus 400 is suitable for a client, and the apparatus 400 may further include a third determining module, a fourth determining module and a second presenting module, wherein the third determining module is configured to determine user preference information based on at least the duration of the user's attention to each display content after determining the duration of the user's attention to each display content, the fourth determining module is configured to store the user preference information and determine recommendation information based on at least the user preference information and a merchandise information database, and the second presenting module is configured to present the recommendation information to the user.
In another embodiment, the apparatus 400 is applied to a server, in an implementation manner, the apparatus 400 may further include a fifth determining module, a sixth determining module, and a second sending module, wherein the fifth determining module is configured to determine, after determining the attention duration of the user to each display content, user preference information based on at least the attention duration of the user to each display content, the sixth determining module is configured to store the user preference information, and determine recommendation information based on at least the user preference information and a product information database, and the second sending module is configured to send the recommendation information to a client. In another embodiment, the apparatus 400 may further include a seventh determining module, configured to determine the user preference information based on at least the length of the user's attention to each display content after determining the length of the user's attention to each display content, and a third sending module, configured to send the user preference information to the client. In another embodiment, the apparatus 400 may further include a fourth sending module, configured to send the duration of attention paid by the user to each display content to the client after determining the duration of attention paid by the user to each display content.
It should be noted that the first obtaining module 410, the first determining module 420, the second determining module 430, the second obtaining module 440, the removing module 450, and the filtering module 460 may be respectively located in the client or the server, for example, the first obtaining module 410 is located in the client, the first determining module 420, the second determining module 430, the second obtaining module 440, the removing module 450, and the filtering module 460 are located in the server, for example, the first obtaining module 410 and the first determining module 420 are located in the client, the second determining module 430, the second obtaining module 440, the removing module 450, and the filtering module 460 are located in the server, for example, the first obtaining module 410, the first determining module 420, and the second determining module 430 are located in the client, the second obtaining module 440, the removing module 450, and the filtering module 460 are located in the server, and the like, that the above data processing process does not limit the execution subject, the display module can be executed by a main body which can realize corresponding functions, that is, other modules can be positioned at a client or a server except that the camera, each display module and the transceiver module are required to be positioned at the client. In addition, all the modules described above may be located in the client, and are not described in detail herein.
In a specific embodiment, the first obtaining module 410, the first determining module 420, the removing module 450, the filtering module 460 and the displaying module are located at a client, and the second determining module 430 and the second obtaining module 440 are located at a server.
In another specific embodiment, the first obtaining module 410 and the presenting module are located at the client, and the first determining module 420, the removing module 450, the second determining module 430, the filtering module 460, and the second obtaining module 440 are located at the server.
Any number of modules, sub-modules, units, sub-units, or at least part of the functionality of any number thereof according to embodiments of the present disclosure may be implemented in one module. Any one or more of the modules, sub-modules, units, and sub-units according to the embodiments of the present disclosure may be implemented by being split into a plurality of modules. Any one or more of the modules, sub-modules, units, sub-units according to embodiments of the present disclosure may be implemented at least in part as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented in any other reasonable manner of hardware or firmware by integrating or packaging a circuit, or in any one of or a suitable combination of software, hardware, and firmware implementations. Alternatively, one or more of the modules, sub-modules, units, sub-units according to embodiments of the disclosure may be at least partially implemented as a computer program module, which when executed may perform the corresponding functions.
For example, any plurality of the first obtaining module 410, the first determining module 420, the second determining module 430, the second obtaining module 440, the removing module 450, and the filtering module 460 may be combined and implemented in one module, or any one of them may be split into a plurality of modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of the other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the first obtaining module 410, the first determining module 420, the second determining module 430, the second obtaining module 440, the removing module 450, and the filtering module 460 may be at least partially implemented as a hardware circuit, such as a Field Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a system on a chip, a system on a substrate, a system on a package, an Application Specific Integrated Circuit (ASIC), or may be implemented by hardware or firmware in any other reasonable manner of integrating or packaging a circuit, or implemented by any one of three implementations of software, hardware, and firmware, or in a suitable combination of any of them. Alternatively, at least one of the first obtaining module 410, the first determining module 420, the second determining module 430, the second obtaining module 440, the removing module 450 and the filtering module 460 may be at least partially implemented as a computer program module, which when executed, may perform a corresponding function.
FIG. 5 schematically shows a block diagram of a computer system suitable for implementing the information processing method according to an embodiment of the present disclosure. The computer system illustrated in FIG. 5 is only one example and should not impose any limitations on the scope of use or functionality of embodiments of the disclosure.
As shown in fig. 5, a computer system 500 according to an embodiment of the present disclosure includes a processor 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM)502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. The processor 501 may comprise, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor and/or associated chipset, and/or a special purpose microprocessor (e.g., an Application Specific Integrated Circuit (ASIC)), among others. The processor 501 may also include onboard memory for caching purposes. Processor 501 may include a single processing unit or multiple processing units for performing different actions of a method flow according to embodiments of the disclosure.
In the RAM 503, various programs and data necessary for the operation of the system 500 are stored. The processor 501, the ROM 502, and the RAM 503 are connected to each other by a bus 504. The processor 501 performs various operations of the method flows according to the embodiments of the present disclosure by executing programs in the ROM 502 and/or the RAM 503. Note that the programs may also be stored in one or more memories other than the ROM 502 and the RAM 503. The processor 501 may also perform various operations of method flows according to embodiments of the present disclosure by executing programs stored in the one or more memories.
According to an embodiment of the present disclosure, system 500 may also include an input/output (I/O) interface 505, input/output (I/O) interface 505 also being connected to bus 504. The system 500 may also include one or more of the following components connected to the I/O interface 505: an input portion 506 including a keyboard, a mouse, and the like; an output portion 507 including a display such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The driver 610 is also connected to the I/O interface 505 as necessary. A removable medium 611 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 610 as necessary, so that a computer program read out therefrom is mounted into the storage section 508 as necessary.
According to embodiments of the present disclosure, method flows according to embodiments of the present disclosure may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable storage medium, the computer program containing program code for performing the method illustrated by the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section 509, and/or installed from the removable medium 611. The computer program, when executed by the processor 501, performs the above-described functions defined in the system of the embodiments of the present disclosure. The systems, devices, apparatuses, modules, units, etc. described above may be implemented by computer program modules according to embodiments of the present disclosure.
The present disclosure also provides a computer-readable storage medium, which may be contained in the apparatus/device/system described in the above embodiments; or may exist separately and not be assembled into the device/apparatus/system. The computer-readable storage medium carries one or more programs which, when executed, implement the method according to an embodiment of the disclosure.
According to embodiments of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, which may include, for example but is not limited to: a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. For example, according to embodiments of the present disclosure, a computer-readable storage medium may include ROM 502 and/or RAM 503 and/or one or more memories other than ROM 502 and RAM 503 described above.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
Those skilled in the art will appreciate that various combinations and/or combinations of features recited in the various embodiments and/or claims of the present disclosure can be made, even if such combinations or combinations are not expressly recited in the present disclosure. In particular, various combinations and/or combinations of the features recited in the various embodiments and/or claims of the present disclosure may be made without departing from the spirit or teaching of the present disclosure. All such combinations and/or associations are within the scope of the present disclosure.
The embodiments of the present disclosure have been described above. However, these examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments are described separately above, this does not mean that the measures in the embodiments cannot be used in advantageous combination. The scope of the disclosure is defined by the appended claims and equivalents thereof. Various alternatives and modifications can be devised by those skilled in the art without departing from the scope of the present disclosure, and such alternatives and modifications are intended to be within the scope of the present disclosure.

Claims (16)

1. An information processing method, the method comprising:
acquiring the coordinate and time information of a visual focus of a user on a display screen;
determining the coordinate range of a visual attention area of a user and corresponding attention duration based on the coordinate and time information of the visual focus on the display screen; and
and determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration which are pre-labeled in the current display interface.
2. The method of claim 1, wherein:
the method is applicable to a client, the client is provided with a camera, and the acquiring of the coordinate and time information of the visual focus of the user on a display screen comprises the following steps:
the camera is used for collecting eyeball movement and focusing information of a user;
acquiring coordinates of a visual focus of the user on a display screen and corresponding time according to the eyeball movement and focusing information of the user;
or
The method is applicable to a server, and the server acquires the coordinate and time information of the visual focus of a user on a display screen by receiving first information sent by a client, wherein the first information comprises the coordinate and time information of the visual focus of the user on the display screen.
3. The method of claim 1, wherein the determining a coordinate range and a corresponding length of attention of a visual attention area of a user based on the coordinate of the visual focus on the display screen and time information comprises:
formatting coordinate and time information of a visual focus of the user on a display screen to obtain a plurality of binary groups, wherein the binary groups comprise coordinates and corresponding time;
sorting the tuples based on time;
acquiring coordinate ranges corresponding to two adjacent different coordinates and maximum time differences corresponding to the two different coordinates;
if at least two coordinate ranges are adjacent, a union set is taken from the adjacent coordinate ranges to obtain a union set coordinate range and a corresponding maximum time difference; and
and respectively taking the coordinate ranges corresponding to the two adjacent different coordinates and the maximum time differences corresponding to the two different coordinates as the coordinate range of the visual attention area of the user and the corresponding attention duration, and/or respectively taking the union set coordinate range and the corresponding maximum time difference as the coordinate range of the visual attention area of the user and the corresponding attention duration.
4. The method according to claim 1, wherein the determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration, which are pre-labeled in the current display interface, comprises:
acquiring each first rectangle formed by the coordinate range of each display content marked in the current display interface and each second rectangle formed by the coordinate range of the visual attention area, wherein the coordinate range of each display content marked in advance is a result of marking the coordinate range of each display content respectively according to different display sizes and/or resolutions;
acquiring the overlapping rate of each first rectangle and each second rectangle;
if the overlapping rate of the first rectangle and the second rectangle is larger than a set threshold, determining that the user pays attention to the display content corresponding to the first rectangle; and
and determining the attention duration of each display content by the user based on the attention duration of the visual attention area corresponding to the second rectangle of which the overlapping rate of the first rectangle is greater than the set threshold.
5. The method of claim 1, further comprising:
after coordinates and time information of visual focuses of a user on a display screen are obtained, the attention duration of the user at the coordinate position of each visual focus on the display screen is obtained;
removing the coordinate and time information of the visual focus on the display screen, wherein the attention duration is less than the removal duration threshold value, and obtaining the coordinate and time information of the effective visual focus on the display screen; and
the determining the coordinate range and the corresponding attention duration of the visual attention area of the user based on the coordinate and the time information of the visual focus on the display screen comprises determining the coordinate range and the corresponding attention duration of the visual attention area of the user based on the coordinate and the time information of the effective visual focus on the display screen.
6. The method of claim 1, further comprising:
after the coordinate range of a visual attention area of a user and the corresponding attention duration are determined, filtering an invalid coordinate range and the corresponding attention duration based on a preset effective maximum area, wherein the invalid coordinate range is a coordinate range of which the area included in the coordinate range is larger than the effective maximum area; and
the determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration pre-labeled in the current display interface comprises: and determining the attention duration of the user to each display content based on the coordinate range of each display content pre-labeled in the current display interface, the coordinate range of the filtered visual attention area and the corresponding attention duration.
7. The method of claim 1, wherein the method is adapted for a client, the method further comprising:
after determining the attention duration of each display content by the user, sending the attention duration of each display content by the user to a server;
receiving recommendation information sent by the server, wherein the recommendation information is determined at least based on the attention duration of the user to each display content; and
and displaying the recommendation information to the user.
8. The method of claim 1, wherein the method is adapted for a client, the method further comprising:
after determining the attention duration of the user to each display content, determining user preference information at least based on the attention duration of the user to each display content;
storing the user preference information and determining recommendation information based at least on the user preference information and a goods information database; and
and displaying the recommendation information to the user.
9. The method of claim 1, wherein the method is adapted for a server, the method further comprising:
determining user preference information based on at least a length of time of user attention to each display content after determining the length of time of user attention to each display content,
storing the user preference information and determining recommendation information based at least on the user preference information and a goods information database,
sending the recommendation information to a client;
or
Determining user preference information based on at least a length of time of user attention to each display content after determining the length of time of user attention to each display content,
sending the user preference information to a client;
or
And after determining the attention duration of the user to each display content, sending the attention duration of the user to each display content to the client.
10. An information processing apparatus, the apparatus comprising:
the first acquisition module is used for acquiring the coordinate and time information of a visual focus of a user on a display screen;
the first determination module is used for determining the coordinate range of a visual attention area of a user and the corresponding attention duration based on the coordinate of the visual focus on the display screen and the time information; and
and the second determination module is used for determining the attention duration of the user to each display content based on the coordinate range of each display content, the coordinate range of the visual attention area and the corresponding attention duration which are pre-labeled in the current display interface.
11. The apparatus of claim 10, wherein the first determining means comprises:
the formatting unit is used for formatting the coordinate and the time information of the visual focus of the user on the display screen to obtain a plurality of binary groups, and the binary groups comprise coordinates and corresponding time;
a sorting unit for sorting the tuples based on time;
the time difference acquisition unit is used for acquiring a coordinate range corresponding to two adjacent different coordinates and a maximum time difference corresponding to the two different coordinates;
the merging unit is used for merging the adjacent coordinate ranges to obtain a union set coordinate range and a corresponding maximum time difference if the at least two coordinate ranges are adjacent; and
and the first determining unit is used for respectively taking the coordinate ranges corresponding to the two adjacent different coordinates and the maximum time difference corresponding to the two different coordinates as the coordinate range of the visual attention area of the user and the corresponding attention duration, and/or respectively taking the union coordinate range and the corresponding maximum time difference as the coordinate range of the visual attention area of the user and the corresponding attention duration.
12. The method of claim 10, wherein the second determination module comprises:
a second obtaining unit, configured to obtain each first rectangle formed by a coordinate range of each display content labeled in the current display interface and obtain each second rectangle formed by a coordinate range of the visual attention area, where the coordinate range of each display content labeled in advance is a result of labeling the coordinate range of each display content for different display sizes and/or resolutions, respectively;
the overlapping rate calculation unit is used for acquiring the overlapping rate of each first rectangle and each second rectangle;
a second determination unit, configured to determine that the user focuses on the display content corresponding to the first rectangle if the overlapping rate of the first rectangle and the second rectangle is greater than a set threshold; and
and the third determining unit is used for determining the attention duration of the user to each display content according to the attention duration of the visual attention area corresponding to the second rectangle of which the overlapping rate of the first rectangle is greater than the set threshold.
13. The apparatus of claim 10, further comprising:
the second acquisition module is used for acquiring the attention duration of the user at the coordinate position of each visual focus on the display screen after acquiring the coordinate and time information of the visual focus of the user on the display screen;
the removing module is used for removing the coordinate and the time information of the visual focus on the display screen, wherein the attention duration of the visual focus is smaller than the removing duration threshold value, and obtaining the coordinate and the time information of the effective visual focus on the display screen; and
the first determining module is specifically configured to determine a coordinate range of a visual attention area of the user and a corresponding attention duration based on the coordinate of the effective visual focus on the display screen and the time information.
14. The apparatus of claim 10, further comprising:
the filtering module is used for filtering an invalid coordinate range and a corresponding attention duration based on a preset effective maximum area after determining a coordinate range of a visual attention area of a user and the corresponding attention duration, wherein the invalid coordinate range is a coordinate range of which the area included in the coordinate range is larger than the effective maximum area; and
the second determining module is specifically configured to determine the attention duration of the user to each display content based on the coordinate range of each display content pre-labeled in the current display interface, the coordinate range of the filtered visual attention area, and the corresponding attention duration.
15. A computer system, comprising:
one or more processors;
a storage device for storing executable instructions which, when executed by the processor, implement the method of any one of claims 1 to 9.
16. A computer readable storage medium having stored thereon executable instructions which, when executed by a processor, implement a method according to any one of claims 1 to 9.
CN201811443393.8A 2018-11-29 2018-11-29 Information processing method, information processing apparatus, computer system, and medium Pending CN111241385A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201811443393.8A CN111241385A (en) 2018-11-29 2018-11-29 Information processing method, information processing apparatus, computer system, and medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201811443393.8A CN111241385A (en) 2018-11-29 2018-11-29 Information processing method, information processing apparatus, computer system, and medium

Publications (1)

Publication Number Publication Date
CN111241385A true CN111241385A (en) 2020-06-05

Family

ID=70865411

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201811443393.8A Pending CN111241385A (en) 2018-11-29 2018-11-29 Information processing method, information processing apparatus, computer system, and medium

Country Status (1)

Country Link
CN (1) CN111241385A (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111640011A (en) * 2020-06-11 2020-09-08 中国银行股份有限公司 Request method, request device, server and storage medium
CN113655882A (en) * 2021-08-17 2021-11-16 中国人民解放军军事科学院战争研究院 Human-computer interface information screening method based on eye movement data measurement
CN114063780A (en) * 2021-11-18 2022-02-18 兰州乐智教育科技有限责任公司 Method and device for determining user concentration degree, VR equipment and storage medium
WO2022041182A1 (en) * 2020-08-31 2022-03-03 华为技术有限公司 Method and device for making music recommendation

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103440038A (en) * 2013-08-28 2013-12-11 中国人民大学 Information collection system based on eye recognition and application thereof
CN105446673A (en) * 2014-07-28 2016-03-30 华为技术有限公司 Screen display method and terminal device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103440038A (en) * 2013-08-28 2013-12-11 中国人民大学 Information collection system based on eye recognition and application thereof
CN105446673A (en) * 2014-07-28 2016-03-30 华为技术有限公司 Screen display method and terminal device

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111640011A (en) * 2020-06-11 2020-09-08 中国银行股份有限公司 Request method, request device, server and storage medium
CN111640011B (en) * 2020-06-11 2023-09-26 中国银行股份有限公司 Request method, request device, server and storage medium
WO2022041182A1 (en) * 2020-08-31 2022-03-03 华为技术有限公司 Method and device for making music recommendation
CN113655882A (en) * 2021-08-17 2021-11-16 中国人民解放军军事科学院战争研究院 Human-computer interface information screening method based on eye movement data measurement
CN114063780A (en) * 2021-11-18 2022-02-18 兰州乐智教育科技有限责任公司 Method and device for determining user concentration degree, VR equipment and storage medium

Similar Documents

Publication Publication Date Title
US10771685B2 (en) Automatic guided capturing and presentation of images
US10621954B2 (en) Computerized system and method for automatically creating and applying a filter to alter the display of rendered media
CN111241385A (en) Information processing method, information processing apparatus, computer system, and medium
US9729792B2 (en) Dynamic image selection
JP2016530593A (en) Assessing the clarity of an image
CA3083486A1 (en) Method, medium, and system for live preview via machine learning models
KR102547527B1 (en) Method and device for labeling objects
CN110825988A (en) Information display method and device and electronic equipment
US20220172276A1 (en) System for shopping mall service using eye tracking technology and computing device for executing same
EP3956810B1 (en) Generating videos
JP7130771B2 (en) Attention information processing method and device, storage medium, and electronic device
CN112967299B (en) Image cropping method and device, electronic equipment and computer readable medium
US11227307B2 (en) Media content tracking of users' gazing at screens
Chu et al. Contrast-sensitivity-function-based clutter metric
EP3327652A1 (en) Automatic selection of items for a computerized graphical advertisement display using a computer-generated multidimensional vector space
CN112015936B (en) Method, device, electronic equipment and medium for generating article display diagram
CN111914861A (en) Target detection method and device
US10776860B2 (en) Retail website user interface, systems, and methods for displaying trending looks
Pourazar et al. A Comprehensive Framework for Evaluation of Stereo Correspondence Solutions in Immersive Augmented and Virtual Realities
CN112395507A (en) House source information recommendation method, device, equipment and readable storage medium
CN113723351A (en) Interface optimization method, system, storage medium and electronic equipment
Jia et al. Character superimposition inpainting in surveillance video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination