WO2019218427A1

WO2019218427A1 - Method and apparatus for detecting degree of attention based on comparison of behavior characteristics

Info

Publication number: WO2019218427A1
Application number: PCT/CN2018/092786
Authority: WO
Inventors: 陈鹏宇; 卢炀; 赵鹏祥
Original assignee: 深圳市鹰硕技术有限公司
Priority date: 2018-05-17
Filing date: 2018-06-26
Publication date: 2019-11-21
Also published as: CN108875785B; CN108875785A

Abstract

The present disclosure relates to a method and apparatus for detecting a degree of attention based on the comparison of behavior characteristics, an electronic device and a storage medium. The method comprises: acquiring video information collected by a video collection device; recognizing voice instruction information, and analyzing a user behavior characteristic in user behavior information within a specified time period after the recognition of the voice instruction information; searching a pre-established standard behavior characteristic model for a standard behavior characteristic corresponding to the current voice instruction information, and comparing the user behavior characteristic with the standard behavior characteristic; and scoring, according to a pre-set scoring standard, a user behavior characteristic inconsistent with the standard behavior characteristic, compiling statistics on scoring results in all the time periods of the video information, and generating the teaching degree of attention to the video information according to the scoring results. According to the present disclosure, a teaching degree of attention can be generated by means of consistency analysis of user behavior.

Description

Attention detection method and device based on behavior feature comparison

Technical field

The present disclosure relates to the field of computer technology, and in particular to a method, device, electronic device, and computer readable storage medium based on behavioral feature comparison.

Background technique

In teaching, the rapid and accurate detection of students' attention to teaching content can not only remind teachers of the key teaching of high-interest teaching content, but also promote students' emphasis on different knowledge points of knowledge, and do more with less. effect.

However, in actual teaching, it is usually through the teacher's personal experience to actually observe the learning state of each student to achieve the student's attention to the teaching content. This method not only occupies the teacher's teaching energy, but also is difficult to Suitable for occasions such as online teaching.

The patent application with the application number CN201110166693.8 discloses a method for quantifying the regional attention of an object, comprising: acquiring a line of sight direction of a human eye; recording a stay time of the line of sight direction in each area of the object; and staying time Long areas give high attention weights, and areas with short stay times give low attention weights. Since the application mainly evaluates the degree of attention through the analysis of the duration of the human eye, and does not comprehensively reflect various attention indicators in the teaching content, it is impossible to comprehensively evaluate the attention of the teaching content.

The patent application with the application number CN201110166693.8 discloses a method and device for evaluating user attention. The method for evaluating user attention includes: detecting a line of sight direction of the user; and determining an area on the screen corresponding to the detected line of sight direction; Obtaining a metric of the user's expression on the determined area for each of the predetermined emotions; and generating a user's attention to the determined area based on the obtained metric. Since the application mainly realizes the evaluation of the degree of attention through the user's measure of the direction of the emotion, it cannot comprehensively reflect various attention indicators in the teaching content, and cannot comprehensively evaluate the attention of the teaching content.

The patent application with the application number CN201110166693.8 discloses a method and system for counting attention degree of display screen, which transmits a wireless access broadcast signal through a wireless access point device; and a user who enters a signal strength distance range of the wireless access point device The terminal receives the radio access broadcast signal and accesses the wireless access point device, and obtains information of the accessed user terminal, such as a unique identifier and access time information, through the wireless access point device; the device accesses the device through the wireless access point device. The information of the user terminal is provided to the server; and the degree of attention of the target display screen is counted by the server according to the information of the accessed user terminal. Since the application mainly collects statistics on the degree of connection of the display by counting the number of connected devices in the wireless access point, the method can only be used in a situation where the user is concentrated, and is not suitable for the situation in which the network teaching user is more dispersed. .

Therefore, it is desirable to provide one or more technical solutions that at least solve the above problems.

It should be noted that the information disclosed in the Background section above is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Summary of the invention

An object of the present disclosure is to provide a method, device, electronic device, and computer readable storage medium based on behavioral feature comparison, thereby at least partially overcoming one or more of the limitations and disadvantages of the related art. problem.

According to an aspect of the present disclosure, a method for detecting attention based on behavior feature comparison is provided, including:

The video information obtaining step is to obtain video information collected by the video collection device, where the video information includes user behavior information and voice instruction information;

a behavior feature recognition step of identifying voice instruction information, and analyzing user behavior characteristics in the user behavior information within a specified time period after the voice instruction information is recognized;

The behavior feature comparison step searches for a standard behavior feature corresponding to the current voice instruction information in the pre-established standard behavior feature model, and compares the user behavior feature with the standard behavior feature;

a degree of interest scoring step of scoring a user behavior feature that is inconsistent with the standard behavior feature according to a pre-designed sub-criteria, and collecting a score result in the entire time period of the video information, and generating the video information according to the score result. Teaching attention.

In an exemplary embodiment of the present disclosure, the behavior feature identifying step includes:

After analyzing the user behavior feature in the user behavior information in the specified time period, the same user behavior feature number in the specified time period is greater than the preset number of user behavior features as a standard behavior feature to be referenced;

The method further includes:

The pre-established standard behavior feature model is trained according to the standard behavior characteristics to be referenced.

In an exemplary embodiment of the present disclosure, comparing the user behavior characteristic to the standard behavior characteristic includes:

Comparing whether the user behavior information in the video information includes a standard behavior characteristic corresponding to the voice instruction information within a specified time period after the voice instruction information is recognized, and if yes, determining whether the retention duration of the user behavior feature satisfies the standard The preset time range for behavioral characteristics.

In an exemplary embodiment of the present disclosure, the method further includes a scoring standard generating step, the scoring standard generating step comprising:

If the retention duration of the user behavior feature satisfies the preset time range of the standard behavior feature, a full score is calculated for the user behavior characteristic of the corresponding user in the specified time period.

If the user behavior information in the video information does not include the standard behavior feature corresponding to the voice instruction information within a specified time period after the voice instruction information is recognized, the user behavior characteristic of the corresponding user in the specified time period is zero.

In an exemplary embodiment of the present disclosure, the user behavior information includes a user's head rotation behavior and an eye focus behavior; and the behavior feature recognition step includes:

Extracting a head moving direction, a moving angle, and an eye focusing state in a user's eye focusing behavior in a user's head rotation behavior;

The head moving direction, the moving angle, and the eye focusing state in the user's eye focusing behavior are taken as user behavior characteristics.

In an exemplary embodiment of the present disclosure, the eye focus behavior includes closed eye information of a user's eyes, and the behavior feature identification step further includes:

The user's eyes are closed according to the closed eye information of the user's eyes.

If the user's eye-closing behavior is determined to be greater than the preset duration according to the user's eye focus behavior, the user behavior characteristic of the corresponding user in the specified time period is zero.

In an exemplary embodiment of the present disclosure, the user behavior obtaining step includes:

Obtaining video information collected by the same video collection device and containing user behavior information of multiple users; or

Acquiring video information collected by one or more video collection devices that includes user behavior information of multiple users.

In an exemplary embodiment of the present disclosure, the method further includes:

After collecting user behavior information of multiple users, respectively generating user behavior feature scores corresponding to the respective users and generating attention degrees;

The user behavior attention of each user is sorted, and each user's attention ranking is sent to the specified object.

In an aspect of the present disclosure, a attention degree detecting apparatus based on behavior feature comparison is provided, including:

a video information acquiring module, configured to acquire video information collected by the video collection device, where the video information includes user behavior information and voice instruction information;

a behavior feature recognition module, configured to identify voice instruction information, and analyze user behavior characteristics in the user behavior information within a specified time period after the voice instruction information is recognized;

a behavior feature comparison module, configured to search for a standard behavior feature corresponding to current voice instruction information in a pre-established standard behavior feature model, and compare the user behavior feature with the standard behavior feature;

a degree of interest scoring module, configured to score a user behavior feature that is inconsistent with the standard behavior feature according to a pre-designed sub-criteria, and collect a score result in the entire time period of the video information, and generate the video according to the score result. The teaching attention of information.

In an aspect of the disclosure, an electronic device is provided, comprising:

Processor;

A memory having stored thereon computer readable instructions that, when executed by the processor, implement the method of any of the above.

In an aspect of the present disclosure, a computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor, implements the method of any of the above.

The behavior feature comparison-based attention degree detection method in the exemplary embodiment of the present disclosure acquires video information collected by the video collection device, identifies voice instruction information, and analyzes the user behavior information within a specified time period after the voice instruction information is recognized and analyzed. User behavior characteristics, searching for a standard behavior feature corresponding to current voice instruction information in a pre-established standard behavior feature model, and comparing the user behavior feature with the standard behavior feature, according to a pre-designed standard The user behavior characteristics in which the standard behavior characteristics are inconsistent are scored, and the score results in the entire time period of the video information are counted, and the teaching attention degree of the video information is generated according to the score result. On the one hand, comparing the voice instruction information with the user behavior characteristics, it is possible to accurately determine whether the user behavior is consistent with the voice instruction information, thereby achieving accurate inference of attention; on the other hand, the duration of the user behavior characteristics in a specified time period The analysis realizes the method of scoring the corresponding proportion of the user behavior characteristics, so that the statistical result of the user attention degree is more accurate.

The above general description and the following detailed description are intended to be illustrative and not restrictive.

DRAWINGS

The above and other features and advantages of the present disclosure will become more apparent from the detailed description.

FIG. 1 illustrates a flowchart of a attention degree detection method based on behavior feature comparison according to an exemplary embodiment of the present disclosure;

FIG. 2 is a schematic diagram showing an application scenario of a attention degree detection method based on behavior feature comparison according to an exemplary embodiment of the present disclosure; FIG.

FIG. 3 is a schematic diagram showing an application scenario of a attention degree detection method based on behavior feature comparison according to an exemplary embodiment of the present disclosure; FIG.

FIG. 4 illustrates a schematic block diagram of a attention degree detecting apparatus based on behavior feature comparison according to an exemplary embodiment of the present disclosure; FIG.

FIG. 5 schematically illustrates a block diagram of an electronic device in accordance with an exemplary embodiment of the present disclosure;

FIG. 6 schematically illustrates a schematic diagram of a computer readable storage medium in accordance with an exemplary embodiment of the present disclosure.

Detailed ways

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be embodied in a variety of forms and should not be construed as being limited to the embodiments set forth herein. To those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and the repeated description thereof will be omitted.

Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are set forth However, one skilled in the art will appreciate that the technical solution of the present disclosure may be practiced without one or more of the specific details, or other methods, components, materials, devices, steps, etc. may be employed. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the present disclosure.

The block diagrams shown in the figures are merely functional entities and do not necessarily have to correspond to physically separate entities. That is, these functional entities may be implemented in software, or implemented in one or more software-hardened modules, or in different network and/or processor devices and/or microcontroller devices. Implement these functional entities.

In the present exemplary embodiment, a method for detecting attention based on behavior feature comparison is first provided, which can be applied to an electronic device such as a computer; as shown in FIG. 1, the method for detecting attention based on behavior feature comparison may include the following step:

The video information obtaining step S110 is performed to acquire video information collected by the video collection device, where the video information includes user behavior information and voice instruction information;

The behavior feature identifying step S120 is to identify the voice instruction information, and analyze the user behavior characteristics in the user behavior information within a specified time period after the voice instruction information is recognized;

The behavior feature comparison step S130 searches for a standard behavior feature corresponding to the current voice instruction information in the pre-established standard behavior feature model, and compares the user behavior feature with the standard behavior feature;

The degree of interest scoring step S140, the user behavior characteristics that are inconsistent with the standard behavior characteristics are scored according to the pre-designed sub-criteria, and the score results in the entire time period of the video information are counted, and the video information is generated according to the score result. Teaching attention.

According to the attention feature detection method based on the behavior feature comparison in the exemplary embodiment, on the one hand, comparing the voice instruction information with the user behavior feature, it is possible to accurately determine whether the user behavior is consistent with the voice instruction information, thereby achieving the attention degree. Accurate inference; on the other hand, analyzing the duration of user behavior characteristics in a specified time period to achieve a proportional proportion of user behavior characteristics, so that the statistical results of user attention are more accurate.

Hereinafter, the attention degree detecting method based on the behavior feature comparison in the present exemplary embodiment will be further described.

In the video information obtaining step S110, the video information collected by the video collection device may be acquired, where the video information includes user behavior information and voice instruction information.

In the exemplary embodiment, in a common teaching scenario, especially in a network teaching scenario, a video capture device may collect a user's video, and by analyzing user behavior information in the video and voice command information in the audio, As a basis for further judging user attention.

In the example implementation, the user behavior obtaining step includes: acquiring video information of user behavior information of multiple users collected by the same video collection device; or acquiring acquisition of one or more video collection devices, including multiple users. Video information of user behavior information. For example, in a realistic teaching scene, a video image of all users in the entire classroom can be collected through a camera placed in front of the teaching, and in the network teaching scene, the camera is viewed through the user video, especially the mobile portable device. , can collect video images of each network teaching user.

In the behavior feature identification step S120, the voice instruction information may be identified, and the user behavior feature in the user behavior information within a specified time period after the voice instruction information is recognized is analyzed.

In the example embodiment, the user's voice instruction information can be identified, and the user's next action behavior can be determined. For example, in a real teaching scenario, all users are watching the content on the blackboard in front of the classroom, and then the teacher voice command. "Please open the third page of the textbook". At this time, all users should turn to read the contents of the book from the blackboard in front of the classroom. This behavior includes the characteristics of the user's behavior; for example, when the teaching content is the first page of the textbook. Then, the teacher voice command "please see the exercises on the second page", at this time the user should have the action of head rotation and eye focus change.

In this example embodiment, the user behavior information includes a user's head rotation behavior and an eye focus behavior; the behavior feature recognition step includes: extracting a head movement direction, a movement angle, and a user's eye in a user's head rotation behavior The eye focus state in the focus behavior; the head movement direction, the movement angle, and the eye focus state in the user's eye focus behavior as the user behavior characteristics. All of the above behaviors can be used as user behavior information as a basis for user attention changes.

In this example embodiment, the behavior feature identifying step includes: after analyzing the user behavior feature in the user behavior information within a specified time period, the same number of user behavior features in the specified time period is greater than a preset number. The user behavior characteristics are taken as standard behavioral features to be referenced. In some teaching scenarios, there may be no voice command information. In the teaching scenario shown in Figure 2, the teacher tells the content of the first page of the text. As the progress of the logic is advanced, the second page is spoken, then the users of all users. The behavior of the head rotation or the focus state of the glasses should have a consistent animation trend. Statistical analysis of such behavior trends as a standard behavioral feature to be referenced can be compared as a basis for judging behavior. The method further includes: training a pre-established standard behavior feature model according to the standard behavior feature to be referenced, and collecting statistics on the results of some standard behavior characteristics to be referenced, which can be stored as a standard behavior feature model, and reduced in actual operation comparison. Calculate the load and speed up the response.

In the behavior feature comparison step S130, a standard behavior feature corresponding to the current voice instruction information may be searched in the pre-established standard behavior feature model, and the user behavior feature is compared with the standard behavior feature.

In this example embodiment, the correspondence between the received voice instruction information and the standard behavior feature may be modeled, and when the voice instruction information is received, the voice instruction information may be directly used as an input to search in the model. The corresponding standard behavior characteristics are obtained, and then the behavioral state of the user can be judged by comparing the standard behavior characteristics with the user behavior characteristics. For example, when the user views the content of the blackboard, the teacher's voice command “Please open the third page of the textbook” is received, and the corresponding standard behavior feature is that the head is turned down sharply and then rotated slightly, and the user who does not perform the standard behavior feature will be Mark and proceed with the next step in data processing.

In this example embodiment, comparing the user behavior feature with the standard behavior feature includes: comparing whether the user behavior information in the video information includes the voice instruction information within a specified time period after the voice instruction information is recognized The corresponding standard behavior feature, if included, determines whether the retention duration of the user behavior feature satisfies a preset time range of the standard behavior feature. The user performs the standard behavior feature according to the requirement of the preset time period, which may be all or part of the duration of the specified time period, within a specified time period of the standard behavior feature.

In the attention degree scoring step S140, the user behavior characteristics that are inconsistent with the standard behavior characteristics may be scored according to the pre-designed sub-criteria, and the scoring results in the entire time period of the video information are counted, and the scoring result is generated according to the scoring result. The teaching attention of video information.

In the present exemplary embodiment, according to the comparison of the user behavior characteristics and the standard behavior characteristics, especially the behavior characteristics comparison in a specified time period, it can be used as an important basis for the teaching attention degree score.

In this example embodiment, the method further includes a scoring standard generating step, the scoring standard generating step comprising: if the retention duration of the user behavior feature satisfies a preset time range of the standard behavior feature, the designating The user's behavioral characteristics corresponding to the user are scored within the time period. The standard behavior feature is completed in a preset time range within a specified time period, indicating that the user's attention level reaches a preset standard, and the user's attention degree in the specified time period is scored.

In this example embodiment, the method further includes a scoring standard generating step, the scoring standard generating step includes: if the comparison is within a specified time period after the voice instruction information is recognized, the user behavior information in the video information does not include The standard behavior feature corresponding to the voice instruction information is zeroed for the user behavior characteristic of the corresponding user in the specified time period. A user behavior that does not have a standard behavior feature within a specified time period indicates that the user's attention does not reach the preset standard at all, and the user's attention at the specified time period is zero.

In the exemplary embodiment, the eye focus behavior includes closed eye information of the user's eyes, and the behavior feature recognition step further includes: counting the user's closed eye duration according to the closed eye information of the user's eyes. The statistics on the duration of closed eyes are mainly to achieve statistics on the situation of dozing in the teaching scene.

In this example, the method further includes a scoring standard generating step, where the scoring standard generating step includes: determining, according to the user's eye focusing behavior, that the user's closed eye duration is greater than a preset duration, within the specified time period, The user's behavior characteristics of the corresponding user are recorded in the specified time period.

In this example embodiment, the user behavior information includes a user's head and face behavior; the behavior feature identification step further includes: extracting a facial feature of the user; searching for the user information in the preset user facial feature database; Correspondence between user information and user behavior characteristics. The face recognition algorithm based on the video collection device can be used as a means for comparing and searching user information in the method. By identifying the user's face and finding the user information of the corresponding user, the automatic rating statistics of the user's attention can be realized and corresponding to the user. There is no need to manually re-identify the recognition calculations.

In this example, the method further includes: after collecting the user behavior information of the multiple users, respectively generating a user behavior feature score corresponding to each user, and generating a teaching attention degree according to the user behavior feature score; The teaching attention is sorted, and the teaching attention is sorted and sent to the specified object. FIG. 3 is an application scenario of a degree of attention display. In the teaching scenario, the user attention ranking is obtained according to the collected user behavior information of all users, and is pushed to all users and/or the instructor for display.

It should be noted that, although the various steps of the method of the present disclosure are described in a particular order in the drawings, this does not require or imply that the steps must be performed in the specific order, or that all the steps shown must be performed. Achieve the desired results. Additionally or alternatively, certain steps may be omitted, multiple steps being combined into one step execution, and/or one step being decomposed into multiple step executions and the like.

Further, in the present exemplary embodiment, a attention degree detecting device based on behavior feature comparison is also provided. Referring to FIG. 4, the behavioral feature comparison-based attention degree detecting apparatus 400 may include: a video information acquiring module 410, a behavior feature identifying module 420, a behavior feature comparing module 430, and a attention degree scoring module 440. among them:

The video information obtaining module 410 is configured to acquire video information collected by the video collection device, where the video information includes user behavior information and voice instruction information;

The behavior feature identification module 420 is configured to identify voice instruction information, and analyze user behavior characteristics in the user behavior information within a specified time period after the voice instruction information is recognized;

The behavior feature comparison module 430 is configured to search for a standard behavior feature corresponding to the current voice instruction information in the pre-established standard behavior feature model, and compare the user behavior feature with the standard behavior feature;

The attention scoring module 440 is configured to score the user behavior characteristics that are inconsistent with the standard behavior characteristics according to the pre-designed sub-criteria, and collect the scoring results in the entire time period of the video information, and generate the The teaching attention of video information.

The specific details of the attention degree detecting device module based on the behavior feature comparison in the above have been described in detail in the corresponding audio paragraph identifying method, and thus will not be described herein.

It should be noted that although several modules or units of the attention-aware detection apparatus 400 based on behavioral feature comparison are mentioned in the above detailed description, such division is not mandatory. Indeed, in accordance with embodiments of the present disclosure, the features and functions of two or more modules or units described above may be embodied in one module or unit. Conversely, the features and functions of one of the modules or units described above may be further divided into multiple modules or units.

Further, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.

Those skilled in the art will appreciate that various aspects of the present invention can be implemented as a system, method, or program product. Accordingly, aspects of the present invention may be embodied in the form of a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or a combination of hardware and software aspects, which may be collectively referred to herein. "Circuit," "module," or "system."

An electronic device 500 in accordance with such an embodiment of the present invention is described below with reference to FIG. The electronic device 500 shown in FIG. 5 is merely an example and should not impose any limitation on the function and scope of use of the embodiments of the present invention.

As shown in FIG. 5, electronic device 500 is embodied in the form of a general purpose computing device. The components of the electronic device 500 may include, but are not limited to, the at least one processing unit 510, the at least one storage unit 520, the bus 530 connecting the different system components (including the storage unit 520 and the processing unit 510), and the display unit 540.

Wherein the storage unit stores program code, which can be executed by the processing unit 510, such that the processing unit 510 performs various exemplary embodiments according to the present invention described in the "Exemplary Method" section of the present specification. The steps of the examples. For example, the processing unit 510 can perform steps S110 to S140 as shown in FIG. 1.

The storage unit 520 can include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 5201 and/or a cache storage unit 5202, and can further include a read only storage unit (ROM) 5203.

The storage unit 520 can also include a program/utility 5204 having a set (at least one) of the program modules 5205, such as but not limited to: an operating system, one or more applications, other program modules, and program data, Implementations of the network environment may be included in each or some of these examples.

Bus 530 may be representative of one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local area using any of a variety of bus structures. bus.

The electronic device 500 can also communicate with one or more external devices 570 (eg, a keyboard, pointing device, Bluetooth device, etc.), and can also communicate with one or more devices that enable the user to interact with the electronic device 500, and/or with Any device (eg, router, modem, etc.) that enables the electronic device 500 to communicate with one or more other computing devices. This communication can take place via an input/output (I/O) interface 550. Also, electronic device 500 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) via network adapter 560. As shown, network adapter 560 communicates with other modules of electronic device 500 via bus 530. It should be understood that although not shown in the figures, other hardware and/or software modules may be utilized in conjunction with electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives. And data backup storage systems, etc.

Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described herein may be implemented by software, or may be implemented by software in combination with necessary hardware. Therefore, the technical solution according to an embodiment of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network. A number of instructions are included to cause a computing device (which may be a personal computer, server, terminal device, or network device, etc.) to perform a method in accordance with an embodiment of the present disclosure.

In an exemplary embodiment of the present disclosure, there is also provided a computer readable storage medium having stored thereon a program product capable of implementing the above method of the present specification. In some possible embodiments, aspects of the present invention may also be embodied in the form of a program product comprising program code for causing said program product to run on a terminal device The terminal device performs the steps according to various exemplary embodiments of the present invention described in the "Exemplary Method" section of the present specification.

Referring to FIG. 6, a program product 600 for implementing the above method, which may employ a portable compact disk read only memory (CD-ROM) and includes program code, and may be in a terminal device, is illustrated in accordance with an embodiment of the present invention. For example running on a personal computer. However, the program product of the present invention is not limited thereto, and in the present document, the readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus or device.

The program product can employ any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive lists) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

The computer readable signal medium may include a data signal that is propagated in the baseband or as part of a carrier, carrying readable program code. Such propagated data signals can take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium can also be any readable medium other than a readable storage medium that can transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

Program code embodied on a readable medium can be transmitted using any suitable medium, including but not limited to wireless, wireline, optical cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, etc., including conventional procedural Programming language—such as the "C" language or a similar programming language. The program code can execute entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the remote computing device on the user computing device, or entirely on the remote computing device or server. Execute on. In the case of a remote computing device, the remote computing device can be connected to the user computing device via any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computing device (eg, provided using an Internet service) Businesses are connected via the Internet).

Further, the above-described drawings are merely illustrative of the processes included in the method according to the exemplary embodiments of the present invention, and are not intended to be limiting. It is easy to understand that the processing shown in the above figures does not indicate or limit the chronological order of these processes. In addition, it is also easy to understand that these processes may be performed synchronously or asynchronously, for example, in a plurality of modules.

Other embodiments of the present disclosure will be apparent to those skilled in the <RTIgt; The present application is intended to cover any variations, uses, or adaptations of the present disclosure, which are in accordance with the general principles of the disclosure and include common general knowledge or common technical means in the art that are not disclosed in the present disclosure. . The specification and examples are to be regarded as illustrative only,

It is to be understood that the invention is not limited to the details of the details and The scope of the disclosure is to be limited only by the appended claims.

Industrial applicability

On the one hand, comparing the voice instruction information with the user behavior characteristics, it is possible to accurately determine whether the user behavior is consistent with the voice instruction information, thereby achieving accurate inference of attention; on the other hand, the duration of the user behavior characteristics in a specified time period The analysis realizes the method of scoring the corresponding proportion of the user behavior characteristics, so that the statistical result of the user attention degree is more accurate.

Claims

A method for detecting attention based on behavioral feature comparison, characterized in that the method comprises:

The video information obtaining step is to obtain video information collected by the video collection device, where the video information includes user behavior information and voice instruction information;

a behavior feature recognition step of identifying voice instruction information, and analyzing user behavior characteristics in the user behavior information within a specified time period after the voice instruction information is recognized;

The behavior feature comparison step searches for a standard behavior feature corresponding to the current voice instruction information in the pre-established standard behavior feature model, and compares the user behavior feature with the standard behavior feature;

The attention degree scoring step scores the user behavior characteristics that are inconsistent with the standard behavior characteristics according to the pre-designed sub-criteria, and collects the scoring results in the entire time period of the video information, and generates the teaching attention of the video information according to the scoring results. degree.
The method of claim 1 wherein said behavioral feature identification step comprises:

After analyzing the user behavior feature in the user behavior information in the specified time period, the same user behavior feature number in the specified time period is greater than the preset number of user behavior features as a standard behavior feature to be referenced;

The method further includes:

The pre-established standard behavior feature model is trained according to the standard behavior characteristics to be referenced.
The method of claim 1 wherein comparing said user behavior characteristics to said standard behavior characteristics comprises:

Comparing whether the user behavior information in the video information includes a standard behavior characteristic corresponding to the voice instruction information within a specified time period after the voice instruction information is recognized, and if yes, determining whether the retention duration of the user behavior feature satisfies the standard The preset time range for behavioral characteristics.
The method of claim 3, wherein the method further comprises a scoring criteria generating step, the scoring criteria generating step comprising:

If the duration of the user behavior feature satisfies the preset time range of the standard behavior feature, a full score is calculated for the user behavior characteristic of the corresponding user in the specified time period.
The method of claim 3, wherein the method further comprises a scoring criteria generating step, the scoring criteria generating step comprising:

If the user behavior information in the video information does not include the standard behavior feature corresponding to the voice instruction information within a specified time period after the voice instruction information is recognized, the user behavior characteristic of the corresponding user in the specified time period is zero.
The method of claim 1, wherein the user behavior information comprises a user's head rotation behavior and an eye focus behavior; and the behavior feature recognition step comprises:

Extracting a head moving direction, a moving angle, and an eye focusing state in a user's eye focusing behavior in a user's head rotation behavior;

The head moving direction, the moving angle, and the eye focusing state in the user's eye focusing behavior are taken as user behavior characteristics.
The method according to claim 6, wherein the eye focus behavior comprises closed eye information of the user's eyes, and the behavior feature identifying step further comprises:

The user's eyes are closed according to the closed eye information of the user's eyes.
The method of claim 7 wherein said method further comprises a scoring criteria generating step, said scoring criteria generating step comprising:

If the user's eye-closing behavior is determined to be greater than the preset duration according to the user's eye focus behavior, the user behavior characteristic of the corresponding user in the specified time period is zero.
The method of claim 1 wherein said user behavior acquisition step comprises:

Obtaining video information collected by the same video collection device and containing user behavior information of multiple users; or

Acquiring video information collected by one or more video collection devices that includes user behavior information of multiple users.
The method of claim 9 wherein the method further comprises:

After collecting user behavior information of multiple users, the user behavior characteristics corresponding to each user are respectively scored, and the teaching attention degree corresponding to each user is generated according to each user behavior characteristic score;

The teaching attention of each user is sorted, and the attention order is sent to the specified object.
A attention degree detecting device based on behavior feature comparison, characterized in that the device comprises:

a video information acquiring module, configured to acquire video information collected by the video collection device, where the video information includes user behavior information and voice instruction information;

a behavior feature recognition module, configured to identify voice instruction information, and analyze user behavior characteristics in the user behavior information within a specified time period after the voice instruction information is recognized;

a behavior feature comparison module, configured to search for a standard behavior feature corresponding to current voice instruction information in a pre-established standard behavior feature model, and compare the user behavior feature with the standard behavior feature;

a degree of interest scoring module, configured to score a user behavior feature that is inconsistent with a standard behavior feature according to a pre-designed sub-criteria, and collect a score result in the entire time period of the video information, and generate the video information according to the score result. Teaching attention.
An electronic device, comprising:

Processor;

A memory having computer readable instructions stored thereon, the computer readable instructions being executed by the processor to implement the method of any one of claims 1 to 9.
A computer readable storage medium having stored thereon a computer program, the computer program being executed by a processor to implement the method of any one of claims 1 to 9.