TWM630474U - Concentration degree recognition device - Google Patents

Abstract

A concentration recognition device, comprising a user terminal device, an image capture device electrically connected to the user terminal device and located in front of the user terminal device, and a concentration map electrically connected to the image capture device It is composed of an image capture comparator and a reforming device electrically connected to the dedicated image capture comparator. Therefore, the present invention proposes to build the dedicated image capture comparator in the existing image capture device, or use the image capture device to capture the image first, and then use the built-in image capture device to capture the image. The attentive image capture comparator in the reforming device uses a deep learning algorithm to evaluate the state of attentiveness, and directly combines the relevant information generated by the calculation with the image, so that the user's attentiveness can be tracked in real time. , without the additional cost of installing dedicated detection software.

Description

Concentration recognition device

This work is about a concentration recognition device, especially a A dedicated image capturing comparator is built in the image capturing device, or the image capturing device is used to capture the image first, and then the dedicated image capturing comparator built in the reforming device is used to capture images. The deep learning algorithm is used to evaluate the state of concentration. In particular, it means that the relevant information generated by the algorithm is directly combined with the image, so that the user's concentration can be tracked in real time, without the additional cost of installing special detection software. By.

Due to the impact of the epidemic, many courses have become remote teaching, and many meetings have also become video Therefore, the practice of providing education for remote teaching and classes by video, or providing remote work and international conferences for enterprises has become one of the main ways for modern people to learn new knowledge and work remotely.

Although e-learning systems or video conferencing systems allow all students or attendees to Breaking through the limitations of space to take classes or meetings together, but there are still many shortcomings in the traditional online learning system or video conference system, which need to be improved urgently.

For example, children often experience difficulty concentrating or The teaching content is too difficult and frustrating and distracting. The existing online learning system cannot grasp the students' listening status, and parents cannot always accompany them to read. It is impossible to quantitatively evaluate whether students are actually in class, whether they are listening carefully to the class, how well the online course interacts, and so on. Therefore, it is impossible to judge whether students are concentrating in class. Similarly, the existing video conference system also has the same problem of not being able to know whether the participants are concentrating on the meeting.

Given that existing online learning systems or videoconferencing systems cannot Students or participants are attentive, and when and under what circumstances will they be more attentive to participate in the course or meeting. For the reason of the job, in order to prevent students or participants from being inattentive in class or meeting, which leads to the problem of poor study or work efficiency. Therefore, it is necessary to develop a set of inventions that can effectively grasp the study or work status of students or participants during online study or video conference.

The main purpose of this creation is to overcome the above-mentioned problems encountered by conventional techniques and Provide a built-in focus image capture comparator in an existing image capture device, or first use the image capture device to capture images, and then pass the focus image built in the reformer device Like the capture comparator, the deep learning algorithm is used to evaluate the concentration state, and the relevant information generated by the algorithm is directly combined with the image, so that the user's concentration level can be tracked in real time, without the need for additional special detectors. Concentration recognition device for measuring software cost.

In order to achieve the above purpose, this creation is a concentration recognition device, which includes: 1. a user terminal device for displaying a video image for the user to watch; an image capture device, which is electrically connected to the user terminal device and located in front of the user terminal device, used to capture the user's several images when watching the video frame; a dedicated image capture comparator is electrically connected to the image capture device for receiving the several images captured by the image capture device, and compares each of the images Comparing the concentration behavior with a user state model, and generating a concentration state data according to the comparison results of the images; and a reforming device electrically connected to the concentration image capturing comparator for receiving and The plurality of concentration status data are integrated to generate a concentration report on the user.

In the above-mentioned embodiment of the present invention, the user terminal device is a personal computer, a flat A tablet computer, or a network device that can be used in video conferencing.

In the above-mentioned embodiment of this invention, the network device is a video phone or an Internet TV. either.

In the above-mentioned embodiment of the present invention, the image capturing device and the user terminal device are integrated in the same device.

In the above-mentioned embodiment of the present invention, the dedicated image capture comparator is built in the image as in the capture device.

In the above-mentioned embodiment of the present invention, the dedicated image capture comparator is built in the in the whole device.

In the above-mentioned embodiment of the present invention, the dedicated image capture comparator includes an operation unit. element and a database unit, the computing unit obtains the eye gaze direction data of the user according to the eye image information in the images when the user watches the video frame, and stores the data in the database unit The user state model compares attentive behaviors to generate the plurality of attentive state data.

In the above-mentioned embodiment of the present creation, the attentive behavior includes the user in the video picture An eye movement path on the surface, an eye gaze position of the user on the video screen, a gaze time of the user at the eye gaze position, an eye opening and closing degree of the user, a person of the user At least one of the face deflection angle and a hand movement of the user.

In the above-mentioned embodiment of the present invention, the reforming device includes an input unit, a processing unit and an output unit, the input unit is used for receiving the plurality of concentration state data, the processing unit integrates the plurality of concentration state data and generates the concentration level report, and outputs the concentration level report through the output unit.

In the above-mentioned embodiment of the present invention, the input unit and the output unit at least include High-definition multimedia interface connector, serial digital interface connector, video graphics array connector, and digital video interface connector.

Please refer to "Picture 1 and Picture 2", which are respectively for the identification of the concentration of this creation. A schematic diagram of the structure of the first embodiment of the device, and a schematic diagram of the structure of the second embodiment of the creative concentration recognition device. As shown in the figure: the invention is a concentration recognition device, which includes a user terminal device 1 , an image capturing device 2 , a concentration image capturing comparator 3 , and a reforming device 4 .

The user terminal device 1 mentioned above can be a personal computer, or can be applied to a video network equipment in a conference. Wherein, the network device can be either a video phone or an Internet TV.

The image capture device 2 is electrically connected to the user terminal device 1, and is located in the the front of the user terminal device 1 . The image capturing device 2 has a main body 21 and an optical lens 22 , and the optical lens 22 is disposed on the main body 21 . Wherein, the image capturing device 2 and the user terminal device 1 can be integrated in the same device, such as a tablet computer or a smart phone.

The dedicated image capture comparator 3 is built in the image capture device 2 and powered Sexually connect the image capture device 2 . The dedicated image capture comparator 3 includes an operation unit 31 and a database unit 32 , and the operation unit 31 is electrically connected to the database unit 32 .

The reforming device 4 is electrically connected to the dedicated image capturing comparator 3 . the refit Device 4 includes an input unit 41, a processing unit 42 and an output unit 43, the processing unit 42 is electrically connected to the input unit 41 and the output unit 43 . Wherein, the input unit 41 and the output unit 43 at least include a high-definition multimedia interface connector, a serial digital interface connector, a video graphics array connector, and a digital video interface connector. In this case, a brand-new attentiveness recognition device is constructed by the device disclosed above.

When in use, the user can watch a video picture through the user terminal device 1 On the other hand, the video screen can be content such as a live online course or a conference. The image capture device 2 captures several images of the user while watching the video frame through the optical lens 22 , and the dedicated image capture comparator 3 receives the several images, and then uses the computing unit 31 to receive the images. According to the eye image information in the images when the user watches the video frame, the eye gaze direction data of the user is obtained, and the data is compared with a user state model stored in the database unit 32 . Concentration behavior comparison, according to the comparison result of each image, a concentration state data is respectively generated. Wherein, the attentive behavior includes an eyeball movement path of the user on the video screen, a gaze position of the user's eyeball on the video screen, a gaze time of the user at the eyeball gaze position, the user At least one of an eye opening and closing degree, a face deflection angle of the user, and a hand movement of the user.

The reforming device 4 receives the plurality of concentration state numbers through the input unit 41 According to the data, the processing unit 42 integrates the plurality of concentration state data and generates a concentration report about the user, and finally outputs the concentration report through the output unit 43 . For example, the concentration report shows that you are only concentrating for a period of time, but you are not concentrating later, it may be that the teaching is not effective; or it may show that you are not concentrating for a while, and it may be something happened in the middle, such as class, class and so on. In this way, by building the attentive image capturing comparator 3 in the existing image capturing device 2, the deep learning algorithm can be directly applied to the captured image to evaluate the attentive state, so that real-time tracking can be achieved. The degree of concentration of users (such as students or office workers) when studying or working without the additional cost of installing special detection software.

In addition to the structure type mentioned in the above-mentioned first embodiment, this creation can also be the second embodiment. The structural type proposed in the embodiment is different in that the dedicated image capturing comparator 3 is built in the reforming device 4, as shown in FIG. 2 . In this way, by building the attentive image capture comparator 3 in the existing reforming device 4, the deep learning algorithm can be applied to the image captured by the image capture device 2 to evaluate the attentive state. It can instantly track the concentration of users (such as students or office workers) when they are studying or working, without the additional cost of installing special detection software.

To sum up, this creation is a concentration recognition device, which can effectively improve the habitual Due to various shortcomings, it is proposed to build a dedicated image capture comparator in the existing image capture device, or first use the image capture device to capture the image, and then use the image capture device to build in the reformer device. The attentive image capture comparator uses a deep learning algorithm to evaluate the state of attentiveness, and directly combines the relevant information generated by the calculation with the image, so that the user's attentiveness can be tracked in real time without any additional The cost of installing special detection software, so that the creation of this creation can be more advanced, more practical, and more in line with the needs of users, it has indeed met the requirements for a new patent application, and a patent application can be filed in accordance with the law.

However, the above are only the preferred embodiments of this creation, and should not be limited to this The scope of implementation of this creation is determined; therefore, any application based on the scope of the patent application for this creation and the contents of the new description Simple equivalent changes and modifications should still fall within the scope of the invention patent.

1: User terminal device 2: Image capture device 21: Main body 22: Optical lens 3: Concentrate on the image capture comparator 31: Operation unit 32: Database unit 4: Reforming device 41: Input unit 42: Processing unit 43: Output unit

Figure 1 is a schematic diagram of the structure of the first embodiment of the creative concentration recognition device. Figure 2 is a schematic diagram of the structure of the second embodiment of the creative concentration recognition device.

1: User terminal device

2: Image capture device

21: Ontology

22: Optical lens

3: Concentrate on the image capture comparator

31: Operation unit

32: Library Unit

4: Reforming device

41: Input unit

42: Processing unit

43: Output unit

Claims (10)

A concentration recognition device, comprising: a user terminal device for displaying a video image for the user to watch; an image capturing device, electrically connected to the user terminal device and located in front of the user terminal device, for capturing several images of the user watching the video frame; An attentive image capture comparator is electrically connected to the image capture device for receiving a plurality of images captured by the image capture device, and compares each of the images with a user state model for attentive behavior Yes, generating a concentration state data according to the comparison result of each of the images; and A reforming device is electrically connected to the attentive image capturing comparator for receiving and integrating the plurality of attentive state data to generate a concentration report about the user. According to the attentiveness recognition device described in item 1 of the scope of the patent application, the user terminal device is a personal computer, or a network device that can be used in a video conference. According to the attentiveness recognition device described in item 2 of the claimed scope, the network device is either a video phone or an Internet TV. The device for recognizing concentration according to claim 1, wherein the image capturing device and the user terminal device are integrated into the same device. The attentiveness recognition device according to claim 1 of the claimed scope, wherein the attentive image capturing comparator is built in the image capturing device. According to the attentiveness recognition device described in claim 1 of the claimed scope, the attentive image capturing comparator is built in the reforming device. The attentiveness recognition device according to claim 1, wherein the attentive image capture comparator includes an arithmetic unit and a database unit, and the arithmetic unit is based on the data when the user watches the video frame. From the eye image information in the image, the eye gaze direction data of the user is obtained, and the concentration behavior is compared with the user state model stored in the database unit to generate the plurality of concentration state data. The attentiveness recognition device according to claim 1 or 7, wherein the attentive behavior includes an eye movement path of the user on the video frame, and a gaze position of the user's eyeball on the video frame , at least one of a gaze time of the user at the eyeball gaze position, an eye opening and closing degree of the user, a face deflection angle of the user, and a hand movement of the user. According to the concentration recognition device described in claim 1, the reforming device includes an input unit, a processing unit and an output unit, the input unit is used for receiving the plurality of concentration state data, the processing unit Integrate the plurality of concentration state data to generate the concentration report, and output the concentration report through the output unit. The concentration recognition device according to item 9 of the scope of the application, wherein the input unit and the output unit at least include a high-definition multimedia interface connector, a serial digital interface connector, a video graphics array connector, and a digital Video interface connector.

Images