PDA equipment
The application is a divisional application of a patent with the application number of 2017111912546 and the application date of 2017, 11 and 24, and the invention creates a name of a real-time user watching content analysis method.
Technical Field
The invention relates to the field of intelligent analysis, in particular to a real-time analysis method for user watching content. Background art pda (personal digital assistant), also known as palm top computer, can help us to work, study, entertain, etc. while on the move. Classified by use, into industrial-grade PDAs and consumer-product PDAs. The industrial-grade PDA is mainly applied to the industrial field, and a common bar code scanner, an RFID reader-writer, a POS machine and the like can be called as the PDA; consumer PDAs include many smart phones, tablet computers, handheld game consoles, and the like.
Pda (personal digital assistant) means a personal digital assistant. The name is a digital tool for assisting the personal work, and mainly provides the functions of keeping a record of events, an address book, business card exchange, scheduling and the like.
Currently, due to the diversity and richness of functions of the PDA device, a user often uses the PDA device for a long time without knowing himself, and in the prior art, the detection of the user's fatigue degree is roughly calculated based on time, and it is not considered that the cumulative effects of different display contents on the user's fatigue degree are different, so that the judgment of the user's fatigue degree is not accurate.
Disclosure of Invention
In order to solve the above problems, the present invention provides a real-time analysis method for user viewing content, which is capable of starting an ultra-clear image capture device when receiving a start detection signal, and receiving a content type in a display interface of a PDA device of a pupil response output from a correlation recognition device or a non-correlation recognition device, wherein the ultra-clear image capture device captures a plurality of front images of a user based on a time sequence, and receives the content type based on the time sequence using a user status analysis device, and the user status analysis device determines a user fatigue degree based on respective durations of various received content types between a current time and a time when the start detection signal is received.
The present invention has at least the following important points:
(1) establishing an image identification equipment selection mechanism based on image resolution numerical values, so that images with various resolutions can be correspondingly and efficiently identified;
(2) the image after the identification processing is subjected to morphological processing based on controllable strength of object complexity in the image, so that the quality of the image for subsequent processing is improved;
(3) and a detection mechanism for determining the fatigue degree of the user based on the respective duration of various content types received from the current moment to the moment of receiving the start detection signal is established, so that the detection precision of the fatigue degree of the user is improved.
According to an aspect of the present invention, there is provided a method for analyzing user viewing content in real time, the method including:
the PDA equipment comprises a use detection starting device, a starting detection device and a shutdown detection device, wherein the use detection starting device is arranged on an integrated circuit board of the PDA equipment and used for sending a starting detection signal when receiving a starting signal of the PDA equipment and sending a shutdown detection signal when receiving a shutdown signal of the PDA equipment;
the method comprises the steps that an ultra-clear camera device is used, is arranged in a front frame of the PDA device and is used for carrying out ultra-clear data real-time acquisition on the front of a user of the PDA device so as to obtain and output a front image of the user;
the pupil identification equipment is arranged on an integrated circuit board of the PDA equipment, is connected with the ultra-clear camera equipment, and is used for receiving the front image of the user and extracting a pupil subimage of the user from the front image of the user based on a preset reference pupil pattern;
and the using data analysis equipment is arranged on an integrated circuit board of the PDA equipment, is connected with the pupil identification equipment, and is used for receiving the user pupil subimage and determining and outputting the real-time resolution of the user pupil subimage.
Preferably, the method further comprises the following steps:
the method comprises the steps that corrosion expansion processing equipment is used, arranged on an integrated circuit board of PDA equipment, connected with the data analysis equipment and used for receiving the user pupil subimages, determining the strength of performing corrosion processing and expansion processing on the user pupil subimages on the basis of the object complexity in the user pupil subimages, and performing corrosion processing and expansion processing on the user pupil subimages to output a morphological image, wherein the higher the object complexity in the user pupil subimages is, the higher the strength of performing corrosion processing and expansion processing is;
using a correlation identification device, arranged on an integrated circuit board of the PDA device, connected with the corrosion expansion processing device, and used for receiving the morphological image, and starting correlation identification processing on the morphological image when the real-time resolution exceeds the limit, wherein the specific correlation identification processing is as follows: determining the number of hidden layers of a deep neural network for executing target recognition based on the real-time resolution, wherein the higher the real-time resolution is, the more the number of determined hidden layers is;
the deep neural network comprises an input layer, hidden layers and an output layer, wherein the input layer inputs morphological images, the hidden layers are one or more and are used for carrying out feature abstraction on the morphological images input by the input layer by layer, and the output layer is connected with the last hidden layer and is used for outputting the result of the feature abstraction of the last hidden layer;
using a non-correlation identification device, arranged on an integrated circuit board of the PDA device, connected with the corrosion expansion processing device, and used for receiving the morphological image, and starting the non-correlation identification processing of the morphological image when the real-time resolution is not over limit, wherein the non-specific correlation identification processing is as follows: the number of hidden layers of a deep neural network for executing target recognition is fixed, wherein the deep neural network consists of an input layer, hidden layers and an output layer, the input layer inputs morphological images, the number of the hidden layers is fixed, the hidden layers are used for performing feature abstraction on the morphological images input by the input layer by layer, and the output layer is connected with the last hidden layer and is used for outputting the result of performing feature abstraction on the last hidden layer;
the output layer of the deep neural network of the correlation identification device and the non-correlation identification device outputs the content type in the display interface of the PDA device which is pupil response;
the user state analysis device is arranged on an integrated circuit board of the PDA device, is respectively connected with the detection starting device, the correlation identification device and the non-correlation identification device, and is used for starting the ultra-clear camera device when receiving the starting detection signal and receiving content types in a display interface of the PDA device of pupil response output by the correlation identification device or the non-correlation identification device, wherein the ultra-clear camera device shoots a plurality of front images of the user on the basis of a time sequence, the user state analysis device receives the content types on the basis of the time sequence, and the user state analysis device determines the fatigue degree of the user on the basis of the respective continuous time of various received content types between the current time and the time of receiving the starting detection signal.
Preferably, when receiving the closing detection signal, the user state analysis device closes the super-definition imaging device, and performs a zero clearing operation on the user fatigue degree.
Preferably, the erosion expansion processing device comprises an image receiving unit, an object analysis unit, an erosion processing unit and an expansion processing unit.
Preferably, in the erosion expansion processing device, the image receiving unit is connected to the data analysis device and configured to receive the user pupil sub-image, the object analysis unit is connected to the image receiving unit and the erosion processing unit respectively and configured to determine the object complexity in the user pupil sub-image, the erosion processing unit is further connected to the expansion processing unit, and the erosion processing unit and the expansion processing unit are configured to perform erosion processing before expansion processing on the user pupil sub-image.
Drawings
Embodiments of the invention will now be described with reference to the accompanying drawings, in which:
fig. 1 is a schematic interface diagram of a user status analysis device of a user-viewed content real-time analysis system according to an embodiment of the present invention.
Fig. 2 is a block diagram illustrating a structure of a user-viewed content real-time analysis system according to an embodiment of the present invention.
Fig. 3 is a flowchart illustrating steps of a method for analyzing content viewed by a user in real time according to an embodiment of the present invention.
Detailed Description
An embodiment of a user viewing content real-time analysis method of the present invention will be described in detail below with reference to the accompanying drawings.
The prior art PDA user eye fatigue determination lacks a viewing content based recognition mechanism. In order to overcome the defects, the invention builds a system and a method for analyzing the content watched by the user in real time, and the specific implementation scheme is as follows.
Fig. 1 is a schematic interface diagram of a user status analysis device of a user-viewed content real-time analysis system according to an embodiment of the present invention. The user state analysis device is implemented by using a control chip as shown in fig. 1.
Fig. 2 is a block diagram showing a structure of a user-viewed content real-time analysis system according to an embodiment of the present invention, the system including:
the PDA device comprises a detection starting device, a detection starting device and a control device, wherein the detection starting device is arranged on an integrated circuit board of the PDA device and used for sending a starting detection signal when receiving a starting signal of the PDA device and sending a closing detection signal when receiving a closing signal of the PDA device;
the super-definition camera equipment is arranged in the front frame of the PDA equipment and is used for carrying out real-time super-definition data acquisition on the front of a user of the PDA equipment so as to obtain and output a front image of the user;
the pupil identification device is arranged on an integrated circuit board of the PDA device, is connected with the ultra-clear camera device, and is used for receiving the front image of the user and extracting pupil subimages of the user from the front image of the user based on a preset reference pupil pattern;
and the data analysis equipment is arranged on an integrated circuit board of the PDA equipment, is connected with the pupil identification equipment, and is used for receiving the user pupil subimage and determining and outputting the real-time resolution of the user pupil subimage.
Next, the detailed structure of the user-viewed content real-time analysis system of the present invention will be further described.
The real-time analysis system for the user-viewed content may further include:
the erosion expansion processing device is connected with the data analysis device and used for receiving the user pupil subimage, determining the strength of erosion processing and expansion processing on the user pupil subimage based on the object complexity in the user pupil subimage, and performing erosion processing and expansion processing on the user pupil subimage to output a morphological image, wherein the higher the object complexity in the user pupil subimage is, the higher the strength of erosion processing and expansion processing is;
and the correlation identification device is connected with the erosion and expansion processing device and used for receiving the morphological image, and when the real-time resolution exceeds the limit, the correlation identification processing of the morphological image is started, wherein the specific correlation identification processing is as follows: determining the number of hidden layers of a deep neural network for executing target recognition based on the real-time resolution, wherein the higher the real-time resolution is, the more the number of the determined hidden layers is, the deep neural network consists of an input layer, hidden layers and an output layer, the input layer inputs morphological images, the hidden layers are one or more, the hidden layers are used for performing feature abstraction on the morphological images input by the input layer by layer, and the output layer is connected with the last hidden layer and is used for outputting the result of performing feature abstraction on the last hidden layer;
and the non-correlation identification device is connected with the erosion and expansion processing device and used for receiving the morphological image, and starting the non-correlation identification processing of the morphological image when the real-time resolution is not over the limit, wherein the non-specific correlation identification processing comprises the following steps: the number of hidden layers of a deep neural network for executing target recognition is fixed, wherein the deep neural network consists of an input layer, hidden layers and an output layer, the input layer inputs morphological images, the number of the hidden layers is fixed, the hidden layers are used for performing feature abstraction on the morphological images input by the input layer by layer, and the output layer is connected with the last hidden layer and is used for outputting the result of performing feature abstraction on the last hidden layer;
the output layer of the deep neural network of the correlation identification device and the non-correlation identification device outputs the content type in the display interface of the PDA device which is pupil response;
and the user state analysis device is respectively connected with the detection starting device, the correlation identification device and the non-correlation identification device, and is used for starting the ultra-clear camera device when receiving the starting detection signal and receiving the content type in the display interface of the PDA device of the pupil response output by the correlation identification device or the non-correlation identification device, wherein the ultra-clear camera device shoots a plurality of front images of the user on the basis of a time sequence, the user state analysis device receives the content type on the basis of the time sequence, and the user state analysis device determines the fatigue degree of the user on the basis of the time for which the various content types received between the current moment and the moment of receiving the starting detection signal respectively last.
In the user viewing content real-time analysis system:
and when the user state analysis equipment receives the closing detection signal, closing the super-clear camera equipment and carrying out zero clearing operation on the fatigue degree of the user.
In the user viewing content real-time analysis system:
the corrosion expansion processing equipment comprises an image receiving unit, an object analysis unit, a corrosion processing unit and an expansion processing unit.
And in the user viewing content real-time analysis system:
in the erosion expansion processing device, the image receiving unit is connected to the data analysis device and configured to receive the user pupil sub-image, the object analysis unit is respectively connected to the image receiving unit and the erosion processing unit and configured to determine object complexity in the user pupil sub-image, the erosion processing unit is further connected to the expansion processing unit, and the erosion processing unit and the expansion processing unit are configured to perform erosion processing before expansion processing on the user pupil sub-image.
Fig. 3 is a flowchart illustrating steps of a method for analyzing content viewed by a user in real time according to an embodiment of the present invention, the method including:
the PDA equipment comprises a use detection starting device, a starting detection device and a shutdown detection device, wherein the use detection starting device is arranged on an integrated circuit board of the PDA equipment and used for sending a starting detection signal when receiving a starting signal of the PDA equipment and sending a shutdown detection signal when receiving a shutdown signal of the PDA equipment;
the method comprises the steps that an ultra-clear camera device is used, is arranged in a front frame of the PDA device and is used for carrying out ultra-clear data real-time acquisition on the front of a user of the PDA device so as to obtain and output a front image of the user;
the pupil identification equipment is arranged on an integrated circuit board of the PDA equipment, is connected with the ultra-clear camera equipment, and is used for receiving the front image of the user and extracting a pupil subimage of the user from the front image of the user based on a preset reference pupil pattern;
and the using data analysis equipment is arranged on an integrated circuit board of the PDA equipment, is connected with the pupil identification equipment, and is used for receiving the user pupil subimage and determining and outputting the real-time resolution of the user pupil subimage.
Next, the specific steps of the user viewing content real-time analysis method of the present invention will be further described.
The real-time user viewing content analysis method may further include:
the method comprises the steps that corrosion expansion processing equipment is used, arranged on an integrated circuit board of PDA equipment, connected with the data analysis equipment and used for receiving the user pupil subimages, determining the strength of performing corrosion processing and expansion processing on the user pupil subimages on the basis of the object complexity in the user pupil subimages, and performing corrosion processing and expansion processing on the user pupil subimages to output a morphological image, wherein the higher the object complexity in the user pupil subimages is, the higher the strength of performing corrosion processing and expansion processing is;
using a correlation identification device, arranged on an integrated circuit board of the PDA device, connected with the corrosion expansion processing device, and used for receiving the morphological image, and starting correlation identification processing on the morphological image when the real-time resolution exceeds the limit, wherein the specific correlation identification processing is as follows: determining the number of hidden layers of a deep neural network for executing target recognition based on the real-time resolution, wherein the higher the real-time resolution is, the more the number of the determined hidden layers is, the deep neural network consists of an input layer, hidden layers and an output layer, the input layer inputs morphological images, the hidden layers are one or more, the hidden layers are used for performing feature abstraction on the morphological images input by the input layer by layer, and the output layer is connected with the last hidden layer and is used for outputting the result of performing feature abstraction on the last hidden layer;
using a non-correlation identification device, arranged on an integrated circuit board of the PDA device, connected with the corrosion expansion processing device, and used for receiving the morphological image, and starting the non-correlation identification processing of the morphological image when the real-time resolution is not over limit, wherein the non-specific correlation identification processing is as follows: the number of hidden layers of a deep neural network for executing target recognition is fixed, wherein the deep neural network consists of an input layer, hidden layers and an output layer, the input layer inputs morphological images, the number of the hidden layers is fixed, the hidden layers are used for performing feature abstraction on the morphological images input by the input layer by layer, and the output layer is connected with the last hidden layer and is used for outputting the result of performing feature abstraction on the last hidden layer;
the output layer of the deep neural network of the correlation identification device and the non-correlation identification device outputs the content type in the display interface of the PDA device which is pupil response;
the user state analysis device is arranged on an integrated circuit board of the PDA device, is respectively connected with the detection starting device, the correlation identification device and the non-correlation identification device, and is used for starting the ultra-clear camera device when receiving the starting detection signal and receiving content types in a display interface of the PDA device of pupil response output by the correlation identification device or the non-correlation identification device, wherein the ultra-clear camera device shoots a plurality of front images of the user on the basis of a time sequence, the user state analysis device receives the content types on the basis of the time sequence, and the user state analysis device determines the fatigue degree of the user on the basis of the respective continuous time of various received content types between the current time and the time of receiving the starting detection signal.
The user watching content real-time analysis method comprises the following steps:
and when the user state analysis equipment receives the closing detection signal, closing the super-clear camera equipment and carrying out zero clearing operation on the fatigue degree of the user.
The user watching content real-time analysis method comprises the following steps:
the corrosion expansion processing equipment comprises an image receiving unit, an object analysis unit, a corrosion processing unit and an expansion processing unit.
The user watching content real-time analysis method comprises the following steps:
in the erosion expansion processing device, the image receiving unit is connected to the data analysis device and configured to receive the user pupil sub-image, the object analysis unit is respectively connected to the image receiving unit and the erosion processing unit and configured to determine object complexity in the user pupil sub-image, the erosion processing unit is further connected to the expansion processing unit, and the erosion processing unit and the expansion processing unit are configured to perform erosion processing before expansion processing on the user pupil sub-image.
In addition, the PDA, i.e., the palm computer, is most characterized by having an open operating system, supporting software and hardware upgrade, integrating information input, storage, management and transmission, and having powerful functions such as common office, entertainment, mobile communication, etc. Thus, the PDA can be referred to as a mobile office at all. Of course, not all of the above functions are available to any PDA; if available, it may not be possible to do so due to lack of corresponding service. However, it is anticipated that the trend and trend of PDA development is the convergence of computing, communication, networking, storage, entertainment, e-commerce, and other functionalities.
There is no definite definition for the palm computer, and someone also refers to the notebook as a palm computer, generally, the palm computer is a palm os, windows ce or other open operating systems, has network functions, and can freely upgrade software and hardware by users. That is, the user can add software besides expanding hardware, and even can develop a program by himself to run on the software.
In the use, he is more simple than desktop computer easy operation, remove convenient, and the function is practical, has eliminated desktop computer's five big restrictions, promptly: mobility restrictions, complexity of use, difficulty of mobile networking, high price, idleness of use.
By adopting the real-time analysis system and method for the user watching content, aiming at the technical problem that a fatigue degree judgment mechanism based on the watching content is lacked in the prior art, the front side of the user of the PDA device is subjected to super-clear data real-time acquisition to obtain and output a front side image of the user, the content type in the display interface of the PDA device watched by the user is identified based on a deep neural network, and the fatigue degree of the user is judged by combining the watching time, so that the technical problem is solved.
It is to be understood that while the present invention has been described in conjunction with the preferred embodiments thereof, it is not intended to limit the invention to those embodiments. It will be apparent to those skilled in the art from this disclosure that many changes and modifications can be made, or equivalents modified, in the embodiments of the invention without departing from the scope of the invention. Therefore, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention are still within the scope of the protection of the technical solution of the present invention, unless the contents of the technical solution of the present invention are departed.