CN114253386A - Communication system based on perception - Google Patents
Communication system based on perception Download PDFInfo
- Publication number
- CN114253386A CN114253386A CN202010951549.4A CN202010951549A CN114253386A CN 114253386 A CN114253386 A CN 114253386A CN 202010951549 A CN202010951549 A CN 202010951549A CN 114253386 A CN114253386 A CN 114253386A
- Authority
- CN
- China
- Prior art keywords
- user
- perception
- information
- module
- equipment
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L51/00—User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
- H04L51/04—Real-time or near real-time messaging, e.g. instant messaging [IM]
- H04L51/046—Interoperability with other network applications or services
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/01—Protocols
- H04L67/10—Protocols in which an application is distributed across nodes in the network
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Human Computer Interaction (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Medical Informatics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Artificial Intelligence (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention discloses a communication system based on perception, which comprises a perception module, a functional module and a functional control module, wherein the perception module is used for detecting the perception of a communication terminal; the perception module is used for acquiring information contained in user behaviors and transmitting the information to the function control module; the function control module is used for processing information transmitted by the sensing module and calling corresponding function units and program interfaces in the corresponding function modules according to the information, the function control module comprises function control logic and an information processing model, and after the information is input into the processing model and user intention information is obtained, the communication and equipment interaction mode close to a natural mode is provided, communication applications and communication equipment are enriched, and communication, communication and interaction are carried out by a user in more natural modes, so that communication efficiency among users is improved, the disturbance of the equipment can be avoided, and user experience is improved.
Description
Technical Field
The invention relates to the technical field of communication equipment, in particular to a communication system based on perception.
Background
With the application of sensor technology, more and more sensors can sense the external environment; the sensor can not only sense environmental characteristics, but also can sense human characteristics and biological characteristics, and developers can develop applications such as action and behavior recognition, environment self-adaptation, user authentication and the like based on the sensor, thereby greatly enriching the interaction modes between users and equipment.
The existing communication modes mainly comprise instant communication and non-instant communication. Instant messaging refers to a communication mode in which two communication parties carry out information transfer in real time, and the two communication parties need to participate simultaneously, so that a user needs to transfer most attention and completely immerse the user in the communication process. Non-instant messaging requires no simultaneous participation of both parties, only a user to divert attention while receiving and sending messages, but does not deliver messages in a timely manner.
The traditional network communication between users needs all immersive participation of communication parties in communication, and the users are required to perform complex operation control communication control, such as friend adding, establishment, contact person selection, communication initiation, communication control, communication ending and the like. When the user communicates in a working scene, the work of the user is easily interrupted, the working efficiency is reduced, and the user experience is not high.
In order to enable the communication process to be more natural and efficient and reduce the disturbance of communication to users, how to use the prior art such as sensors to perfect interaction and improve the user experience also become the direction of efforts in various industries.
Disclosure of Invention
It is an object of the present invention to provide a communication system based on sensing to solve the above-mentioned problems in the background art.
In order to achieve the purpose, the invention provides the following technical scheme:
a communication system based on perception is characterized by comprising a perception module, a function module and a function control module;
the perception module is used for acquiring information contained in user behaviors and transmitting the information to the function control module;
the function control module is used for processing the information transmitted by the sensing module and calling corresponding function units and program interfaces in the corresponding function modules according to the information, the function control module comprises function control logic and an information processing model, and after the information is input into the processing model to obtain user intention information, the function control logic calls the functions and interfaces of the corresponding function modules according to the user intention;
the information processing model is composed of pattern recognition logic and a machine learning model and is used for converting information contained in user behaviors into specific intentions of a user, the model can perform transfer learning according to operation data, and the pre-training model can be updated to improve accuracy and efficiency of information data.
As a further scheme of the invention: the sensing module comprises a face and sight tracking unit, a mouth shape and audio recognition unit, a behavior recognition unit, a state recognition unit and a biological information recognition unit;
a face view and line tracking unit: interaction and control with equipment are realized by nodding, shaking or blinking eyes, and functions of identity recognition and state perception are realized by facial features;
the behavior recognition unit is used for realizing interaction and control with the equipment in a mode of shaking hands or crossing left and right hands by the body actions of the user; the concentration state of the user is judged by sensing the actions of the user such as head bending, forehead supporting and the like, so that a basis is provided for equipment control. Further realizing the interaction with the equipment, the function expansion and the control
The mouth shape and audio recognition unit is used for realizing interaction, function expansion and control with equipment by opening the mouth into a certain mouth shape and combining the sound characteristics and the voice content of a user;
and the state identification unit is used for carrying out modeling analysis on the user through a TOF technology and providing a basis for equipment control. Interaction with equipment, function expansion and control are further realized;
the biological information identification unit provides a basis for equipment control through the disturbance of body temperature, respiratory frequency and vital signs on the detection signals. Interaction and control with equipment are further realized;
the perception module can be a combination form of the information sources when acquiring the user behavior information, and can also be an information source provided by peripheral equipment expanded through an equipment interface.
As a still further scheme of the invention: the function module realizes application and function based on perception and communication facing to application scene by means of perception function, including voice communication application based on perception for voice communication, video communication application based on perception for video communication, data transmission application based on perception for data transmission, information display application based on perception for displaying information, information notification application facing to user state for reminding various matters of user, and equipment control and scheduling logic facing to user state and based on perception; the user can use the interface provided by the system to realize the expansion of the function according to the requirement.
The content of the data transmission application at least comprises a data communication service based on a TCP/IP protocol stack, and also comprises value-added services such as payment, transfer, electronic signature and live broadcast application based on the TCP/IP protocol stack, and the user can install and use different applications to expand the functions of the system.
As a still further scheme of the invention: the sensing module comprises a device servo, and after the sensing module is analyzed and processed, the sensing module judges that a user is not in a device using state, and a device control and scheduling logic in the functional module controls the device, and the carried function and service to enter a standby state. And when the user returns to the state of using the equipment, automatically awakening the equipment to enter a working state.
As a still further scheme of the invention: the perception module also comprises a user identity recognition function based on perception so as to realize the management of user authority, ensure the privacy of the user and reduce the authentication cost of the user.
As a still further scheme of the invention: the function module also comprises an interaction unit interacting with the cloud end, and the communication system can expand the functions of the equipment by means of cloud service and improve the usability and usability of the equipment.
As a still further scheme of the invention: the communication system can work as a computer peripheral, an audio and video input source, a live broadcast source and a sensing sensor peripheral, and can also work independently as a live broadcast and video recording device and an intelligent device with a sensing function.
As a still further scheme of the invention: the function module also comprises an application interface and an application store, wherein the application interface and the application store are used for developing and using the application facing the perception equipment, and enriching and expanding the function of the equipment and the coverage range of the use scene.
Compared with the prior art, the invention has the beneficial effects that: the invention separates the communication function from the user's equipment, and fixes the communication function as an independent equipment near the user interaction equipment such as a display, and controls the operation of the communication equipment by using the perception technology. Various problems of communication control are solved by sensing the user state, and based on the sensing technology, the communication function is enriched and the use scene is expanded; when a user needs to communicate, the user only needs to wake up the opposite side by facing the equipment like ordinary face-to-face communication naturally, and speaks out the intention to realize the related function; avoiding cumbersome steps. The use experience of the user is improved, the disturbance to the work is reduced, and the pain point of the communication mode is solved.
Drawings
FIG. 1 is a schematic structural diagram of the present invention.
FIG. 2 is a diagram of a state aware state machine according to the present invention.
Detailed Description
In the description of the present invention, it is to be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like, indicate orientations or positional relationships based on those shown in the drawings, and are used only for convenience in describing the present invention and for simplicity in description, and do not indicate or imply that the referenced devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and thus, are not to be construed as limiting the present invention. Furthermore, the terms "first", "second", etc. are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first," "second," etc. may explicitly or implicitly include one or more of that feature. In the description of the present invention, "a plurality" means two or more unless otherwise specified.
In the description of the present invention, it should be noted that, unless otherwise explicitly specified or limited, the terms "mounted," "connected," and "connected" are to be construed broadly, e.g., as meaning either a fixed connection, a removable connection, or an integral connection; can be mechanically or electrically connected; they may be connected directly or indirectly through intervening media, or they may be interconnected between two elements. The specific meaning of the above terms in the present invention can be understood by those of ordinary skill in the art through specific situations.
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1-2, in an embodiment of the present invention, a communication system based on sensing includes a sensing module, a function module, and a function calling module;
the sensing module is used for acquiring instruction information sent by a user and transmitting the instruction information to the function calling module;
the function control module is used for processing the information transmitted by the sensing module and calling corresponding function units and program interfaces in the corresponding function modules according to the information, the function control module comprises function control logic and an information processing model, and after the information is input into the processing model to obtain user intention information, the function control logic calls the functions and interfaces of the corresponding function modules according to the user intention;
the information processing model is composed of pattern recognition logic and a machine learning model and is used for converting information contained in user behaviors into specific intentions of a user, the model can perform transfer learning according to operation data, and the pre-training model can be updated to improve accuracy and efficiency of information data.
The sensing module comprises a face and sight tracking unit, a mouth shape and audio recognition unit, a behavior recognition unit, a state recognition unit and a biological information recognition unit, wherein a plurality of existing awakening modes are awakened through voice.
A face gaze tracking unit: the interaction and control with the equipment are realized by recognizing the behaviors of nodding, shaking or blinking eyes and the like; the behavior recognition unit is used for realizing interaction and control with the equipment through action instructions sent by a user, such as shaking hands or crossing left and right hands; the mouth shape recognition unit is used for realizing interaction and control with equipment by opening the mouth into a certain mouth shape.
The sensing module further comprises a basic information identification unit and a physical information identification unit, wherein the physical information comprises factors such as beating, pressing, light shielding, material, temperature, heat radiation and the like, so that the sensing range and accuracy are improved, and interaction and control with equipment are realized if people or other animals are identified and if the humanoid doll is a living body is identified.
The function module comprises a perception-based voice communication application for voice communication, a perception-based video communication application for video communication, a perception-based data transmission application for data transmission, a perception-based information display application for displaying information, a user state-oriented message notification application for reminding various matters of a user, and a user state-oriented device control and scheduling logic. The user can use the interface provided by the system to realize the expansion of the function according to the requirement.
The sensing module comprises an equipment servo, when the sensing module is analyzed and processed, the sensing module judges that a user is not in the state of using the equipment, and equipment control and scheduling logic in the functional module controls the device and the carried functions and services to enter a standby state. When the user returns to the state of using the equipment, the equipment is automatically awakened to enter the working state, so that the energy consumption of the equipment is reduced, and the interference to the user is reduced as much as possible.
The perception module also comprises a user identity recognition function based on perception so as to realize the management of user authority, ensure the privacy of the user and reduce the authentication cost of the user.
The function module also comprises an interaction unit interacting with the cloud end, and the communication system can expand the functions of the equipment by means of cloud service and improve the usability and usability of the equipment.
The function module also comprises an application interface and an application store, wherein the application interface and the application store are used for developing and using the application facing the perception equipment, and enriching and expanding the function of the equipment and the coverage range of the use scene.
The communication system can work as a computer peripheral, an audio and video input source, a live broadcast source and a sensing sensor peripheral. The intelligent video recorder can also independently work as live broadcast and video recording equipment and intelligent equipment with a sensing function.
The working principle is as follows: when the device is actually used, a user can interact with the device in a natural mode, such as watching the device by sight, calling by voice and the like to wake up the device, after the device senses the behavior of a user, the intention of the user is analyzed through a processing model, and the device is awakened after the awakening condition is judged to be met. Then, the device continuously senses the user intention, such as 'calling for Xiaoming' and 'notifying for Xiaoming to open a meeting', and the like, and the function calling module calls a corresponding function according to the instruction or adjusts the working state of the function according to the user intention to realize the control of the function; when the sleep condition is satisfied (if the user does not pay attention to the device after waving hands), the whole system enters a sleep standby state, so that the power consumption is reduced. Compared with the traditional immersive communication mode, the interactive mode based on perception can not influence the work at hand of the worker, improves the working efficiency and greatly improves the experience of the user.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference sign in a claim should not be construed as limiting the claim concerned.
Furthermore, it should be understood that although the present description refers to embodiments, not every embodiment may contain only a single embodiment, and such description is for clarity only, and those skilled in the art should integrate the description, and the embodiments may be combined as appropriate to form other embodiments understood by those skilled in the art.
Claims (9)
1. A communication system based on perception is characterized by comprising a perception module, a function module and a function control module;
the perception module is used for acquiring information contained in user behaviors and transmitting the information to the function control module;
the function control module is used for processing the information transmitted by the sensing module and calling corresponding function units and program interfaces in the corresponding function modules according to the information, the function control module comprises function control logic and an information processing model, and after the information is input into the processing model to obtain user intention information, the function control logic calls the functions and interfaces of the corresponding function modules according to the user intention;
the information processing model is composed of pattern recognition logic and a machine learning model and is used for converting information contained in user behaviors into specific intentions of a user, the model can perform transfer learning according to operation data, and the pre-training model can be updated to improve accuracy and efficiency of information data.
2. The perception-based communication system of claim 1, wherein the perception module includes a face and gaze tracking unit, a mouth shape and audio recognition unit, a behavior recognition unit, a state recognition unit, a biometric information recognition unit;
a face view and line tracking unit: interaction and control with equipment are realized by nodding, shaking or blinking eyes, and functions of identity recognition and state perception are realized by facial features;
the behavior recognition unit is used for realizing interaction and control with the equipment in a mode of shaking hands or crossing left and right hands by the body actions of the user; the concentration state of the user is judged by sensing the actions of the user such as scratching the head, holding the forehead and the like, so that a basis is provided for equipment control, and interaction with the equipment, function expansion and control are further realized;
the mouth shape and audio recognition unit is used for realizing interaction, function expansion and control with equipment by opening the mouth into a certain mouth shape and combining the sound characteristics and the voice content of a user;
the state recognition unit carries out modeling analysis on a user through a TOF technology, provides a basis for equipment control, and further realizes interaction, function expansion and control with the equipment;
the biological information identification unit provides a basis for equipment control through the disturbance of body temperature, respiratory frequency and vital signs on the detection signals, so that interaction and control with the equipment are realized; the perception module can be a combination form of the information sources when acquiring the user behavior information, and can also be an information source provided by peripheral equipment expanded through an equipment interface.
3. The awareness-based communication system according to claim 1, wherein the functional module implements awareness and communication-based applications and functions in an application-oriented context by means of awareness functionality, including an awareness-based voice communication application for voice communication, an awareness-based video communication application for video communication, an awareness-based data transfer application for data transmission, an awareness-based information presentation application for displaying information, a user-state-oriented messaging application for alerting a user of various items, user-state-oriented device control and scheduling logic;
the content of the data transmission application at least comprises a data communication service based on a TCP/IP protocol stack and value-added services based on the TCP/IP protocol stack, such as payment, transfer, electronic signature, live broadcast and virtual office application, a user can install and use the functions of different application expanding systems, and the user can also use an interface provided by the systems to realize the expansion of the functions according to needs.
4. The communication system according to claim 1, wherein the sensing module comprises a device server, and when the sensing module is analyzed and processed, it is determined that the user is not in a device-using state, the device control and scheduling logic in the function module controls the apparatus and the functions and services of the bearer to enter a standby state, and when the user returns to the device-using state, the device is automatically woken up to enter a working state.
5. The perception-based communication system of claim 1, wherein the perception module further comprises a user identity recognition function based on perception to manage user rights, ensure user privacy and reduce authentication cost of users.
6. The perception-based communication system of claim 1, wherein the functional modules further include an interaction unit interacting with a cloud, and the communication system can expand functions of the device by means of cloud services, and improve usability and usability of the device.
7. The awareness-based communication system according to claim 1, wherein the functional modules further comprise an application interface and an application store, and the application interface and the application store are used for developing and using the awareness-oriented device application, and enriching and expanding the coverage of the functions and usage scenarios of the device.
8. The perception-based communication system according to claim 1, wherein the communication system is capable of operating as a computer peripheral, as an audio/video input source, a live broadcast source, a perception sensor peripheral, or independently as a live broadcast and video recording device or an intelligent device with a perception function.
9. A communication device, characterised in that a perception based communication system according to any of claims 1-7 is used.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010951549.4A CN114253386A (en) | 2020-09-11 | 2020-09-11 | Communication system based on perception |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010951549.4A CN114253386A (en) | 2020-09-11 | 2020-09-11 | Communication system based on perception |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114253386A true CN114253386A (en) | 2022-03-29 |
Family
ID=80788071
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010951549.4A Pending CN114253386A (en) | 2020-09-11 | 2020-09-11 | Communication system based on perception |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114253386A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105045122A (en) * | 2015-06-24 | 2015-11-11 | 张子兴 | Intelligent household natural interaction system based on audios and videos |
| JP2019079204A (en) * | 2017-10-23 | 2019-05-23 | 佐藤 良治 | Information input-output control system and method |
| CN110178125A (en) * | 2017-01-18 | 2019-08-27 | 索尼公司 | Information processing equipment, information processing method and program |
| EP3557498A1 (en) * | 2018-04-20 | 2019-10-23 | Facebook, Inc. | Processing multimodal user input for assistant systems |
| US20190327331A1 (en) * | 2018-04-20 | 2019-10-24 | Facebook, Inc. | Aggregating Semantic Information for Improved Understanding of Users |
-
2020
- 2020-09-11 CN CN202010951549.4A patent/CN114253386A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105045122A (en) * | 2015-06-24 | 2015-11-11 | 张子兴 | Intelligent household natural interaction system based on audios and videos |
| CN110178125A (en) * | 2017-01-18 | 2019-08-27 | 索尼公司 | Information processing equipment, information processing method and program |
| JP2019079204A (en) * | 2017-10-23 | 2019-05-23 | 佐藤 良治 | Information input-output control system and method |
| EP3557498A1 (en) * | 2018-04-20 | 2019-10-23 | Facebook, Inc. | Processing multimodal user input for assistant systems |
| US20190327331A1 (en) * | 2018-04-20 | 2019-10-24 | Facebook, Inc. | Aggregating Semantic Information for Improved Understanding of Users |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102173479B1 (en) | Method, user terminal and server for information exchange communications | |
| US11108722B2 (en) | Data processing terminals, icon badges, and methods of making and using the same | |
| CN205427842U (en) | Individual work systems of movable state combination and adjustment | |
| CN104777911B (en) | A kind of intelligent interactive method based on holographic technique | |
| CN107756395A (en) | Control system, the method and apparatus of intelligent robot | |
| US20180123629A1 (en) | Smart-ring methods and systems | |
| CN104008678A (en) | Intelligent network terminal for collecting and encrypting classroom teaching multimedia information in real time and working method | |
| CN108536099A (en) | A kind of information processing method, device and mobile terminal | |
| TW202018587A (en) | Intelligent device control method and apparatus and storage medium | |
| CN109619801A (en) | Intelligent cosmetic mirror | |
| CN110154056A (en) | Service robot and its man-machine interaction method | |
| CN111541951B (en) | Video-based interactive processing method and device, terminal and readable storage medium | |
| WO2019140697A1 (en) | Interphone extension intelligent robot device | |
| CN112749956A (en) | Information processing method, device and equipment | |
| CN109891357A (en) | Emotion intelligently accompanies device | |
| US20220284254A1 (en) | Reconfigurable interaction badge | |
| CN211045028U (en) | Wisdom laboratory robot system | |
| CN104794096A (en) | Personal work system capable of being dynamically combined and adjusted | |
| CN114253386A (en) | Communication system based on perception | |
| CN104915646A (en) | Method and terminal for conference management | |
| CN205003753U (en) | Discernment monitoring device | |
| CN111985252A (en) | Dialogue translation method and device, storage medium and electronic equipment | |
| CN108139811A (en) | Record performs the method for screen and the electronic equipment of processing this method | |
| CN113486730A (en) | Intelligent reminding method based on face recognition and related device | |
| CN209543191U (en) | A kind of multifunctional application remote training all-in-one machine |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |