CN114598817A - Human-computer interaction judgment method and device based on multi-person interaction action judgment - Google Patents
Human-computer interaction judgment method and device based on multi-person interaction action judgment Download PDFInfo
- Publication number
- CN114598817A CN114598817A CN202210214053.8A CN202210214053A CN114598817A CN 114598817 A CN114598817 A CN 114598817A CN 202210214053 A CN202210214053 A CN 202210214053A CN 114598817 A CN114598817 A CN 114598817A
- Authority
- CN
- China
- Prior art keywords
- judgment
- interaction
- image
- data
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 61
- 238000000034 method Methods 0.000 title claims abstract description 39
- 230000009471 action Effects 0.000 title claims abstract description 23
- 230000002452 interceptive effect Effects 0.000 claims abstract description 17
- 230000008859 change Effects 0.000 claims description 10
- 230000008569 process Effects 0.000 claims description 9
- 238000004590 computer program Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 5
- 238000009499 grossing Methods 0.000 claims description 4
- 230000001960 triggered effect Effects 0.000 claims description 3
- 206010063385 Intellectualisation Diseases 0.000 abstract description 3
- 238000010801 machine learning Methods 0.000 abstract description 2
- 230000009977 dual effect Effects 0.000 abstract 1
- 230000000875 corresponding effect Effects 0.000 description 10
- 230000000694 effects Effects 0.000 description 2
- 239000003550 marker Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/61—Control of cameras or camera modules based on recognised objects
- H04N23/611—Control of cameras or camera modules based on recognised objects where the recognised objects include parts of the human body
Abstract
The invention discloses a human-computer interaction judgment method and device based on multi-person interaction action judgment, and relates to the field of machine learning. The method and the device realize dynamic tracking of multi-person interaction actions and execution of triggering actions corresponding to the actions through the camera, and fuse the actions with a traditional man-machine interaction mode, thereby realizing intelligent shooting. The invention aims to solve the problem that the multi-person interactive mode cannot be dynamically identified and processed in the prior art. The system disclosed by the invention can realize a dual interaction mode of multi-person interaction and man-machine interaction, and has stronger interactivity and more intellectualization.
Description
Technical Field
The invention relates to the field of machine learning, in particular to a human-computer interaction judgment method and device based on multi-person interaction action judgment.
Background
Along with the coming of the AI era, products in daily life of people are developing towards intellectualization and electronization, meanwhile, fast paced life not only brings convenience to people in life, but also brings higher requirements to life style of people due to improvement of time cost, and mutual promotion of intelligence and life helps people to better enter the intellectualization era, so that more and more intelligent human-computer interaction modes are presented to meet the pursuit of individualization of people. For example, a mobile phone and a related interaction system thereof, a user can already execute some operations on the mobile phone in a non-touch manner, but the prior art can only support the operation of realizing instructions by recognizing inherent gestures, and lacks a method for realizing the execution of intelligent equipment by multi-person interaction, single image recognition only makes simple 0-1 judgment, and cannot dynamically track and execute corresponding actions, that is, the prior art is difficult to realize real intelligent operations.
Disclosure of Invention
In order to solve the problem that the multi-user interaction mode cannot be dynamically identified and processed in the prior art, the invention provides a man-machine interaction judgment method and a man-machine interaction judgment device based on multi-user interaction action judgment.
In view of the above circumstances, the present invention provides a human-computer interaction determination method based on multi-person interaction determination, comprising:
s1: starting a camera, and selecting and setting interaction judgment data from the interaction judgment data set;
s2: determining human body feature points to be identified according to the interaction judgment data, identifying and marking corresponding feature points of a plurality of people in the photographing area through the camera;
s3: the method comprises the following steps that a camera acquires and identifies N frames of images with continuous time sequences, and the N frames of images are compared with interactive judgment data for judgment;
s4: and returning a judgment result to the target equipment, and triggering a photographing instruction when the judgment result is the same.
As a preferred embodiment, the interaction judgment data set in S1 includes movement data of marker points corresponding to different multi-person interactions, including but not limited to kissing and clapping, where the marker points are set according to different interactions.
As a preferred embodiment, the interaction judgment data is mark point data and mark point position relation data corresponding to images arranged in time sequence, the position movement relation of the mark point is obtained according to the interaction judgment data set in S1, the initial state is a comparison preparation state, the change of the position of the mark point in any continuous M frames of images is sequentially calculated in the time sequence continuous images acquired by the camera, when the distance change relation and the change of the position of the mark point in any continuous M frames of images in the set interaction judgment data satisfy a given condition, the matching state is determined, then, the matching operation is executed, and the photographing instruction is triggered.
As a preferred embodiment, after the comparison judgment is started, if the position change relationship of any continuous K frames of mark points in the identification image cannot meet a given condition, terminating the current process, and entering the comparison preparation state again, where K is a natural number greater than 2. .
As a preferred embodiment, after entering a matching state, smoothing a difference image in an image acquired by a camera, where the difference image is an image when there is a difference between mark point data and preset interaction judgment data in a non-termination state, and the smoothing includes: and correcting the difference image into the state which is the same as the corresponding comparison judgment data, or removing the difference image from the identification data collected by the camera.
As a preferred embodiment, the photographing region identified by the camera in S2 is a local spatial region of the image based on the feature point of the judgment data, and the camera captures the image, selects a local image region corresponding to the feature point of the judgment data, and performs the comparison process.
A human-computer interaction system device based on multi-person interaction action judgment comprises:
the image acquisition module is used for acquiring a time sequence frame image acquired by the camera;
the image judgment module is used for comparing the time sequence frame image acquired by the image acquisition module with preset comparison judgment data and returning a judgment result;
and the image output module is used for executing a corresponding instruction according to a judgment result returned by the image judgment module, triggering a photographing function and outputting a photographed image if the judgment is successful, and continuing to judge until the current process is terminated if the judgment is failed.
An electronic device comprising a memory for storing computer instructions executable on a processor, a processor for implementing a method according to any of the present disclosure when executing the computer instructions.
A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements any of the methods of the present disclosure.
Compared with the prior art, the invention has the following beneficial effects: the system disclosed by the invention has stronger interactivity and is more intelligent compared with the traditional judgment mode that the relation of the posture mark point cannot be obtained essentially.
Drawings
FIG. 1 is a flow chart of a comparison and judgment method provided by the present invention;
FIG. 2 is a human-computer interaction system apparatus employing the method of the present invention;
fig. 3 is an electronic device employing the method of the present invention.
Detailed Description
In order to make the technical means, the creation characteristics, the achievement purposes and the effects of the invention easy to understand, the invention is further described with the specific embodiments.
As shown in fig. 1, fig. 1 illustrates an exemplary interaction determination process, where the method may be performed by a human-computer interaction system, and the method may include:
s1: starting a camera, and selecting and setting interaction judgment data from the interaction judgment data set; the camera in the step is embedded into the existing intelligent equipment, and the intelligent equipment has the function of networking and uploading the image captured by the camera to the cloud so as to carry out the next comparison work;
the interactive judgment data set is an action sequence stored in a cloud in advance, after a user selects an action in the interactive data set, the interactive data can be stored locally on the intelligent device as a comparison source and comprises human body characteristic points and motion trajectory data which need to be identified, the kiss-making action disclosed by the embodiment is taken as an example, namely after a human face is selected through a frame, the middle positions of two human lips are respectively acquired as a mark point 1 and a mark point 2, the Euclidean distance between the mark point 1 and the mark point 2 is determined as a reference relationship according to the position reduction relationship of a time sequence, and the reference relationship is stored in a storage module of the intelligent device;
s2: determining human body feature points to be identified according to the interaction judgment data, identifying and marking corresponding feature points of a plurality of people in a photographing area through a camera, starting to capture image contents of the camera identification area after the camera receives a comparison data setting signal set by a user in S1, marking the image contents after finding a mark point 1 and a mark point 2 required by the action, and prompting on intelligent equipment if a plurality of face contents appear in an identification result to confirm that only the personnel required by the action are in the identification area;
s3: the method comprises the following steps that a camera acquires and identifies 10 frames of images with continuous time sequences, and the 10 frames of images are compared with interactive judgment data for judgment; the camera automatically enters a comparison state from a comparison preparation state after determining an identification area and a mark point, the position relationship of two mark points between every two frames of images is recorded, if the change relationship of two corresponding mark points between three continuous frames of images is the same as the change relationship track of the set mark points of the interactive judgment data, the comparison is continued until the final position relationship of the mark points is obtained, in the embodiment, the mark point 1 and the mark point 2 represent the middle positions of the lips of two photographing parties, the kiss action is completed, the Euclidean distance of the mark point 1 and the mark point 2 needs to be continuously reduced until the camera identifies that the distance between the two is 0, and compared with the traditional fixed posture for photographing, the method disclosed by the invention can effectively enhance the interaction, in the embodiment, the fixed kiss posture for photographing may limit the kiss angle or the face direction, the traditional judgment mode cannot essentially acquire the relationship of the gesture mark points, so that the interactivity is poor, and the recognition effect is also poor;
or, an error threshold is set for the set interactive judgment data, in the range of the error threshold, in the identification image acquired by the camera between the images captured by the camera, the first 6 frames of images can meet the track requirement of the interactive judgment data, but the position relation of the mark points between 5 frames and 7 frames cannot meet the condition, the 7 th frame of image is corrected, the position of the mark point of the frame of image is repositioned, the image within the allowable range of the error threshold is corrected into the image meeting the position relation of the mark points in the interactive judgment data, or the current frame of image is subjected to rejection processing.
S4: in this embodiment, when the position relationship between the mark point 1 and the mark point 2 is 0, the system can determine that the motion trajectory of the mark point captured by the camera is the same as the set trajectory of the kiss mark point, so that the image data currently acquired by the camera is matched with the set interaction determination data, the camera is triggered to start the photographing function, and the process is automatically terminated after the photographing is completed.
Fig. 2 provides a human-computer interaction system device, as shown in fig. 2, the device at least comprises:
the image acquisition module is used for acquiring a time sequence frame image acquired by the camera;
the image judgment module is used for comparing the time sequence frame image acquired by the image acquisition module with preset comparison judgment data and returning a judgment result;
and the image output module is used for executing a corresponding instruction according to a judgment result returned by the image judgment module, triggering a photographing function and outputting a photographed image if the judgment is successful, and continuing the judgment until the current process is terminated if the judgment is failed.
Fig. 3 provides an electronic device, as shown in fig. 3, comprising a memory for storing computer instructions executable on a processor and a processor for implementing any of the methods of the present invention when executing the computer instructions.
The invention also discloses a computer-readable storage medium on which a computer program is stored, which program, when being executed by a processor, is adapted to carry out any of the methods of the invention
As will be appreciated by one skilled in the art, one or more embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, one or more embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Although the present invention has been described in detail with reference to the foregoing embodiments, it will be apparent to those skilled in the art that various changes in the embodiments and/or modifications of the invention can be made, and equivalents and modifications of some features of the invention can be made without departing from the spirit and scope of the invention.
Claims (9)
1. A human-computer interaction judgment method based on multi-person interaction action judgment is characterized by comprising the following steps:
s1: starting a camera, and selecting and setting interaction judgment data from the interaction judgment data set;
s2: determining human body feature points to be identified according to the interaction judgment data, identifying and marking corresponding feature points of a plurality of people in the photographing area through the camera;
s3: the method comprises the following steps that a camera acquires and identifies N frames of images with continuous time sequences, and the N frames of images are compared with interactive judgment data for judgment;
s4: and returning a judgment result to the target equipment, and triggering a photographing instruction when the judgment result is the same.
2. The human-computer interaction judgment method based on multi-person interaction action judgment as claimed in claim 1, wherein: s1, the interactive judgment data set includes mark point movement data corresponding to different multi-person interactive actions, including but not limited to kiss and palms, and the mark points are set according to different interactive actions.
3. The human-computer interaction judgment method based on multi-person interaction action judgment as claimed in claim 1, wherein: the interactive judgment data is mark point data and mark point position relation data corresponding to images arranged according to a time sequence, the position movement relation of the mark point is obtained according to the interactive judgment data set in S1, the initial state is a comparison preparation state, the change of the position of the mark point in any continuous M frames of images is sequentially calculated in the time sequence continuous images acquired by the camera, when the distance change relation and the change of the position of the mark point in any continuous M frames of images in the set interactive judgment data meet given conditions, the matching state is judged, then matching operation is executed, and a photographing instruction is triggered.
4. The human-computer interaction judgment method based on multi-person interaction action judgment of claim 3, wherein: after the comparison judgment is started, if the position change relation of any continuous K frame mark points in the identification image cannot meet the given condition, the current process is terminated, and the comparison preparation state is entered again, wherein K is a natural number larger than 2.
5. The human-computer interaction judgment method based on multi-person interaction action judgment of claim 3, wherein: after entering a matching state, smoothing a difference image in an image acquired by a camera, wherein the difference image is an image when a difference exists between mark point data and preset interaction judgment data in a non-termination state, and the smoothing comprises the following steps: and correcting the difference image into the state which is the same as the corresponding comparison judgment data, or removing the difference image from the identification data collected by the camera.
6. The human-computer interaction judgment method based on multi-person interaction action judgment as claimed in claim 1, wherein: in S2, the photographing region identified by the camera is a local spatial region of the image based on the feature point of the determination data, and the camera collects the image, selects a local image region corresponding to the feature point of the determination data, and performs a comparison process.
7. A human-computer interaction system device based on multi-person interaction action judgment is characterized by comprising:
the image acquisition module is used for acquiring a time sequence frame image acquired by the camera;
the image judgment module is used for comparing the time sequence frame image acquired by the image acquisition module with preset comparison judgment data and returning a judgment result;
and the image output module is used for executing a corresponding instruction according to a judgment result returned by the image judgment module, triggering a photographing function and outputting a photographed image if the judgment is successful, and continuing the judgment until the current process is terminated if the judgment is failed.
8. An electronic device, characterized in that: the apparatus comprising a memory for storing computer instructions executable on a processor, the processor for implementing the method of any one of claims 1 to 7 when executing the computer instructions.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210214053.8A CN114598817A (en) | 2022-03-04 | 2022-03-04 | Human-computer interaction judgment method and device based on multi-person interaction action judgment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210214053.8A CN114598817A (en) | 2022-03-04 | 2022-03-04 | Human-computer interaction judgment method and device based on multi-person interaction action judgment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114598817A true CN114598817A (en) | 2022-06-07 |
Family
ID=81807577
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210214053.8A Pending CN114598817A (en) | 2022-03-04 | 2022-03-04 | Human-computer interaction judgment method and device based on multi-person interaction action judgment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114598817A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2016015078A (en) * | 2014-07-03 | 2016-01-28 | Ｎｅｃパーソナルコンピュータ株式会社 | Display control device, display control method, and program |
| CN107230226A (en) * | 2017-05-15 | 2017-10-03 | 深圳奥比中光科技有限公司 | Determination methods, device and the storage device of human body incidence relation |
| CN107395966A (en) * | 2017-07-19 | 2017-11-24 | 维沃移动通信有限公司 | A kind of photographic method, electronic equipment and computer-readable recording medium |
| CN108307110A (en) * | 2018-01-18 | 2018-07-20 | 维沃移动通信有限公司 | A kind of image weakening method and mobile terminal |
| CN110348419A (en) * | 2019-07-18 | 2019-10-18 | 三星电子（中国）研发中心 | Method and apparatus for taking pictures |
| WO2021077426A1 (en) * | 2019-10-25 | 2021-04-29 | 深圳市欢太科技有限公司 | Image processing method and apparatus, and electronic device |
| CN113194254A (en) * | 2021-04-28 | 2021-07-30 | 上海商汤智能科技有限公司 | Image shooting method and device, electronic equipment and storage medium |
| US20220007816A1 (en) * | 2020-07-07 | 2022-01-13 | Perfect Mobile Corp. | System and method for navigating user interfaces using a hybrid touchless control mechanism |
-
2022
- 2022-03-04 CN CN202210214053.8A patent/CN114598817A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2016015078A (en) * | 2014-07-03 | 2016-01-28 | Ｎｅｃパーソナルコンピュータ株式会社 | Display control device, display control method, and program |
| CN107230226A (en) * | 2017-05-15 | 2017-10-03 | 深圳奥比中光科技有限公司 | Determination methods, device and the storage device of human body incidence relation |
| CN107395966A (en) * | 2017-07-19 | 2017-11-24 | 维沃移动通信有限公司 | A kind of photographic method, electronic equipment and computer-readable recording medium |
| CN108307110A (en) * | 2018-01-18 | 2018-07-20 | 维沃移动通信有限公司 | A kind of image weakening method and mobile terminal |
| CN110348419A (en) * | 2019-07-18 | 2019-10-18 | 三星电子（中国）研发中心 | Method and apparatus for taking pictures |
| WO2021077426A1 (en) * | 2019-10-25 | 2021-04-29 | 深圳市欢太科技有限公司 | Image processing method and apparatus, and electronic device |
| US20220007816A1 (en) * | 2020-07-07 | 2022-01-13 | Perfect Mobile Corp. | System and method for navigating user interfaces using a hybrid touchless control mechanism |
| CN113194254A (en) * | 2021-04-28 | 2021-07-30 | 上海商汤智能科技有限公司 | Image shooting method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102193626B (en) | Gesture recognition apparatus, and method for controlling gesture recognition apparatus | |
| US9892316B2 (en) | Method and apparatus for pattern tracking | |
| JP4711885B2 (en) | Remote control device and method | |
| CN108596092B (en) | Gesture recognition method, device, equipment and storage medium | |
| US8525876B2 (en) | Real-time embedded vision-based human hand detection | |
| US20110273551A1 (en) | Method to control media with face detection and hot spot motion | |
| US9639744B2 (en) | Method for controlling and requesting information from displaying multimedia | |
| US8897490B2 (en) | Vision-based user interface and related method | |
| WO2014021760A2 (en) | Improved identification of a gesture | |
| CN106155315A (en) | The adding method of augmented reality effect, device and mobile terminal in a kind of shooting | |
| CN104914989B (en) | The control method of gesture recognition device and gesture recognition device | |
| CN109308437B (en) | Motion recognition error correction method, electronic device, and storage medium | |
| CN107832736B (en) | Real-time human body action recognition method and real-time human body action recognition device | |
| CN108647633B (en) | Identification tracking method, identification tracking device and robot | |
| CN115565241A (en) | Gesture recognition object determination method and device | |
| CN108875506B (en) | Face shape point tracking method, device and system and storage medium | |
| CN107357424B (en) | Gesture operation recognition method and device and computer readable storage medium | |
| WO2023169282A1 (en) | Method and apparatus for determining interaction gesture, and electronic device | |
| CN112655021A (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| CN111651038A (en) | Gesture recognition control method based on ToF and control system thereof | |
| CN114598817A (en) | Human-computer interaction judgment method and device based on multi-person interaction action judgment | |
| CN114816045A (en) | Method and device for determining interaction gesture and electronic equipment | |
| CN115061577A (en) | Hand projection interaction method, system and storage medium | |
| CN113537128A (en) | Method, system and equipment for comparing and analyzing continuous actions based on deep learning posture assessment | |
| CN113766130A (en) | Video shooting method, electronic equipment and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Ma Yong Inventor after: Li Qing Inventor before: Ma Yong |
|
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |