CN108596056A - A kind of taxi operation behavior act recognition methods and system - Google Patents
A kind of taxi operation behavior act recognition methods and system Download PDFInfo
- Publication number
- CN108596056A CN108596056A CN201810317515.2A CN201810317515A CN108596056A CN 108596056 A CN108596056 A CN 108596056A CN 201810317515 A CN201810317515 A CN 201810317515A CN 108596056 A CN108596056 A CN 108596056A
- Authority
- CN
- China
- Prior art keywords
- human
- joint points
- video sequence
- identified
- point feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 230000009471 action Effects 0.000 claims abstract description 93
- 230000006399 behavior Effects 0.000 claims description 55
- 238000012549 training Methods 0.000 claims description 14
- 230000002159 abnormal effect Effects 0.000 claims description 8
- 239000000284 extract Substances 0.000 claims description 8
- 230000001052 transient effect Effects 0.000 claims description 7
- 230000000007 visual effect Effects 0.000 claims description 7
- 238000013144 data compression Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 claims description 3
- 238000012545 processing Methods 0.000 abstract description 11
- 238000012544 monitoring process Methods 0.000 abstract description 8
- 230000001737 promoting effect Effects 0.000 abstract 1
- 238000003062 neural network model Methods 0.000 description 10
- 230000000875 corresponding effect Effects 0.000 description 7
- 238000010586 diagram Methods 0.000 description 6
- 230000008859 change Effects 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000006073 displacement reaction Methods 0.000 description 4
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 description 3
- 206010000117 Abnormal behaviour Diseases 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000004071 biological effect Effects 0.000 description 1
- 230000000903 blocking effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000000203 mixture Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004321 preservation Methods 0.000 description 1
- 238000012954 risk control Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/48—Matching video sequences
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G06Q50/40—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Traffic Control Systems (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
A kind of taxi operation behavior act recognition methods of present invention offer and system, wherein method include：The human synovial point feature for belonging to same people in every frame image of video sequence to be identified is extracted, at least one human joint points characteristic sequence is obtained；Classification and Identification is carried out to the human action in each human joint points characteristic sequence, obtains action recognition result；Wherein human synovial point feature includes artis title and artis position coordinates.This method and system effectively realize the monitoring to taxi operation behavior; it is simultaneously human joint points characteristic sequence by original image processing; the Human biology feature in original image is concealed; the effective protection privacy of passenger in taxi and driver; and the storage burden of server is reduced to a certain extent, be conducive to the overall performance for promoting server.
Description
Technical field
The present invention relates to technical field of image processing, more particularly, to a kind of taxi operation behavior act identification side
Method and system.
Background technology
In taxi operation management, operation behaviorist risk control action identification is an important process, and identification car multiplies
The body language of visitor and driver are also a necessary link.
However, in existing taxi operation management, usually going out that photographic device is installed in by bus, and utilize photographic device
The raw video image of passenger in taxi and driver is acquired, and raw video image is preserved, subsequently again to preservation
Behavior act in raw video image carries out analysis and identification.Wherein, by raw video image preserved too much utilize and
The biological property for illustrating passenger and driver exposes citizen privacy to a certain extent, and then brings personal secrets hidden danger.
Simultaneously as the EMS memory occupation amount of raw video image is larger, the service of virtually increasing is preserved to raw video image
The storage of device is born, and reduces the overall performance of server to a certain extent.
In view of this, it is urgent to provide a kind of taxi operation behavior act recognition methods and system, taxi fortune is being realized
While seeking behavior act identification, the privacy of passenger and driver are protected, and reduces the storage burden of service, and then is conducive to be promoted
The overall performance of server.
Invention content
The present invention is for the personal secrets hidden danger and server storage in the presence of overcoming existing taxi operation to manage
The problem that burden is big, performance is low provides a kind of taxi operation behavior act recognition methods and system.
On the one hand, the present invention provides a kind of taxi operation behavior act recognition methods, including：
S1 extracts the human synovial point feature for belonging to same people in every frame image of video sequence to be identified, obtains at least
One human joint points characteristic sequence；
S2 carries out Classification and Identification to the human action in each human joint points characteristic sequence, obtains action recognition
As a result；
The wherein described human joint points feature includes artis title and artis position coordinates.
Preferably, the step S1 further comprises：
All human synovials in every frame image of the video sequence to be identified are extracted using default human skeleton model
Point feature, and all human joint points features in the image per frame are divided, it obtains and belongs in the image per frame
The human synovial point feature of same people；
The human synovial point feature for belonging to same people in each frame image is combined, at least one human body is obtained and closes
Node diagnostic sequence.
Preferably, further include before the step S1：
Video sequence sample is obtained, to belonging to the human joint points of same people in every frame image of the video sequence sample
Feature is labeled, and forms training sample；
The default human skeleton model is trained using the training sample；
Wherein, the video sequence sample includes single video sequence sample and more people's video sequence samples.
Preferably, the step S2 further comprises：
In each human joint points characteristic sequence, two neighboring human synovial point feature is compared；
When the two neighboring human synovial point feature is inconsistent, the two neighboring human synovial point feature is carried out
Mark, and Classification and Identification is carried out to human action according to the difference between the two neighboring human synovial point feature, it is moved
Make recognition result.
Preferably, further include after the step S2：
When the action recognition result belongs to default anomaly classification, by the abnormal mark of video sequence mark to be identified
Label, and send pre-warning signal to preset client；
When the action recognition result belongs to default normal classification, by the normal mark of video sequence mark to be identified
Label.
Preferably, further include after the step S2：
The action recognition result is subjected to data compression, the compressed action recognition result is stored.
Preferably, further include before the step S1：
Image Acquisition is carried out to human body to be identified from multiple default visual angles, obtains the video sequence to be identified.
On the one hand, the present invention provides a kind of taxi operation behavior act identifying system, including：
Characteristic extracting module belongs to the human joint points of same people in every frame image for extracting video sequence to be identified
Feature obtains at least one human joint points characteristic sequence；
Action recognition module, for carrying out classification knowledge to the human action in each human joint points characteristic sequence
Not, action recognition result is obtained；
The wherein described human joint points feature includes artis title and artis position coordinates.
On the one hand, the present invention provides a kind of equipment of taxi operation behavior act recognition methods, including：
At least one processor；And
At least one processor being connect with the processor communication, wherein：
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program to refer to
It enables and is able to carry out any of the above-described method.
On the one hand, the present invention provides a kind of non-transient computer readable storage medium, and the non-transient computer is readable to deposit
Storage media stores computer instruction, and the computer instruction makes the computer execute any of the above-described method.
A kind of taxi operation behavior act recognition methods provided by the invention and system, by shooting the original in taxi
Beginning video image obtains video sequence to be identified from raw video image, in the every frame image for extracting video sequence to be identified
The human synovial point feature for belonging to same people obtains at least one human joint points characteristic sequence；To each human joint points spy
The human action levied in sequence carries out Classification and Identification, obtains action recognition result；Effectively realize to taxi operation behavior
Monitoring, while being human joint points characteristic sequence by original image processing, the Human biology feature in original image has been concealed,
The effective protection privacy of passenger in taxi and driver, and the storage burden of server is to a certain extent reduced, be conducive to
Promote the overall performance of server.
Description of the drawings
Fig. 1 is a kind of overall flow schematic diagram of taxi operation behavior act recognition methods of the embodiment of the present invention；
Fig. 2 is a kind of flow diagram of human joint points characteristic sequence extracting method of the embodiment of the present invention；
Fig. 3 is the flow signal of the classification of motion recognition methods based on human joint points characteristic sequence of the embodiment of the present invention
Figure；
Fig. 4 is a kind of overall structure diagram of taxi operation behavior act identifying system of the embodiment of the present invention；
Fig. 5 is a kind of structural framing signal of the equipment of taxi operation behavior act recognition methods of the embodiment of the present invention
Figure.
Specific implementation mode
With reference to the accompanying drawings and examples, the specific implementation mode of the present invention is described in further detail.Implement below
Example is not limited to the scope of the present invention for illustrating the present invention.
Fig. 1 is a kind of overall flow schematic diagram of taxi operation behavior act recognition methods of the embodiment of the present invention, such as
Shown in Fig. 1, the present invention provides a kind of taxi operation behavior act recognition methods, including：
S1 extracts the human synovial point feature for belonging to same people in every frame image of video sequence to be identified, obtains at least
One human joint points characteristic sequence；
Specifically, by the video acquisition device installed in taxi in taxi passenger and/or driver regard
Frequency is shot, and obtains raw video image, video sequence to be identified is obtained from raw video image, is wrapped in video sequence to be identified
Include the behavior act of passenger and/or driver.In order to protect the privacy of passenger in taxi and driver, in the present embodiment, knowledge is treated
Other video is processed, and for every frame image in video sequence to be identified, is extracted and is closed per all human bodies in frame image
Node diagnostic is conducive to protect passenger in taxi and driver to conceal the Human biology feature in original image
Privacy.
In view of may include one or more human bodies in every frame image, in order to single human action into line trace, need to be
The human synovial point feature for belonging to same people is detected in per all human joint points features in frame image.On this basis,
Then can be a human joint points spy by video sequence to be identified processing when only including single human body in video sequence to be identified
Levy sequence；Then can be that multiple human bodies are closed by video sequence to be identified processing when video sequence to be identified includes multiple human bodies
Node diagnostic sequence.
S2 carries out Classification and Identification to the human action in each human joint points characteristic sequence, obtains action recognition result；
Wherein human synovial point feature includes artis title and artis position coordinates.
It specifically, can be by human synovial after by video sequence to be identified processing for human joint points characteristic sequence
Point feature sequence is preserved, and to substitute in the form of original image is preserved, protects passenger and Si to a certain extent
The privacy of machine.Wherein each of human joint points characteristic sequence body joint point feature include single human body included it is all
The location of artis title and all artis coordinate.Human joint points typically refer to the head of human body, shoulder, trunk,
The joint parts such as four limbs, the quantity of human joint points may include 15 artis, 18 artis, 21 artis etc., can
To pre-set the quantity of the required artis extracted according to actual demand, it is not specifically limited herein.
Meanwhile it being based on above-mentioned human joint points characteristic sequence, it can be to the human body in each human joint points characteristic sequence
Action carries out Classification and Identification.In each human joint points characteristic sequence, exist when between two neighboring human synovial point feature
When difference, then it can determine there is human action at this time.Determine that there are artis in two neighboring human synovial point feature simultaneously
The artis title of change in location, you can by human action specific to the movement of some artis, and according to artis position
Specific variation, you can determine the information such as the direction of motion, moving displacement and the deviation angle of the artis.
If for example, there is only single human bodies in raw video image to be identified, and wherein certain two adjacent moment corresponds to
Raw video image in there are when the headwork of human body, then corresponding two neighboring human joint points of the adjacent two field pictures
Feature can change, and can specifically determine that the position coordinates of joint of head point are changed, and then can determine that human body is deposited at this time
In headwork, while according to the relationship between the front and back position coordinates of joint of head point action, you can determine human body head
The information such as the direction of motion, moving displacement and deviation angle, and then determine the specific type of action of human body head.
Based on the above technical solution, by the way that the human action in video sequence to be identified is identified, you can
It realizes and effective monitoring is carried out to the action behavior of passenger in taxi and/or driver.
A kind of taxi operation behavior act recognition methods provided by the invention, by shooting the original video in taxi
Image obtains video sequence to be identified from raw video image, extracts in every frame image of video sequence to be identified and belongs to same
The human synovial point feature of one people obtains at least one human joint points characteristic sequence；To each human joint points characteristic sequence
In human action carry out Classification and Identification, obtain action recognition result；The monitoring to taxi operation behavior effectively is realized, together
When by original image processing be human joint points characteristic sequence, concealed the Human biology feature in original image, effectively protect
Passenger in taxi and the privacy of driver have been protected, and has reduced the storage burden of server to a certain extent, has been conducive to promote clothes
The overall performance of business device.
Based on any of the above-described embodiment, a kind of taxi operation behavior act recognition methods is provided, as shown in Fig. 2, step
S1 further comprises：
S11 extracts all human synovials in every frame image of video sequence to be identified using default human skeleton model
Point feature, and all human joint points features in every frame image are divided, it obtains and belongs to same people's per in frame image
Human synovial point feature；
Specifically, it for every frame image in video sequence to be identified, is extracted per frame figure using default human skeleton model
All human joint points features as in, then all human joint points features in every frame image are divided, it obtains per frame
Belong to the human synovial point feature of same people in image.Wherein default skeleton pattern can be neural network model, for example, can be with
Using multistage convolutional neural networks, in each stage, including two convolution branches；First convolution branch is used for obtaining figure
As confidence map, a confidence level is generated for each pixel, so for each joint of human body generate a confidence map to get
To human joint points；Second convolution branch is used for obtaining the affine domain in part, and part is affine, and domain is a bivector set, this
Include the location coordinate information of artis in a bivector set.People is obtained using the confidence map that first convolution branch obtains
Body artis information obtains the weight information of each human joint points using the affine domain in part, obtains the maximum side connection of weight
Mode finally obtains the human joint points for belonging to same people.
It can be obtained the human joint points for belonging to same people in every frame image of video image to be identified using aforesaid way,
And then the extractable human synovial point feature for belonging to same people.In addition, can also be carried in other embodiments using other modes
The human synovial point feature in every frame image is taken, can be configured according to actual demand, be not specifically limited herein.
The human synovial point feature for belonging to same people in each frame image is combined by S12, is obtained at least one human body and is closed
Node diagnostic sequence.
Specifically, it after being obtained using aforesaid way per the human synovial point feature of same people is belonged in frame image, will wait for
The human synovial point feature for belonging to same people in the corresponding each frame image of identification video sequence is combined.When video sequence to be identified
When only including single human body in row, then a human joint points characteristic sequence can get by combination；When video sequence to be identified
When including multiple human bodies, then multiple human joint points characteristic sequences can get by combination.
A kind of taxi operation behavior act recognition methods provided by the invention is waited for using the extraction of default human skeleton model
Identify all human joint points features in every frame image of video sequence, and special to all human joint points in every frame image
Sign is divided, and is obtained per the human synovial point feature for belonging to same people in frame image；It will belong to same people's in each frame image
Human synovial point feature is combined, and obtains at least one human joint points characteristic sequence.This method will can accurately wait knowing
Other video sequence processing is human joint points characteristic sequence, has concealed the Human biology feature in original image, effective protection
The privacy of passenger in taxi and driver.
Based on any of the above-described embodiment, a kind of taxi operation behavior act recognition methods is provided, is also wrapped before step S1
It includes：
Video sequence sample is obtained, to belonging to the human synovial point feature of same people in every frame image of video sequence sample
It is labeled, forms training sample；
It should be noted that utilizing the people preset during human skeleton model extracts every frame image of video sequence to be identified
Before the point feature of body joint, also need to be trained default human skeleton model.
Specifically, first from video acquisition device shoot raw video image in selecting video sequence samples, for regarding
Every frame image of frequency sequence sample, marks out owner's body joint point feature in every frame image, and by all human joint points
Feature is divided, and then the human synovial point feature for belonging to same people is marked out in every frame image, by the video after mark
Sequence samples are as training sample, to be trained to default human skeleton model.
Default human skeleton model is trained using training sample；
Wherein, video sequence sample includes single video sequence sample and more people's video sequence samples.
It should be noted that due in taxi may only include single people in different time, it is also possible to including multiple
People may include single people or multiple people in video sequence that is, to be identified, so single video sample and more people's videos need to be passed through
Sample is trained default human skeleton model.In view of this, the video sequence sample of above-mentioned acquisition includes single video
Sample and more people's video sequence samples, wherein only including a human body in every frame image of single video sequence sample, i.e., per frame
All human joint points features in image belong to the human synovial point feature of same people；Every frame of more people's video sequence samples
Image includes at least two human bodies, thus need to be divided to all human joint points features in every frame image, is belonged to
In the human synovial point feature of same people.
Further, it builds and presets human skeleton model, wherein default human skeleton model can be neural network model.
For example, the neural network model of structure includes two convolution branches, each convolution branch includes multiple convolutional layers and pond layer.This
Outside, the target error of neural network model can also be set.
Further, the training sample of above-mentioned acquisition is inputted and presets human skeleton model, to presetting human skeleton model
It is trained, the wherein quantity of training sample and specific frequency of training can be configured according to actual demand, not done herein
It is specific to limit.When the error of the reality output of default human skeleton model and desired output reaches preset target error, knot
Training of the beam to default human skeleton model.
It should be noted that being only the present invention's using being trained acquisition to preset human skeleton model to neural network
A kind of embodiment can also use other modes to obtain and preset human skeleton model in other embodiments, can be according to reality
Demand is configured, and is not specifically limited herein.
A kind of taxi operation behavior act recognition methods provided by the invention, by obtaining video sequence sample, to regarding
The human synovial point feature for belonging to same people in every frame image of frequency sequence sample is labeled, and forms training sample；Utilize instruction
Practice sample to be trained default human skeleton model, is conducive to accurately obtain and presets human skeleton model, and then be conducive to root
According to the human synovial point feature for belonging to same people in default the human skeleton model accurately every frame image of extraction.
Based on any of the above-described embodiment, a kind of taxi operation behavior act recognition methods is provided, as shown in figure 3, step
S2 further comprises：
Two neighboring human synovial point feature is compared in each human joint points characteristic sequence by S21；
Specifically, for the human joint points characteristic sequence of above-mentioned acquisition, two neighboring human synovial point feature is carried out
It compares, if human action is not present in adjacent two field pictures, two neighboring human synovial point feature is consistent, if adjacent two frames figure
There are human action as in, then two neighboring human synovial point feature has differences.
It should be noted that for each of each human joint points characteristic sequence body joint point feature, needing will be every
A human joint points feature two human joint points features adjacent thereto are compared, i.e., by current frame image and previous frame image
Corresponding human synovial point feature is compared, consequently facilitating being detected to the human action at each moment.
S22 marks two neighboring human synovial point feature when two neighboring human synovial point feature is inconsistent
Note, and Classification and Identification is carried out to human action according to the difference between two neighboring human synovial point feature, obtain action recognition
As a result.
Specifically, when two neighboring human synovial point feature is inconsistent, then it can determine there is human action at this time, thus
Two neighboring human synovial point feature can be labeled, be conducive to that related personnel is prompted to there is human action at this time.Meanwhile root
Classification and Identification is carried out to human action according to the difference between two neighboring human synovial point feature, obtains action recognition result.Its
In difference between two neighboring human synovial point feature be mainly reflected in same artis position coordinates variation.
It should be noted that a complete human action, for example arm is lifted, often it is made of multiple image, in view of
This need to be according to entire human joint points characteristic sequence after difference between the two neighboring human synovial point feature of above-mentioned acquisition
In each adjacent two human synovial point feature between difference to human action carry out Classification and Identification, obtain action recognition knot
Fruit.
In practical applications, classification knowledge can be carried out to above-mentioned human action by training grader or neural network model
Not.By taking neural network model as an example, neural network model can be trained by video sequence sample, first known to acquisition
The video sequence sample of type of action obtains the corresponding human joint points characteristic sequence of video sequence sample, and to each human body
Two neighboring human synovial point feature carries out Difference Calculation in artis characteristic sequence, obtains difference result, each human body is closed
The corresponding all difference result composition difference sequences of node diagnostic sequence input neural network model, are carried out to neural network model
Training.It on this basis, can be according to adjacent in each human joint points characteristic sequence using trained neural network model
Difference between two human joint points features carries out Classification and Identification to human action, obtains action recognition result.
A kind of taxi operation behavior act recognition methods provided by the invention, in each human joint points characteristic sequence
In, two neighboring human synovial point feature is compared；When two neighboring human synovial point feature is inconsistent, to adjacent two
A human joint points feature is labeled, and is carried out to human action according to the difference between two neighboring human synovial point feature
Classification and Identification obtains action recognition as a result, it is possible to effectively and accurately be identified to the human body behavior act in taxi, thus
Be conducive to the operation behavior to taxi to be monitored.
Based on any of the above-described embodiment, a kind of taxi operation behavior act recognition methods is provided, is also wrapped after step S2
It includes：
When action recognition result belongs to default anomaly classification, by the abnormal label of video sequence mark to be identified, and to
Preset client sends pre-warning signal；
Specifically, for video sequence to be identified, after obtaining corresponding action recognition result, if action recognition result category
In default anomaly classification, then video sequence to be identified is marked into abnormal label, follow-up related personnel checks the video in taxi
When image, the video of abnormal label can be just looked at, saves monitoring period of the related personnel to video image.At the same time, to
Preset client sends pre-warning signal, abnormal behaviour occurs to remind in time in related personnel's taxi.Wherein abnormal point
Class, which includes all, may cause the personal safety of passenger and driver in taxi dangerous behavior, for example driver is in drive the cross
Head is stretched to equal behaviors outside window by the object for appreciation behaviors such as mobile phone or doze or passenger in journey；In addition, further include all passengers and driver it
Between there are the behaviors of unlawful profit-making, such as driver to collect the tip etc. of passenger privately.
When action recognition result belongs to default normal classification, video sequence to be identified is marked into normal tag.
Specifically, when action recognition result belongs to normal classification, then video sequence to be identified is marked into normal tag, i.e.,
It determines and abnormal behaviour is not present in video sequence to be identified.On this basis, follow-up related personnel checks the video in taxi
When image, the video for checking normal tag can be removed from, save monitoring period of the related personnel to video image.Wherein normal point
Class includes the limb action all gone well.
A kind of taxi operation behavior act recognition methods provided by the invention, when action recognition result belongs to default abnormal
When classification, by the abnormal label of video sequence mark to be identified, and pre-warning signal is sent to preset client；Work as action recognition
When as a result belonging to default normal classification, video sequence to be identified is marked into normal tag；Be conducive to related personnel to taxi
Operation behavior is effectively and accurately monitored, and saves the monitoring period of related personnel to a certain extent.
Based on any of the above-described embodiment, a kind of taxi operation behavior act recognition methods is provided, is also wrapped after step S2
It includes：Action recognition result is subjected to data compression, compressed action recognition result is stored.
Specifically, in the present embodiment, action recognition result needs to indicate size and Orientation, so action recognition result is total
Dimension is 2 times of dimensions of human joint points quantity.If for example, a human body includes 18 artis, then action recognition result is
For 36 dimension datas.In view of this, in order to be further reduced the memory space of action recognition result, in the present embodiment, action is known
Other result carries out data compression, by taking above-mentioned 36 dimension data as an example, first unifies 36 dimension data into three-dimensional coordinate system,
It is projected into two-dimensional space coordinate system, is finally projected into one-dimensional space coordinate system again, obtain one-dimensional action recognition
As a result.Finally, compressed action recognition result is stored, i.e., stored above-mentioned one-dimensional action recognition result.
Action recognition result is carried out data pressure by a kind of taxi operation behavior act recognition methods provided by the invention
Contracting, compressed action recognition result is stored, further reduces the memory space shared by action recognition result, favorably
It is born in the storage for reducing server, and then is conducive to be promoted the overall performance of server.
Based on any of the above-described embodiment, a kind of taxi operation behavior act recognition methods is provided, is also wrapped before step S1
It includes：Image Acquisition is carried out to human body to be identified from multiple default visual angles, obtains video sequence to be identified.
Specifically, the action behavior due to passenger and driver in taxi is in a single direction there may be blocking, than
If there are behavior acts in human body front, then it is dynamic that the front behavior can not be then obtained from the video image that the human body back side is shot
Make.In view of this, in order to carry out overall monitor to the operation behavior in taxi, in the present embodiment, video acquisition is utilized
Device carries out Image Acquisition from multiple default visual angles to the human body in taxi, and the video image of multiple default visual angle acquisitions is made
For video sequence to be identified.Plurality of default visual angle can be pre-set according to actual demand, not done herein specific
It limits.
A kind of taxi operation behavior act recognition methods provided by the invention, from multiple default visual angles to human body to be identified
Image Acquisition is carried out, video sequence to be identified is obtained, is conducive in all directions be monitored taxi operation behavior, further really
Protect the accuracy of monitored results.
Fig. 4 is a kind of overall structure diagram of taxi operation behavior act identifying system of the embodiment of the present invention, such as
Shown in Fig. 4, the present invention provides a kind of taxi operation behavior act identifying system, including：
Characteristic extracting module 1 belongs to the human synovial of same people in every frame image for extracting video sequence to be identified
Point feature obtains at least one human joint points characteristic sequence；
Specifically, by the video acquisition device installed in taxi in taxi passenger and/or driver regard
Frequency is shot, and obtains raw video image, video sequence to be identified is obtained from raw video image, is wrapped in video sequence to be identified
Include the behavior act of passenger and/or driver.In order to protect the privacy of passenger in taxi and driver, in the present embodiment, knowledge is treated
Other video is processed, and for every frame image in video sequence to be identified, every frame figure is extracted using characteristic extracting module 1
All human joint points features as in are conducive to protection and hire out to conceal the Human biology feature in original image
The privacy of passenger inside the vehicle and driver.
In view of that may include one or more human bodies in every frame image, in order to, into line trace, be utilized to single human action
Characteristic extracting module 1 detects that the human joint points for belonging to same people are special in all human joint points features in every frame image
Sign.On this basis, then can be one by video sequence to be identified processing when only including single human body in video sequence to be identified
A human joint points characteristic sequence；It, then can will be at video sequence to be identified when video sequence to be identified includes multiple human bodies
Reason is multiple human joint points characteristic sequences.
Action recognition module 2 is obtained for carrying out Classification and Identification to the human action in each human joint points characteristic sequence
Obtain action recognition result；
Wherein human synovial point feature includes artis title and artis position coordinates.
It specifically, can be by human synovial after by video sequence to be identified processing for human joint points characteristic sequence
Point feature sequence is preserved, and to substitute in the form of original image is preserved, protects passenger and Si to a certain extent
The privacy of machine.Wherein each of human joint points characteristic sequence body joint point feature include single human body included it is all
The location of artis title and all artis coordinate.Human joint points typically refer to the head of human body, shoulder, trunk,
The joint parts such as four limbs, the quantity of human joint points may include 15 artis, 18 artis, 21 artis etc., can
To pre-set the quantity of the required artis extracted according to actual demand, it is not specifically limited herein.
Meanwhile it being based on above-mentioned human joint points characteristic sequence, it can be to each human joint points using action recognition module 2
Human action in characteristic sequence carries out Classification and Identification.In each human joint points characteristic sequence, when two neighboring human body closes
When being had differences between node diagnostic, then it can determine there is human action at this time.Simultaneously in two neighboring human synovial point feature
There are the artis titles of artis change in location for middle determination, you can by human action specific to the movement of some artis, and
According to the specific variation of artis position, you can determine the information such as the direction of motion, moving displacement and the deviation angle of the artis.
If for example, there is only single human bodies in raw video image to be identified, and wherein certain two adjacent moment corresponds to
Raw video image in there are when the headwork of human body, then corresponding two neighboring human joint points of the adjacent two field pictures
Feature can change, and can specifically determine that the position coordinates of joint of head point are changed, and then can determine that human body is deposited at this time
In headwork, while according to the relationship between the front and back position coordinates of joint of head point action, you can determine human body head
The information such as the direction of motion, moving displacement and deviation angle, and then determine the specific type of action of human body head.
A kind of taxi operation behavior act identifying system provided by the invention, by shooting the original video in taxi
Image obtains video sequence to be identified from raw video image, extracts in every frame image of video sequence to be identified and belongs to same
The human synovial point feature of one people obtains at least one human joint points characteristic sequence；To each human joint points characteristic sequence
In human action carry out Classification and Identification, obtain action recognition result；The monitoring to taxi operation behavior effectively is realized, together
When by original image processing be human joint points characteristic sequence, concealed the Human biology feature in original image, effectively protect
Passenger in taxi and the privacy of driver have been protected, and has reduced the storage burden of server to a certain extent, has been conducive to promote clothes
The overall performance of business device.
Fig. 5 shows a kind of structure diagram of the equipment of taxi operation behavior act recognition methods of the embodiment of the present invention.
Reference Fig. 5, the equipment of the taxi operation behavior act recognition methods, including：Processor (processor) 51, memory
(memory) 52 and bus 53；Wherein, the processor 51 and memory 52 complete mutual communication by the bus 53；
The processor 51 is used to call the program instruction in the memory 52, to execute the side that above-mentioned each method embodiment is provided
Method, such as including：The human synovial point feature for belonging to same people in every frame image of video sequence to be identified is extracted, is obtained at least
One human joint points characteristic sequence；Classification and Identification is carried out to the human action in each human joint points characteristic sequence, is obtained
Action recognition result；Wherein human synovial point feature includes artis title and artis position coordinates.
The present embodiment discloses a kind of computer program product, and the computer program product includes being stored in non-transient calculating
Computer program on machine readable storage medium storing program for executing, the computer program include program instruction, when described program instruction is calculated
When machine executes, computer is able to carry out the method that above-mentioned each method embodiment is provided, such as including：Extract video sequence to be identified
The human synovial point feature for belonging to same people in every frame image of row, obtains at least one human joint points characteristic sequence；To every
Human action in a human joint points characteristic sequence carries out Classification and Identification, obtains action recognition result；Wherein human joint points
Feature includes artis title and artis position coordinates.
The present embodiment provides a kind of non-transient computer readable storage medium, the non-transient computer readable storage medium
Computer instruction is stored, the computer instruction makes the computer execute the method that above-mentioned each method embodiment is provided, example
Such as include：The human synovial point feature for belonging to same people in every frame image of video sequence to be identified is extracted, is obtained at least one
Human joint points characteristic sequence；Classification and Identification is carried out to the human action in each human joint points characteristic sequence, is acted
Recognition result；Wherein human synovial point feature includes artis title and artis position coordinates.
One of ordinary skill in the art will appreciate that：Realize that all or part of step of above method embodiment can pass through
The relevant hardware of program instruction is completed, and program above-mentioned can be stored in a computer read/write memory medium, the program
When being executed, step including the steps of the foregoing method embodiments is executed；And storage medium above-mentioned includes：ROM, RAM, magnetic disc or light
The various media that can store program code such as disk.
The embodiments such as the equipment of taxi operation behavior act recognition methods described above are only schematical,
Described in the unit that illustrates as separating component may or may not be physically separated, the portion shown as unit
Part may or may not be physical unit, you can be located at a place, or may be distributed over multiple network lists
In member.Some or all of module therein can be selected according to the actual needs to achieve the purpose of the solution of this embodiment.This
Field those of ordinary skill is not in the case where paying performing creative labour, you can to understand and implement.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It is realized by the mode of software plus required general hardware platform, naturally it is also possible to pass through hardware.Based on this understanding, on
Stating technical solution, substantially the part that contributes to existing technology can be expressed in the form of software products in other words, should
Computer software product can store in a computer-readable storage medium, such as ROM/RAM, magnetic disc, CD, including several fingers
It enables and using so that a computer equipment (can be personal computer, server or the network equipment etc.) executes each implementation
Method described in certain parts of example or embodiment.
Finally, the present processes are only preferable embodiment, are not intended to limit the scope of the present invention.It is all
Within the spirit and principles in the present invention, any modification, equivalent replacement, improvement and so on should be included in the protection of the present invention
Within the scope of.
Claims (10)
1. a kind of taxi operation behavior act recognition methods, which is characterized in that including：
S1 extracts the human synovial point feature for belonging to same people in every frame image of video sequence to be identified, obtains at least one
Human joint points characteristic sequence；
S2 carries out Classification and Identification to the human action in each human joint points characteristic sequence, obtains action recognition result；
The wherein described human joint points feature includes artis title and artis position coordinates.
2. according to the method described in claim 1, it is characterized in that, the step S1 further comprises：
All human joint points extracted using default human skeleton model in every frame image of the video sequence to be identified are special
Sign, and all human joint points features in the image per frame are divided, it obtains in the image per frame and belongs to same
The human synovial point feature of people；
The human synovial point feature for belonging to same people in each frame image is combined, at least one human joint points are obtained
Characteristic sequence.
3. according to the method described in claim 2, it is characterized in that, further including before the step S1：
Video sequence sample is obtained, to belonging to the human synovial point feature of same people in every frame image of the video sequence sample
It is labeled, forms training sample；
The default human skeleton model is trained using the training sample；
Wherein, the video sequence sample includes single video sequence sample and more people's video sequence samples.
4. according to the method described in claim 1, it is characterized in that, the step S2 further comprises：
In each human joint points characteristic sequence, two neighboring human synovial point feature is compared；
When the two neighboring human synovial point feature is inconsistent, the two neighboring human synovial point feature is marked
Note, and Classification and Identification is carried out to human action according to the difference between the two neighboring human synovial point feature, it is acted
Recognition result.
5. according to the method described in claim 1, it is characterized in that, further including after the step S2：
When the action recognition result belongs to default anomaly classification, the video sequence to be identified is marked into abnormal label,
And send pre-warning signal to preset client；
When the action recognition result belongs to default normal classification, the video sequence to be identified is marked into normal tag.
6. according to the method described in claim 1, it is characterized in that, further including after the step S2：
The action recognition result is subjected to data compression, the compressed action recognition result is stored.
7. according to the method described in claim 1, it is characterized in that, further including before the step S1：
Image Acquisition is carried out to human body to be identified from multiple default visual angles, obtains the video sequence to be identified.
8. a kind of taxi operation behavior act identifying system, which is characterized in that including：
Characteristic extracting module, the human joint points that same people is belonged in every frame image for extracting video sequence to be identified are special
Sign, obtains at least one human joint points characteristic sequence；
Action recognition module is obtained for carrying out Classification and Identification to the human action in each human joint points characteristic sequence
Obtain action recognition result；
The wherein described human joint points feature includes artis title and artis position coordinates.
9. a kind of equipment of taxi operation behavior act recognition methods, which is characterized in that including：
At least one processor；And
At least one processor being connect with the processor communication, wherein：
The memory is stored with the program instruction that can be executed by the processor, and the processor calls described program to instruct energy
Enough methods executed as described in claim 1 to 7 is any.
10. a kind of non-transient computer readable storage medium, which is characterized in that the non-transient computer readable storage medium is deposited
Computer instruction is stored up, the computer instruction makes the computer execute the method as described in claim 1 to 7 is any.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810317515.2A CN108596056A (en) | 2018-04-10 | 2018-04-10 | A kind of taxi operation behavior act recognition methods and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810317515.2A CN108596056A (en) | 2018-04-10 | 2018-04-10 | A kind of taxi operation behavior act recognition methods and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108596056A true CN108596056A (en) | 2018-09-28 |
Family
ID=63621786
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810317515.2A Pending CN108596056A (en) | 2018-04-10 | 2018-04-10 | A kind of taxi operation behavior act recognition methods and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108596056A (en) |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109635783A (en) * | 2019-01-02 | 2019-04-16 | 上海数迹智能科技有限公司 | Video monitoring method, device, terminal and medium |
| CN109711343A (en) * | 2018-12-27 | 2019-05-03 | 北京思图场景数据科技服务有限公司 | Behavioral structure method based on the tracking of expression, gesture recognition and expression in the eyes |
| CN110110631A (en) * | 2019-04-25 | 2019-08-09 | 深兰科技（上海）有限公司 | It is a kind of to identify the method and apparatus made a phone call |
| CN110147729A (en) * | 2019-04-16 | 2019-08-20 | 深圳壹账通智能科技有限公司 | User emotion recognition methods, device, computer equipment and storage medium |
| CN110231605A (en) * | 2019-05-09 | 2019-09-13 | 深圳市速腾聚创科技有限公司 | Recognition methods, device, computer equipment and the storage medium of human body behavior |
| CN110705510A (en) * | 2019-10-16 | 2020-01-17 | 上海极链网络科技有限公司 | Action determination method, device, server and storage medium |
| CN111539352A (en) * | 2020-04-27 | 2020-08-14 | 支付宝(杭州)信息技术有限公司 | Method and system for judging human body joint motion direction |
| CN111553320A (en) * | 2020-05-14 | 2020-08-18 | 支付宝(杭州)信息技术有限公司 | Feature extraction method for protecting personal data privacy, model training method and hardware |
| CN112487913A (en) * | 2020-11-24 | 2021-03-12 | 北京市地铁运营有限公司运营四分公司 | Labeling method and device based on neural network and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150186713A1 (en) * | 2013-12-31 | 2015-07-02 | Konica Minolta Laboratory U.S.A., Inc. | Method and system for emotion and behavior recognition |
| CN105787439A (en) * | 2016-02-04 | 2016-07-20 | 广州新节奏智能科技有限公司 | Depth image human body joint positioning method based on convolution nerve network |
| CN106803083A (en) * | 2017-02-04 | 2017-06-06 | 北京旷视科技有限公司 | The method and device of pedestrian detection |
| CN107741784A (en) * | 2017-10-09 | 2018-02-27 | 济南大学 | A kind of amusement exchange method suitable for leaden paralysis patient |
| CN107832708A (en) * | 2017-11-09 | 2018-03-23 | 云丁网络技术(北京)有限公司 | A kind of human motion recognition method and device |
-
2018
- 2018-04-10 CN CN201810317515.2A patent/CN108596056A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150186713A1 (en) * | 2013-12-31 | 2015-07-02 | Konica Minolta Laboratory U.S.A., Inc. | Method and system for emotion and behavior recognition |
| CN105787439A (en) * | 2016-02-04 | 2016-07-20 | 广州新节奏智能科技有限公司 | Depth image human body joint positioning method based on convolution nerve network |
| CN106803083A (en) * | 2017-02-04 | 2017-06-06 | 北京旷视科技有限公司 | The method and device of pedestrian detection |
| CN107741784A (en) * | 2017-10-09 | 2018-02-27 | 济南大学 | A kind of amusement exchange method suitable for leaden paralysis patient |
| CN107832708A (en) * | 2017-11-09 | 2018-03-23 | 云丁网络技术(北京)有限公司 | A kind of human motion recognition method and device |
Cited By (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109711343A (en) * | 2018-12-27 | 2019-05-03 | 北京思图场景数据科技服务有限公司 | Behavioral structure method based on the tracking of expression, gesture recognition and expression in the eyes |
| CN109635783A (en) * | 2019-01-02 | 2019-04-16 | 上海数迹智能科技有限公司 | Video monitoring method, device, terminal and medium |
| CN110147729A (en) * | 2019-04-16 | 2019-08-20 | 深圳壹账通智能科技有限公司 | User emotion recognition methods, device, computer equipment and storage medium |
| CN110110631A (en) * | 2019-04-25 | 2019-08-09 | 深兰科技（上海）有限公司 | It is a kind of to identify the method and apparatus made a phone call |
| CN110110631B (en) * | 2019-04-25 | 2021-06-29 | 深兰科技（上海）有限公司 | Method and equipment for recognizing and making call |
| CN110231605A (en) * | 2019-05-09 | 2019-09-13 | 深圳市速腾聚创科技有限公司 | Recognition methods, device, computer equipment and the storage medium of human body behavior |
| CN110705510A (en) * | 2019-10-16 | 2020-01-17 | 上海极链网络科技有限公司 | Action determination method, device, server and storage medium |
| CN110705510B (en) * | 2019-10-16 | 2023-09-05 | 杭州优频科技有限公司 | Action determining method, device, server and storage medium |
| CN111539352A (en) * | 2020-04-27 | 2020-08-14 | 支付宝(杭州)信息技术有限公司 | Method and system for judging human body joint motion direction |
| CN111553320A (en) * | 2020-05-14 | 2020-08-18 | 支付宝(杭州)信息技术有限公司 | Feature extraction method for protecting personal data privacy, model training method and hardware |
| CN111553320B (en) * | 2020-05-14 | 2021-12-21 | 支付宝(杭州)信息技术有限公司 | Feature extraction method for protecting personal data privacy, model training method and hardware |
| CN112487913A (en) * | 2020-11-24 | 2021-03-12 | 北京市地铁运营有限公司运营四分公司 | Labeling method and device based on neural network and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108596056A (en) | A kind of taxi operation behavior act recognition methods and system | |
| CN106941602B (en) | Locomotive driver behavior identification method and device | |
| US11188783B2 (en) | Reverse neural network for object re-identification | |
| CN108596277B (en) | Vehicle identity recognition method and device and storage medium | |
| CN108388888B (en) | Vehicle identification method and device and storage medium | |
| CN110223511A (en) | A kind of automobile roadside is separated to stop intelligent monitoring method and system | |
| CN108197326A (en) | A kind of vehicle retrieval method and device, electronic equipment, storage medium | |
| CN109102475B (en) | Image rain removing method and device | |
| CN109446936A (en) | A kind of personal identification method and device for monitoring scene | |
| CN109508636A (en) | Vehicle attribute recognition methods, device, storage medium and electronic equipment | |
| CN111091044B (en) | Network appointment-oriented in-vehicle dangerous scene identification method | |
| US20200250401A1 (en) | Computer system and computer-readable storage medium | |
| CN112287875A (en) | Abnormal license plate recognition method, device, equipment and readable storage medium | |
| CN111553247A (en) | Video structuring system, method and medium based on improved backbone network | |
| US20200311962A1 (en) | Deep learning based tattoo detection system with optimized data labeling for offline and real-time processing | |
| CN112434566A (en) | Passenger flow statistical method and device, electronic equipment and storage medium | |
| CN114937293B (en) | GIS-based agricultural service management method and system | |
| CN110443119B (en) | Method and device for identifying state of goods in carriage | |
| CN106940904B (en) | Attendance checking system based on recognition of face and speech recognition | |
| CN110717456A (en) | Object monitoring method, device, system, electronic equipment and storage medium | |
| CN115908831A (en) | Image detection method and device | |
| CN115731179A (en) | Track component detection method, terminal and storage medium | |
| CN109544855B (en) | Computer vision-based closed circuit television system for comprehensively monitoring fire disaster in rail transit and implementation method | |
| CN115311680A (en) | Human body image quality detection method and device, electronic equipment and storage medium | |
| CN114296545A (en) | Unmanned aerial vehicle gesture control method based on vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180928 |