CN114419607A - Method and system for detecting non-standard behaviors in ship cockpit - Google Patents
Method and system for detecting non-standard behaviors in ship cockpit Download PDFInfo
- Publication number
- CN114419607A CN114419607A CN202111504662.9A CN202111504662A CN114419607A CN 114419607 A CN114419607 A CN 114419607A CN 202111504662 A CN202111504662 A CN 202111504662A CN 114419607 A CN114419607 A CN 114419607A
- Authority
- CN
- China
- Prior art keywords
- key point
- driver
- action
- ship
- behaviors
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000006399 behavior Effects 0.000 title claims abstract description 97
- 238000000034 method Methods 0.000 title claims abstract description 44
- 230000009471 action Effects 0.000 claims abstract description 67
- 238000001514 detection method Methods 0.000 claims abstract description 22
- 238000010586 diagram Methods 0.000 claims abstract description 21
- 230000008569 process Effects 0.000 claims description 16
- 230000011218 segmentation Effects 0.000 claims description 15
- 238000013461 design Methods 0.000 claims description 12
- 230000001788 irregular Effects 0.000 claims description 6
- 238000012546 transfer Methods 0.000 claims description 6
- 230000009286 beneficial effect Effects 0.000 abstract description 6
- 230000000007 visual effect Effects 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 239000000284 extract Substances 0.000 description 3
- 238000007689 inspection Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 230000002265 prevention Effects 0.000 description 2
- 206010000117 Abnormal behaviour Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000005764 inhibitory process Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000007704 transition Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides a method and a system for detecting non-standard behaviors in a ship cockpit, wherein the method comprises the steps of obtaining a picture of the ship cockpit through image acquisition equipment and segmenting different driving areas and drivers; identifying the identity of a driver, and detecting key points of the driver by using a skeleton identification network to generate a key point action timing diagram; designing a timing sequence automatic machine recognition model of the driver behaviors, carrying out timing sequence matching on the action timing sequence chart of the key point, and outputting the matched behaviors in real time. Based on the detection method, the invention also provides a system for detecting the non-standard behaviors in the ship cockpit, and the system is based on the difficult problem of identifying the driving behaviors in the ship cockpit of the logic sequence, thereby being beneficial to real-time early warning and stopping of the non-standard behaviors in the ship cockpit, overcoming the dilemma that the traditional action identification system depends on key scenes and key objects, and being capable of respectively tracking and identifying actions of a plurality of drivers in the same scene.
Description
Technical Field
The invention belongs to the technical field of ship driving behavior detection, and particularly relates to a method and a system for detecting non-standard behaviors in a ship cockpit.
Background
The detection of the non-standard behaviors in the ship cockpit has great application in both military and civil fields. A good non-standard behavior detection algorithm in the cockpit can timely detect and early warn the behavior of a driver in the cockpit. Through real-time output and early warning of the algorithm, the driving safety of the ship can be enhanced greatly. Meanwhile, the personal safety of the driver and all crews in the cabin can be guaranteed to a certain extent.
The common identification algorithm for abnormal behaviors in the cabin in the prior art can be divided into two types: recognition algorithms for strong category behavior and recognition algorithms for strong logic behavior. The recognition of strong category behaviors is often accompanied by the appearance of key people, objects, clothing and the like in scenes in a cockpit, and can be recognized in a traditional image semantic understanding mode, so that the solution is easy. The existing recognition algorithm of strong logic behaviors has no key recognition object in the recognition process, mostly depends on the experience knowledge of actions, and the difficulty of recognizing the normative and non-normative behaviors in the cockpit is increased.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method and a system for detecting the non-standard behaviors in a ship cockpit, which solve the problem of identifying the driving behaviors in the ship cockpit based on a logic sequence and are beneficial to real-time early warning and stopping of the non-standard behaviors in the ship cockpit.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method for detecting non-standard behaviors in a ship cockpit comprises the following steps:
acquiring a picture of a ship cab through image acquisition equipment, and dividing different driving areas and drivers;
identifying the identity of a driver, and detecting key points of the driver by using a skeleton identification network to generate a key point action timing diagram;
designing a timing sequence automatic machine recognition model of the driver behaviors, carrying out timing sequence matching on the action timing sequence chart of the key point, and outputting the matched behaviors in real time.
Further, the process of acquiring the picture of the ship cab through the image acquisition device and segmenting different driving areas and drivers comprises the following steps:
acquiring a ship cab image and a driver image through image acquisition equipment arranged in a ship cab;
the cab image is divided into different driving areas by predefined area coordinates, and the contour of the driver is segmented by a pedestrian segmentation algorithm.
Further, the driver is subjected to identity recognition, and the process of detecting key points of the driver by using the skeleton recognition network is as follows:
adopting a pedestrian re-recognition algorithm to carry out personnel matching on a driver, and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven;
and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
Further, the process of generating the key point action timing chart is as follows:
grouping key point sets of different drivers and a priori key point set by utilizing personnel matching information, and matching the priori key point sequence information with a plurality of current human body key point sets; and fusing the key point information of each ship driver with the prior key point sequence information of the current personnel to generate a key point information sequence chart of the driver at the current moment.
Further, the prior key point sequence information is a human body key point information database.
The method for detecting the irregular behaviors in the ship cockpit according to claim 1, wherein the process of designing the time sequence automatic machine recognition model of the driver behaviors comprises the following steps:
breaking down non-canonical actions into a set of sequence of sub-actions
Wherein p isiA number representing a keypoint; j represents a sub-action number; the sub-action is a combination of any position and any atomic behavior; atomic behavior set a ═ a1,a2...an}；
Obtaining the state transfer function of each action time sequence automaton by constructing the sub-action sequence set of each action
Wherein SxAnd SyAre two states belonging to two elements in the state set S.
The invention also provides a system for detecting the non-standard behaviors in the ship cockpit, which comprises a segmentation module, an identification detection module and a design matching module;
the segmentation module is used for acquiring a picture of a ship cab through image acquisition equipment and segmenting different driving areas and drivers;
the detection module is used for identifying the identity of the driver and detecting key points of the driver by using a skeleton recognition network to generate a key point action timing diagram;
the matching module is used for designing a timing sequence automatic machine recognition model of the driver behaviors, performing timing sequence matching on the action timing diagram of the key point and outputting the matched behaviors in real time.
Further, the process executed by the segmentation module is as follows: acquiring a ship cab image and a driver image through image acquisition equipment arranged in a ship cab; the cab image is divided into different driving areas by predefined area coordinates, and the contour of the driver is segmented by a pedestrian segmentation algorithm.
Further, the identification detection module comprises an identification module and a detection module;
the recognition module is used for matching the personnel of the driver by adopting a pedestrian re-recognition algorithm and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven; and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
The detection module is used for grouping the key point sets of different drivers and the prior key point sets by utilizing the personnel matching information and matching the prior key point sequence information with a plurality of current human body key point sets; and fusing the key point information of each ship driver with the prior key point sequence information of the current personnel to generate a key point information sequence chart of the driver at the current moment.
Further, the design matching module comprises a design module and a matching module;
the design module is used for splitting the non-standard actions into a sub-action sequence set and a sub-action sequence set
Wherein p isiA number representing a keypoint; j represents a sub-action number; the sub-action is a combination of any position and any atomic behavior; atomic behavior set a ═ a1,a2...an}; obtaining the state transfer function of each action time sequence automaton by constructing the sub-action sequence set of each action
Wherein SxAnd SyAre two states belonging to two elements in the state set S.
The matching module is used for carrying out time sequence matching on the key point action timing diagram and outputting the matched action in real time.
The effect provided in the summary of the invention is only the effect of the embodiment, not all the effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
the invention provides a method and a system for detecting non-standard behaviors in a ship cockpit, wherein the method comprises the steps of obtaining a picture of the ship cockpit through image acquisition equipment and segmenting different driving areas and drivers; identifying the identity of a driver, and detecting key points of the driver by using a skeleton identification network to generate a key point action timing diagram; designing a timing sequence automatic machine recognition model of the driver behaviors, carrying out timing sequence matching on the action timing sequence chart of the key point, and outputting the matched behaviors in real time. The invention is based on a method for detecting the non-standard behaviors in the ship cockpit, and also provides a system for detecting the non-standard behaviors in the ship cockpit.
The invention extracts the associated information from the input current key point information and the historical key point sequence chart of the drivers in the cabin, and establishes an optimized logic associated function aiming at each action. Therefore, the result can be optimized by itself along with the increase of the input data, and the accuracy of the non-standard action recognition is improved.
The invention simultaneously supports the real-time tracking and identification of a plurality of people in the ship cockpit.
Drawings
Fig. 1 is a flowchart of a method for detecting irregular behaviors in a ship cockpit according to embodiment 1 of the present invention;
fig. 2 is a schematic diagram of a non-canonical action recognition model based on a time-series automaton in embodiment 1 of the present invention;
fig. 3 is a schematic diagram of a system for detecting irregular behaviors in a ship cockpit according to embodiment 2 of the present invention.
Detailed Description
In order to clearly explain the technical features of the present invention, the following detailed description of the present invention is provided with reference to the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different features of the invention. To simplify the disclosure of the present invention, the components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and procedures are omitted so as to not unnecessarily limit the invention.
Example 1
The embodiment 1 of the invention provides a method for detecting non-standard behaviors in a ship cockpit, which is based on a difficult problem of driving behavior recognition in the ship cockpit of a logic sequence and is beneficial to real-time early warning and inhibition of the non-standard behaviors in the ship cockpit.
Fig. 1 shows a flow chart of a method for detecting non-standard behaviors in a ship cockpit according to embodiment 1 of the present invention;
firstly, a picture of a ship cab is obtained through image acquisition equipment, and different driving areas and drivers are segmented.
Acquiring a ship cab image and a driver image through image acquisition equipment arranged in a ship cab;
the cab image is divided into different driving areas by predefined area coordinates, and the contour of the driver is divided by a pedestrian segmentation algorithm. The specific process is as follows: adopting a pedestrian re-recognition algorithm to carry out personnel matching on a driver, and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven; and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
Secondly, identity recognition is carried out on the driver, meanwhile, key point detection is carried out on the driver by using a skeleton recognition network, and a key point action sequence diagram is generated.
Adopting a pedestrian re-recognition algorithm to carry out personnel matching on a driver, and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven;
and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
The process of generating the key point action time sequence chart comprises the following steps:
grouping key point sets of different drivers and a priori key point set by utilizing personnel matching information, and matching the priori key point sequence information with a plurality of current human body key point sets; and fusing the key point information of each ship driver with the prior key point sequence information of the current personnel to generate a key point information sequence chart of the driver at the current moment. Wherein the prior key point sequence information is a human body key point information database.
And thirdly, designing a time sequence automatic machine recognition model of the driver behaviors, performing time sequence matching on the action timing chart of the key point, and outputting the matched behaviors in real time. Fig. 2 is a schematic diagram of a non-canonical action recognition model based on a time-series automaton in embodiment 1 of the present invention.
Constructing time sequence automatic recognition model of different behaviors (including normative behaviors and non-normative behaviors)
For any definable non-canonical action, the non-canonical action is decomposed into a set of sub-action sequences
Wherein p isiA number representing a keypoint; j represents a sub-action number; the left eye, the right palm, the left shoulder and the like of the human body can be arbitrarily marked by the human body. A sub-action may be a combination of any one location and any one atomic behavior. Atomic behavior set a ═ a1,a2...an}; each element in the set can be any one of the atomic actions such as "interval
2 seconds shaking up and down, "hold still," and so forth. By constructing a set of sub-action sequences for each action, the state transition function of the action time sequence automaton can be obtained. Obtaining the state transfer function of each action time sequence automaton by constructing the sub-action sequence set of each action
Wherein SxAnd SyAre two states belonging to two elements in the state set S. There are two special elements S in the state set ScorAnd SerrRespectively representing whether the current state of the ship pilot is a normative state or a non-normative state.
The scene facing the ship cockpit is explained by taking the action sequence of 'periodic inspection instrument dial' as an example:
the definition inspection instrument panel is mainly divided into three action logic sequences:
after the cabin driver lowers the head to observe the equipment, the driver must look over the head immediately to look over the distance
The cabin driver lowers the head for a long time and does not raise the head, the violation is considered, and the time can be configured
After the cabin driver lowers the head to observe the equipment, if the head raises and the right looks ahead without holding, the driver is considered as violation
In an algorithmic implementation, we split this sequence of actions into three sub-actions: and a visual device for visual observation of the front and other directions. A normal logical sequence of actions is when a "visual device" is present, followed by a series of "visual front" actions. If the action of 'looking at other directions', the violation is judged.
And finally, respectively inputting the key point information sequence diagrams of different drivers at the moment into different time sequence automatic machine recognition models, and outputting the current behavior states of the different ship drivers.
The method for detecting the non-standard behaviors in the ship cockpit, which is provided by the embodiment 1 of the invention, is based on the difficult problem of identifying the driving behaviors in the ship cockpit of a logic sequence, is beneficial to real-time early warning and prevention of the non-standard behaviors in the ship cockpit, overcomes the dilemma that the traditional action recognition system depends on key scenes and key objects, and can respectively track and recognize actions of a plurality of drivers in the same scene.
The method for detecting the non-standard behaviors in the ship cockpit provided by the embodiment 1 of the invention extracts the association information from the input current key point information and the historical key point sequence diagram of the pilot in the ship cockpit, and establishes an optimized logic association function for each action. Therefore, the result can be optimized by itself along with the increase of the input data, and the accuracy of the non-standard action recognition is improved.
The method for detecting the non-standard behaviors in the ship cockpit provided by the embodiment 1 of the invention simultaneously supports the real-time tracking and identification of multiple persons in the ship cockpit.
Example 2
Based on the method for detecting the non-standard behavior in the ship cockpit, which is provided by the embodiment 1 of the invention, the embodiment 2 of the invention also provides a system for detecting the non-standard behavior in the ship cockpit. Fig. 3 is a schematic diagram of a system for detecting irregular behaviors in a ship cockpit according to embodiment 2 of the present invention, where the system includes a segmentation module, an identification detection module, and a design matching module;
the segmentation module is used for acquiring a picture of a ship cab through image acquisition equipment and segmenting different driving areas and drivers;
the detection module is used for identifying the identity of the driver and detecting key points of the driver by using the skeleton recognition network to generate a key point action timing diagram;
the matching module is used for designing a timing sequence automatic machine recognition model of the driver behavior, performing timing sequence matching on the action timing diagram of the key point, and outputting the matched behavior in real time.
The process executed by the segmentation module is as follows: acquiring a ship cab image and a driver image through image acquisition equipment arranged in a ship cab; the cab image is divided into different driving areas by predefined area coordinates, and the contour of the driver is segmented by a pedestrian segmentation algorithm.
The identification detection module comprises an identification module and a detection module;
the recognition module is used for matching the personnel of the driver by adopting a pedestrian re-recognition algorithm and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven; and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
The detection module is used for grouping the key point sets of different drivers and the prior key point sets by utilizing the personnel matching information and matching the prior key point sequence information with a plurality of current human body key point sets; and fusing the key point information of each ship driver with the prior key point sequence information of the current personnel to generate a key point information sequence chart of the driver at the current moment.
The design matching module comprises a design module and a matching module;
the design module is used for splitting the non-standard action into a sub-action sequence set
Wherein p isiA number representing a keypoint; j represents a sub-action number; the sub-action is a combination of any position and any atomic behavior; atomic behavior set a ═ a1,a2...an}; obtaining the state transfer function of each action time sequence automaton by constructing the sub-action sequence set of each action
Wherein SxAnd SyAre two states belonging to two elements in the state set S.
And the matching module is used for performing time sequence matching on the key point action timing diagram and outputting the matched action in real time.
The system for detecting the non-standard behaviors in the ship cockpit, which is provided by the embodiment 2 of the invention, is based on the difficult problem of identifying the driving behaviors in the ship cockpit of a logic sequence, is beneficial to real-time early warning and prevention of the non-standard behaviors in the ship cockpit, overcomes the dilemma that the traditional action identification system depends on key scenes and key objects, and can respectively track and identify actions of a plurality of drivers in the same scene.
The system for detecting the non-standard behavior in the ship cockpit, which is provided by the embodiment 2 of the invention, extracts the association information from the input current key point information and the historical key point sequence diagram of the pilot in the ship cockpit, and establishes an optimized logic association function for each action. Therefore, the result can be optimized by itself along with the increase of the input data, and the accuracy of the non-standard action recognition is improved.
The nonstandard behavior detection system in the ship cockpit simultaneously supports real-time tracking and identification of multiple persons in the ship cockpit.
It is noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Furthermore, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include elements inherent in the list. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element. In addition, parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of corresponding technical solutions in the prior art, are not described in detail so as to avoid redundant description.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, the scope of the present invention is not limited thereto. Various modifications and alterations will occur to those skilled in the art based on the foregoing description. And are neither required nor exhaustive of all embodiments. On the basis of the technical scheme of the invention, various modifications or changes which can be made by a person skilled in the art without creative efforts are still within the protection scope of the invention.
Claims (10)
1. A method for detecting non-standard behaviors in a ship cockpit is characterized by comprising the following steps:
acquiring a picture of a ship cab through image acquisition equipment, and dividing different driving areas and drivers;
identifying the identity of a driver, and detecting key points of the driver by using a skeleton identification network to generate a key point action timing diagram;
designing a timing sequence automatic machine recognition model of the driver behaviors, carrying out timing sequence matching on the action timing sequence chart of the key point, and outputting the matched behaviors in real time.
2. The method as claimed in claim 1, wherein the process of obtaining the picture of the ship cab and dividing different driving areas and drivers by the image acquisition device comprises:
acquiring a ship cab image and a driver image through image acquisition equipment arranged in a ship cab;
the cab image is divided into different driving areas by predefined area coordinates, and the contour of the driver is segmented by a pedestrian segmentation algorithm.
3. The method for detecting the non-standard behaviors in the ship cockpit according to claim 2, wherein the process of identifying the identity of the driver and simultaneously detecting the key points of the driver by using the skeleton recognition network comprises the following steps:
adopting a pedestrian re-recognition algorithm to carry out personnel matching on a driver, and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven;
and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
4. The method for detecting the irregular behaviors in the ship cockpit according to claim 3, wherein the process of generating the action time sequence chart of the key point comprises the following steps:
grouping key point sets of different drivers and a priori key point set by utilizing personnel matching information, and matching the priori key point sequence information with a plurality of current human body key point sets; and fusing the key point information of each ship driver with the prior key point sequence information of the current personnel to generate a key point information sequence chart of the driver at the current moment.
5. The method according to claim 4, wherein the prior keypoint sequence information is a human body keypoint information database.
6. The method for detecting the irregular behaviors in the ship cockpit according to claim 1, wherein the process of designing the time sequence automatic machine recognition model of the driver behaviors comprises the following steps:
breaking down non-canonical actions into a set of sequence of sub-actions
Wherein p isiA number representing a keypoint; j represents a sub-action number; the sub-action is a combination of any position and any atomic behavior; atomic behavior set a ═ a1,a2...an}；
Obtaining a state transfer function delta of each action time sequence automaton by constructing a sub-action sequence set of each action:
wherein SxAnd SyIs two states, belonging to a state setTwo elements in S.
7. A detection system for non-standard behaviors in a ship cockpit is characterized by comprising a segmentation module, an identification detection module and a design matching module;
the segmentation module is used for acquiring a picture of a ship cab through image acquisition equipment and segmenting different driving areas and drivers;
the detection module is used for identifying the identity of the driver and detecting key points of the driver by using a skeleton recognition network to generate a key point action timing diagram;
the matching module is used for designing a timing sequence automatic machine recognition model of the driver behaviors, performing timing sequence matching on the action timing diagram of the key point and outputting the matched behaviors in real time.
8. The system according to claim 7, wherein the segmentation module performs the following process: acquiring a ship cab image and a driver image through image acquisition equipment arranged in a ship cab; the cab image is divided into different driving areas by predefined area coordinates, and the contour of the driver is segmented by a pedestrian segmentation algorithm.
9. The system of claim 7, wherein the identification and detection module comprises an identification module and a detection module;
the recognition module is used for matching the personnel of the driver by adopting a pedestrian re-recognition algorithm and determining a standard behavior recognition model set and a non-standard behavior recognition model set which need to be driven; and recognizing skeleton models of different drivers by using a human body skeleton recognition algorithm, and extracting key points from the skeleton models to construct a plurality of human body key point sets.
The detection module is used for grouping the key point sets of different drivers and the prior key point sets by utilizing the personnel matching information and matching the prior key point sequence information with a plurality of current human body key point sets; and fusing the key point information of each ship driver with the prior key point sequence information of the current personnel to generate a key point information sequence chart of the driver at the current moment.
10. The system of claim 7, wherein the design matching module comprises a design module and a matching module;
the design module is used for splitting the non-standard actions into a sub-action sequence set and a sub-action sequence set
Wherein p isiA number representing a keypoint; j represents a sub-action number; the sub-action is a combination of any position and any atomic behavior; atomic behavior set a ═ a1,a2...an}; obtaining a state transfer function delta of each action time sequence automaton by constructing a sub-action sequence set of each action:
wherein SxAnd SyAre two states belonging to two elements in the state set S.
The matching module is used for carrying out time sequence matching on the key point action timing diagram and outputting the matched action in real time.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111504662.9A CN114419607A (en) | 2021-12-08 | 2021-12-08 | Method and system for detecting non-standard behaviors in ship cockpit |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111504662.9A CN114419607A (en) | 2021-12-08 | 2021-12-08 | Method and system for detecting non-standard behaviors in ship cockpit |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114419607A true CN114419607A (en) | 2022-04-29 |
Family
ID=81264606
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111504662.9A Pending CN114419607A (en) | 2021-12-08 | 2021-12-08 | Method and system for detecting non-standard behaviors in ship cockpit |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114419607A (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019047593A1 (en) * | 2017-09-05 | 2019-03-14 | 百度在线网络技术（北京）有限公司 | Method and device for processing automatic driving training data |
| US20200089977A1 (en) * | 2018-09-17 | 2020-03-19 | Honda Motor Co., Ltd. | Driver behavior recognition and prediction |
| CN111428699A (en) * | 2020-06-10 | 2020-07-17 | 南京理工大学 | Driving fatigue detection method and system combining pseudo-3D convolutional neural network and attention mechanism |
| CN111553209A (en) * | 2020-04-15 | 2020-08-18 | 同济大学 | Driver behavior identification method based on convolutional neural network and time sequence diagram |
| CN112329689A (en) * | 2020-11-16 | 2021-02-05 | 北京科技大学 | Abnormal driving behavior identification method based on graph convolution neural network under vehicle-mounted environment |
-
2021
- 2021-12-08 CN CN202111504662.9A patent/CN114419607A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019047593A1 (en) * | 2017-09-05 | 2019-03-14 | 百度在线网络技术（北京）有限公司 | Method and device for processing automatic driving training data |
| US20200089977A1 (en) * | 2018-09-17 | 2020-03-19 | Honda Motor Co., Ltd. | Driver behavior recognition and prediction |
| CN111553209A (en) * | 2020-04-15 | 2020-08-18 | 同济大学 | Driver behavior identification method based on convolutional neural network and time sequence diagram |
| CN111428699A (en) * | 2020-06-10 | 2020-07-17 | 南京理工大学 | Driving fatigue detection method and system combining pseudo-3D convolutional neural network and attention mechanism |
| CN112329689A (en) * | 2020-11-16 | 2021-02-05 | 北京科技大学 | Abnormal driving behavior identification method based on graph convolution neural network under vehicle-mounted environment |
Non-Patent Citations (1)
| Title |
|---|
| 朱玉斌;延向军;申旭奇;卢兆林;: "基于级联宽度学习的疲劳驾驶检测", 计算机工程与设计, no. 02, 16 February 2020 (2020-02-16), pages 245 - 249 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Islam et al. | Real time hand gesture recognition using different algorithms based on American sign language | |
| Yan et al. | Video-based classification of driving behavior using a hierarchical classification system with multiple features | |
| CN104637246A (en) | Driver multi-behavior early warning system and danger evaluation method | |
| Choi et al. | Driver drowsiness detection based on multimodal using fusion of visual-feature and bio-signal | |
| Kalsh et al. | Sign language recognition system | |
| CN114005167A (en) | Remote sight estimation method and device based on human skeleton key points | |
| Yu et al. | AR marker aided obstacle localization system for assisting visually impaired | |
| Li et al. | Distracted driving detection by combining ViT and CNN | |
| Yi et al. | Safety belt wearing detection algorithm based on human joint points | |
| Sultana et al. | Vision based gesture recognition for alphabetical hand gestures using the SVM classifier | |
| Li et al. | A review on vision-based pedestrian detection for intelligent vehicles | |
| Ju et al. | Vision-based moving objects detection for intelligent automobiles and a robustness enhancing method | |
| CN114419607A (en) | Method and system for detecting non-standard behaviors in ship cockpit | |
| CN112069898A (en) | Method and device for recognizing human face group attribute based on transfer learning | |
| CN111694980A (en) | Robust family child learning state visual supervision method and device | |
| Jadhav et al. | GoogLeNet application towards gesture recognition for ASL character identification | |
| Kumar et al. | Traffic sign and drowsiness detection using open-cv | |
| Hwu et al. | Matching Representations of Explainable Artificial Intelligence and Eye Gaze for Human-Machine Interaction | |
| CN115147921A (en) | Key area target abnormal behavior detection and positioning method based on multi-domain information fusion | |
| Luo | The Application and Development of Image Recognition Technology in the Field of Computer Vision | |
| Feng et al. | A distracted driving discrimination method based on the facial feature triangle and Bayesian network | |
| US20240242378A1 (en) | In-cabin monitoring method and related pose pattern categorization method | |
| Cho et al. | Hierarchical spatial object detection for atm vandalism surveillance | |
| Uke et al. | A Vision based Driver Support System for Road Sign Detection | |
| Evstafev et al. | Controlling driver behaviour in ADAS with emotions recognition system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |