CN115294649B - Method, apparatus, storage medium and processor for identifying behavior using mobile device - Google Patents
Method, apparatus, storage medium and processor for identifying behavior using mobile device Download PDFInfo
- Publication number
- CN115294649B CN115294649B CN202210923201.3A CN202210923201A CN115294649B CN 115294649 B CN115294649 B CN 115294649B CN 202210923201 A CN202210923201 A CN 202210923201A CN 115294649 B CN115294649 B CN 115294649B
- Authority
- CN
- China
- Prior art keywords
- rectangular area
- head
- mobile device
- area
- rectangular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/161—Detection; Localisation; Normalisation
Landscapes
- Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application provides a method, a device, a processor and a storage medium for identifying behavior of using mobile equipment. Comprising the following steps: collecting video data of a fisheye camera to obtain a plurality of video frames, inputting the video frames into a head detection algorithm model, detecting a rectangular region containing a head, inputting the rectangular region into an identification model, determining a first rectangular region with a mobile device behavior, adjusting the focal length of the fisheye camera to obtain a second rectangular region, inputting the second rectangular region into the mobile device detection model and a hand detection model, determining a distance between the mobile device and the head of a user in the second rectangular region under the condition that the mobile device and the hand detection model are simultaneously included, determining an overlapping area between the region of the mobile device and the region of the hand of the user under the condition that the distance is smaller than a distance threshold, and determining the behavior of the user using the mobile device in the second rectangular region under the condition that the overlapping area is larger than the overlapping threshold.
Description
Technical Field
The present application relates to the field of computer technology, and in particular, to a method, an apparatus, a storage medium, and a processor for identifying behavior using a mobile device.
Background
The identification of making a call has very important application scenes in the field of security and protection, such as gas stations, oil refineries, oil houses, oil fields and cruise ships, petroleum and natural gas in the sites have volatility, and electric sparks generated by making a call are likely to ignite volatile gas, so that fire disaster is caused, immeasurable consequences are caused, and serious potential safety hazards are often accompanied.
In the prior art, the method for identifying the calling behavior mainly monitors the behavior of a driver in real time, and judges whether a pedestrian is calling or not through comprehensive analysis of images. Common traditional methods and procedures are generally: the face is detected firstly, then the characteristics are extracted, and finally a classifier is trained. The method has limitation on the recognition range of the phone calling behavior, is difficult to recognize in the global range, can not accurately recognize the phone calling behavior of the hand phone placed on the head, and is easy to generate false alarm for the similar phone calling behavior of the hand phone touching.
Disclosure of Invention
An object of an embodiment of the application is to provide a method, a device, a storage medium and a processor for identifying behavior of a mobile device.
To achieve the above object, a first aspect of the present application provides a method for identifying a behavior using a mobile device, including:
collecting video data of a fisheye camera to obtain a plurality of video frames included in the video data;
inputting each video frame into a head detection algorithm model in sequence, so as to detect a head rectangular area containing the head of a user in each video frame through the head detection algorithm model;
inputting each head rectangular area into a recognition model to determine that a first rectangular area using the mobile device behavior exists in each head rectangular area through the recognition model;
determining a target fisheye camera corresponding to the first rectangular area, and adjusting the focal length of the target fisheye camera to amplify the first rectangular area to obtain a corresponding second rectangular area;
inputting the second rectangular region into a mobile device detection model and a hand detection model;
judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model;
judging the overlapping area between the area where the mobile equipment is located and the area where the human hand is located under the condition that the interval distance is smaller than the distance threshold value;
in the case where the overlapping area is greater than the overlapping threshold, it is determined that there is a behavior of using the mobile device by the user of the second rectangular area.
In the embodiment of the application, N continuous video frames of the same fisheye camera are sequentially input into a head detection algorithm model, so that a head rectangular area containing a user head in each video frame is detected through the head detection algorithm model; the method further comprises the steps of: steps S3 to S8 are performed for each of N consecutive video frames; in the case where the overlapping area of the N consecutive video frames is greater than the overlapping threshold, it is determined that a user included in the N consecutive video frames has a behavior of using the mobile device.
In an embodiment of the present application, first region coordinates of a first rectangular region are determined; determining a region center point of a first rectangular region according to the first region coordinates; and adjusting the focal length of the target fisheye camera to amplify the first rectangular region to obtain a corresponding second rectangular region.
In an embodiment of the present application, determining a region height of a first rectangular region according to a first region coordinate; acquiring an initial focal length and picture height pixels of a fisheye camera; determining the zoom multiple of the fisheye camera according to the picture height pixels and the region height; determining a target zoom focal length of the fisheye camera according to the zoom multiple and the initial focal length; and adjusting the focal length of the fisheye camera to the target zoom focal length so as to enlarge the first rectangular area to obtain a corresponding second rectangular area.
In the embodiment of the application, determining a second area coordinate corresponding to the enlarged first rectangular area according to the zoom multiple and the first area coordinate; and determining the rectangular area corresponding to the second area coordinate as a second rectangular area.
In an embodiment of the present application, it is determined that no user has an action of using the mobile device if any of the following is satisfied: inputting each head rectangular area into a recognition model, wherein the recognition model does not detect that any head rectangular area has a mobile device using behavior; the mobile equipment is included in the second rectangular area which is not detected by the mobile equipment detection model, and/or the human hand is included in the second rectangular area which is not detected by the hand detection model; the separation distance is greater than or equal to a distance threshold; the overlap area is greater than or equal to the overlap threshold.
In an embodiment of the present application, an alarm mechanism is triggered in case it is determined that the user of the second rectangular area is present to use the mobile device.
A second aspect of the present application provides an apparatus for identifying usage behavior of a mobile device, comprising: the data acquisition module is used for acquiring video data of the fisheye camera to obtain a plurality of video frames included in the video data; adjusting the focal length of the fisheye camera to amplify the first rectangular region to obtain a corresponding second rectangular region; the first global judging module is used for inputting each video frame into the head detection algorithm model in sequence so as to detect a head rectangular area containing the head of the user in each video frame through the head detection algorithm model; the second global judging module is used for inputting each head rectangular area into the recognition model so as to determine that a first rectangular area using the mobile equipment behavior exists in each head rectangular area through the recognition model; the first local judging module is used for inputting the second rectangular area into the mobile equipment detection model and the hand detection model; judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model; the second local judging module is used for judging the overlapping area between the area where the mobile equipment is located and the area where the human hand is located under the condition that the interval distance is smaller than the distance threshold value; and the behavior prediction module is used for judging that the user in the second rectangular area has the behavior of using the mobile device under the condition that the overlapping area is larger than the overlapping threshold value.
A third aspect of the present application provides a machine-readable storage medium having stored thereon instructions which, when executed by a processor, cause the processor to be configured to perform the above-described method of identifying usage mobile device behavior.
A fourth aspect of the present application provides a processor configured to perform the above-described method of identifying usage mobile device behavior.
According to the method for recognizing the behavior of the using mobile device, the video data of the fisheye camera are collected to obtain a plurality of video frames included in the video data, each video frame is sequentially input to the head detection algorithm model, a head rectangular area containing the head of the user in each video frame is detected through the head detection algorithm model, each head rectangular area is input to the recognition model, a first rectangular area using the behavior of the mobile device is determined to exist in each head rectangular area through the recognition model, the focal length of the fisheye camera is adjusted to amplify the first rectangular area to obtain a corresponding second rectangular area, the second rectangular area is input to the mobile device detection model and the hand detection model, when the mobile device detection model and the hand detection model determine that the second rectangular area simultaneously comprise the mobile device and the hand of the human body, the distance between the mobile device and the head of the user is judged, when the distance between the area where the mobile device is located and the area where the hand of the human body is located is judged to be smaller than the distance threshold, and when the overlapping area is larger than the overlapping threshold, the behavior of the user using the mobile device in the second rectangular area is judged to exist. By the method, the call making action can be identified in all directions, the identification accuracy is improved, the occurrence of accidents is reduced, and the life and property safety of people is ensured.
Additional features and advantages of embodiments of the present application will be set forth in the detailed description that follows.
Drawings
The accompanying drawings are included to provide a further understanding of embodiments of the present application and are incorporated in and constitute a part of this specification, illustrate embodiments of the present application and together with the description serve to explain, without limitation, the embodiments of the present application. In the drawings:
FIG. 1 schematically illustrates a flow diagram of a method of identifying usage mobile device behavior in accordance with an embodiment of the present application;
FIG. 2 schematically illustrates a flow diagram of a method of identifying usage mobile device behavior according to another embodiment of the present application;
FIG. 3 schematically illustrates a block diagram of an apparatus for identifying usage of mobile device behavior in accordance with an embodiment of the present application;
fig. 4 schematically shows an internal structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
For the purposes of making the objects, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it should be understood that the specific implementations described herein are only for illustrating and explaining the embodiments of the present application, and are not intended to limit the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present application based on the embodiments herein.
Fig. 1 schematically shows a flow diagram of a method of identifying usage mobile device behavior according to an embodiment of the present application. As shown in fig. 1, in an embodiment of the present application, there is provided a method for identifying a behavior using a mobile device, including the steps of:
Step 103, inputting each head rectangular area into the recognition model to determine that a first rectangular area using the mobile device behavior exists in each head rectangular area through the recognition model.
And 104, adjusting the focal length of the fisheye camera to enlarge the first rectangular area to obtain a corresponding second rectangular area.
And step 106, judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model.
And step 107, judging the overlapping area between the area where the mobile device is located and the area where the human hand is located under the condition that the interval distance is smaller than the distance threshold value.
The camera may use a fisheye camera, a wide angle camera, or the like, through which video data is acquired, the video data including a plurality of video frames, the video frames being a measure for measuring the number of display frames. The frame rate of the video is generally 25 frames/second, and the video is acquired in a frame extraction mode, wherein 5 frames are acquired per second. The head detection algorithm model comprises a head detection algorithm model of yolov3 and yolov5, is used for detecting the region where the head appears, inputs each video frame into the head detection algorithm model, detects the head rectangular region containing the head of the user in each video frame through the head detection algorithm model, and acquires the coordinate values (x 1, y1, x2 and y 2) of the head rectangular region. Inputting each obtained head rectangular area into a recognition model, determining whether the behavior of using the mobile equipment exists in each head rectangular area through the recognition model, terminating recognition if the behavior of using the mobile equipment does not exist, and determining the head rectangular area as a first rectangular area if the behavior of using the mobile equipment exists. Adjusting the focal length of the fisheye camera, keeping the head area to appear in the center of the fisheye camera picture, amplifying the head area to 1/2 higher than the picture height to obtain a corresponding second rectangular area, wherein the coordinates of the second rectangular area are (x 1, y1, x2 and y 2), and calculating x1, y1, x2, y2, x1 = x 1/zoom multiple, y1 = y 1/zoom multiple, x2 = x2 x zoom multiple and y2 = y2 x zoom multiple according to the zoom multiple adjusted by the fisheye camera. And inputting the second rectangular region into a mobile device detection model and a hand detection model, wherein the mobile device detection model can detect the specific position of the mobile device in the image according to the image, and the hand detection model can detect the specific position of the hand in the image according to the image. Under the condition that the mobile equipment and the human hand are simultaneously included in the second rectangular area through the mobile equipment detection model and the hand detection model, acquiring coordinates of the mobile equipment in the second rectangular area as (phone_x1, phone_y1, phone_x2, phone_y2), acquiring coordinates of the human hand in the second rectangular area as (hand_x1, hand_y1, hand_x2, hand_y2), calculating a distance between the mobile equipment and the user head in the second rectangular area according to the acquired coordinates of the mobile equipment in the second rectangular area and the acquired coordinates of the user head in the second rectangular area, calculating an overlapping area between the area of the mobile equipment and the area of the human hand in the second rectangular area according to the acquired coordinates of the mobile equipment in the second rectangular area and the acquired coordinates of the human hand in the second rectangular area, and judging that the overlapping area between the area of the mobile equipment and the area of the human hand in the second rectangular area is larger than the second rectangular area exists when the acquired coordinates of the mobile equipment in the second rectangular area and the human hand in the second rectangular area is larger than the threshold, and the mobile equipment is used. By the method, the call making action can be identified in all directions, the identification accuracy is improved, the occurrence of accidents is reduced, and the life and property safety of people is ensured.
As shown in fig. 2, fig. 2 is a flow chart of a method for identifying the behavior of using a mobile device, capturing video data of a fisheye camera, and sequentially inputting each captured video frame into a head detection algorithm model to detect a head rectangular region containing a user head in each video frame through the head detection algorithm model. And inputting each head rectangular area into the recognition model, judging whether the mobile equipment using behavior exists in each head rectangular area through the recognition model, if the mobile equipment using behavior does not exist, continuing to collect video data of the fish-eye camera, and if the mobile equipment using behavior exists, determining the area as a first rectangular area. And adjusting the focal length of the fisheye camera to amplify the first rectangular region to obtain a corresponding second rectangular region, inputting the second rectangular region into the mobile device detection model and the hand detection model, and determining whether the mobile device and the human hand are simultaneously included in the second rectangular region through the mobile device detection model and the hand detection model. If the second rectangular area does not simultaneously comprise the mobile equipment and the human hand, continuing to collect video data of the fish-eye camera, and if the second rectangular area simultaneously comprises the mobile equipment and the human hand, judging whether the overlapping area of the mobile phone and the human hand is larger than or equal to a set threshold value. If the overlapping area of the mobile phone and the human hand is smaller than the set threshold value, continuing to collect video data of the fish-eye camera, and if the overlapping area of the mobile phone and the human hand is larger than or equal to the set threshold value, judging that the user in the second rectangular area has the action of using the mobile device. Judging whether the continuous multiframe recognizes that the user has the action of using the mobile equipment, if the continuous multiframe recognizes that the user has the action of using the mobile equipment, judging that the user has the action of using the mobile equipment, triggering an alarm device, and if the continuous multiframe does not recognize that the user has the action of using the mobile equipment, continuing to collect video data of the fisheye camera, and repeating the operation.
In one embodiment, N continuous video frames of the same fisheye camera are sequentially input to a head detection algorithm model, so that a head rectangular area containing a user head in each video frame is detected through the head detection algorithm model; the method further comprises the steps of: steps S3 to S8 are performed for each of N consecutive video frames; in the case where the overlapping area of the N consecutive video frames is greater than the overlapping threshold, it is determined that a user included in the N consecutive video frames has a behavior of using the mobile device.
Video data are collected through a camera of the fisheye camera, the video data collected through the camera comprise video frames, N continuous video frames of the same fisheye camera are input into the video frames, and a head rectangular area containing a user head in each video frame is detected through a head detection algorithm model. Inputting each obtained head rectangular area into a recognition model, determining whether the behavior of using the mobile equipment exists in each head rectangular area through the recognition model, terminating recognition if the behavior of using the mobile equipment does not exist, and determining the head rectangular area as a first rectangular area if the behavior of using the mobile equipment exists. And adjusting the focal length of the fisheye camera to obtain a corresponding second rectangular region. And inputting the second rectangular region into a mobile device detection model and a hand detection model, wherein the mobile device detection model can detect the specific position of the mobile device in the image according to the image, and the hand detection model can detect the specific position of the hand in the image according to the image. And when the overlapping area between the area of the mobile device in the second rectangular area and the area of the human hand in the second rectangular area is larger than the overlapping threshold value, judging that the user in the second rectangular area has the action of using the mobile device, repeatedly executing the method and determining the action of using the mobile device for the user contained in N continuous video frames.
In one embodiment, a first area coordinate of the first rectangular area is determined, an area center point of the first rectangular area is determined according to the first area coordinate, and a focal length of the fisheye camera is adjusted according to the center point to enlarge the first rectangular area to obtain a corresponding second rectangular area.
Video data are collected through a camera of the fisheye camera, the video data collected through the camera comprise video frames, N continuous video frames of the same fisheye camera are input into the head rectangular area, which contains the head of a user, in each video frame is detected through a head detection algorithm model, and coordinate values (x 1, y1, x2 and y 2) of the head rectangular area are obtained. Inputting each obtained head rectangular area into a recognition model, determining whether the behavior of using the mobile equipment exists in each head rectangular area through the recognition model, and if the behavior of using the mobile equipment exists, determining the head rectangular area as a first rectangular area. And determining a region center point of the first rectangular region according to the first region coordinates, wherein the center point coordinates are ((x 1+ x 2)/2, (y 1+ y 2)/2), adjusting the focal length of the fisheye camera according to the center point, keeping the center of a picture of the fisheye camera in the head region, and amplifying the head region to 1/2 of the height of the picture to obtain a corresponding second rectangular region.
In one embodiment, the zone height of the first rectangular zone is determined from the first zone coordinates; acquiring an initial focal length and picture height pixels of a fisheye camera; determining the zoom multiple of the fisheye camera according to the picture height pixels and the region height; determining a target zoom focal length of the fisheye camera according to the zoom multiple and the initial focal length; and adjusting the focal length of the fisheye camera to the target zoom focal length so as to enlarge the first rectangular area to obtain a corresponding second rectangular area.
Determining a region height of the first rectangular region from the first region coordinates (x 1, y1, x2, y 2); obtaining an initial focal length F0 and a picture height pixel H0 of the fisheye camera, calculating the height of a head region, determining a zoom multiple of the fisheye camera according to the picture height pixel and the region height according to the following formula (1), determining a target zoom focal length of the fisheye camera according to the following formula (2), the zoom multiple= (H0/2)/H0 (2), determining a target zoom focal length of the fisheye camera according to the zoom multiple and the initial focal length, and adjusting the focal length of the fisheye camera to the target zoom focal length according to the following formula (3), wherein the zoom focal length = the zoom multiple x the initial focal length F0 (3), so as to amplify the first rectangular region to obtain a corresponding second rectangular region.
In one embodiment, determining a second region coordinate corresponding to the enlarged first rectangular region according to the zoom multiple and the first region coordinate; and determining the rectangular area corresponding to the second area coordinate as a second rectangular area.
The coordinates of the second rectangular area are (x 1, y1, x2, y 2), x1, y1, x2, y2 are calculated according to the zoom magnification adjusted by the fisheye camera, x1 = x 1/zoom magnification (4), y1 = y 1/zoom magnification (5), x2 = x2 x zoom magnification (6), y2 = y2 x zoom magnification (7) according to formulas (4), (5), (6), (7). And determining the rectangular area corresponding to the second area coordinate as a second rectangular area.
In one embodiment, it is determined that no user is present in the act of using the mobile device if any of the following is satisfied: inputting each head rectangular area into a recognition model, wherein the recognition model does not detect that any head rectangular area has a mobile device using behavior; the mobile equipment is included in the second rectangular area which is not detected by the mobile equipment detection model, and/or the human hand is included in the second rectangular area which is not detected by the hand detection model; the separation distance is greater than or equal to a distance threshold; the overlap area is less than or equal to the overlap threshold.
And inputting each video frame into a head detection algorithm model through video data acquired by a camera, detecting a head rectangular area containing the head of a user in each video frame through the head detection algorithm model, inputting each head rectangular area into a recognition model, and judging that no mobile equipment using behavior exists in any head rectangular area if the recognition model does not detect the mobile equipment using behavior. If the recognition model detects that the mobile equipment using behavior exists in the head rectangular area, determining that a first rectangular area using the mobile equipment behavior exists in each head rectangular area through the recognition model, adjusting the focal length of the fisheye camera to enlarge the first rectangular area to obtain a corresponding second rectangular area, inputting the second rectangular area into the mobile equipment detection model and the hand detection model, and if the mobile equipment detection model does not detect that the mobile equipment is included in the second rectangular area and/or the hand detection model does not detect that the human hand is included in the second rectangular area, judging that no user using the mobile equipment exists. And if the distance between the mobile device and the head of the user in the second rectangular area is greater than or equal to the distance threshold value, judging that no action of using the mobile device exists for the user. And if the interval distance between the mobile device and the user head in the second rectangular area is smaller than the distance threshold value, judging that no action of using the mobile device exists by the user when the overlapping area between the area of the mobile device in the second rectangular area and the area of the human hand in the second rectangular area is smaller than the overlapping threshold value.
In one embodiment, an alert mechanism is triggered in the event that it is determined that the user of the second rectangular area is present to use the mobile device.
And triggering an alarm mechanism under the condition that the user in the second rectangular area has the action of using the mobile equipment, wherein the offending mode can be used for alarming through a wireless alarm device or sending illegal information in a short message mode.
In one embodiment, as shown in fig. 3, there is provided an apparatus for recognizing a behavior using a mobile device, including:
the data acquisition module 301 is configured to acquire video data of the fisheye camera, and obtain a plurality of video frames included in the video data; and adjusting the focal length of the fisheye camera to amplify the first rectangular region to obtain a corresponding second rectangular region.
The first global judging module 302 is configured to sequentially input each video frame to the head detection algorithm model, so as to detect a head rectangular area including a user head in each video frame through the head detection algorithm model.
The second global determining module 303 is configured to input each head rectangular area into the recognition model, so as to determine, through the recognition model, that there is a first rectangular area in each head rectangular area that uses the mobile device to behave.
The first local judging module 304 is configured to input the second rectangular region into the mobile device detection model and the hand detection model; and judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model.
The second local judging module 305 is configured to judge an overlapping area between the area where the mobile device is located and the area where the human hand is located, when the separation distance is less than the distance threshold.
A behavior prediction module 306, configured to determine that the user in the second rectangular area has a behavior of using the mobile device when the overlapping area is greater than the overlapping threshold.
In one embodiment, the data preprocessing module 301 is further configured to determine a first region coordinate of the first rectangular region; determining a region center point of a first rectangular region according to the first region coordinates; and adjusting the focal length of the fisheye camera according to the center point to amplify the first rectangular region to obtain a corresponding second rectangular region.
Embodiments of the present application provide a storage medium having stored thereon instructions that when executed by a processor cause the processor to be configured to perform the above-described method of identifying usage mobile device behavior.
The embodiments of the present application provide a processor configured to perform the above-described method of identifying usage mobile device behavior.
In one embodiment, a computer device is provided, which may be a terminal, and the internal structure of which may be as shown in fig. 4. The computer apparatus includes a processor a01, a network interface a02, a display screen a04, an input device a05, and a memory (not shown in the figure) which are connected through a system bus. Wherein the processor a01 of the computer device is adapted to provide computing and control capabilities. The memory of the computer device includes an internal memory a03 and a nonvolatile storage medium a06. The nonvolatile storage medium a06 stores an operating system B01 and a computer program B02. The internal memory a03 provides an environment for the operation of the operating system B01 and the computer program B02 in the nonvolatile storage medium a06. The network interface a02 of the computer device is used for communication with an external terminal through a network connection. The computer program, when executed by the processor a01, implements a method of identifying usage behavior of a mobile device. The display screen a04 of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device a05 of the computer device may be a touch layer covered on the display screen, or may be a key, a track ball or a touch pad arranged on a casing of the computer device, or may be an external keyboard, a touch pad or a mouse.
Those skilled in the art will appreciate that the structures shown in FIG. 4 are block diagrams only and do not constitute a limitation of the computer device on which the present aspects apply, and that a particular computer device may include more or less components than those shown, or may combine some of the components, or have a different arrangement of components.
The embodiment of the application provides equipment, which comprises a processor, a memory and a program stored on the memory and capable of running on the processor, wherein the processor executes the program to realize the following steps:
collecting video data of a fisheye camera to obtain a plurality of video frames included in the video data, sequentially inputting each video frame into a head detection algorithm model to detect a head rectangular area containing a head of a user in each video frame through the head detection algorithm model, inputting each head rectangular area into an identification model to determine that a first rectangular area using a mobile device behavior exists in each head rectangular area through the identification model, adjusting the focal length of the fisheye camera to amplify the first rectangular area to obtain a corresponding second rectangular area, inputting the second rectangular area into the mobile device detection model and the hand detection model, judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the interval distance is smaller than a distance threshold, and judging that the user in the second rectangular area uses the mobile device behavior under the condition that the overlapping area is larger than an overlapping threshold. By the method, the call making action can be identified in all directions, the identification accuracy is improved, the occurrence of accidents is reduced, and the life and property safety of people is ensured.
In one embodiment, N continuous video frames of the same fisheye camera are sequentially input to a head detection algorithm model, so that a head rectangular area containing a user head in each video frame is detected through the head detection algorithm model; the method further comprises the steps of: steps S3 to S8 are performed for each of N consecutive video frames; in the case where the overlapping area of the N consecutive video frames is greater than the overlapping threshold, it is determined that a user included in the N consecutive video frames has a behavior of using the mobile device.
In one embodiment, first region coordinates of a first rectangular region are determined; determining a region center point of a first rectangular region according to the first region coordinates; and adjusting the focal length of the fisheye camera according to the center point to amplify the first rectangular region to obtain a corresponding second rectangular region.
In one embodiment, the zone height of the first rectangular zone is determined from the first zone coordinates; acquiring an initial focal length and picture height pixels of a fisheye camera; determining the zoom multiple of the fisheye camera according to the picture height pixels and the region height; determining a target zoom focal length of the fisheye camera according to the zoom multiple and the initial focal length; and adjusting the focal length of the fisheye camera to the target zoom focal length so as to enlarge the first rectangular area to obtain a corresponding second rectangular area.
In one embodiment, determining a second region coordinate corresponding to the enlarged first rectangular region according to the zoom multiple and the first region coordinate; and determining the rectangular area corresponding to the second area coordinate as a second rectangular area.
In one embodiment, it is determined that no user is present in the act of using the mobile device if any of the following is satisfied: inputting each head rectangular area into a recognition model, wherein the recognition model does not detect that any head rectangular area has a mobile device using behavior; the mobile equipment is included in the second rectangular area which is not detected by the mobile equipment detection model, and/or the human hand is included in the second rectangular area which is not detected by the hand detection model; the separation distance is greater than or equal to a distance threshold; the overlap area is greater than or equal to the overlap threshold.
In one embodiment, an alert mechanism is triggered in the event that it is determined that the user of the second rectangular area is present to use the mobile device.
In one embodiment, an apparatus for identifying usage behavior of a mobile device is provided, comprising: the data acquisition module is used for acquiring video data of the fisheye camera to obtain a plurality of video frames included in the video data; adjusting the focal length of the fisheye camera according to the center point to amplify the first rectangular region to obtain a corresponding second rectangular region; the first global judging module is used for inputting each video frame into the head detection algorithm model in sequence so as to detect a head rectangular area containing the head of the user in each video frame through the head detection algorithm model; the second global judging module is used for inputting each head rectangular area into the recognition model so as to determine that a first rectangular area using the mobile equipment behavior exists in each head rectangular area through the recognition model; the first local judging module is used for inputting the second rectangular area into the mobile equipment detection model and the hand detection model; judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model; the second local judging module is used for judging the overlapping area between the area where the mobile equipment is located and the area where the human hand is located under the condition that the interval distance is smaller than the distance threshold value; and the behavior prediction module is used for judging that the user in the second rectangular area has the behavior of using the mobile device under the condition that the overlapping area is larger than the overlapping threshold value.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and changes may be made to the present application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc. which are within the spirit and principles of the present application are intended to be included within the scope of the claims of the present application.
Claims (10)
1. A method of identifying usage of a mobile device behavior, the method comprising:
s1, acquiring video data of a fisheye camera, and obtaining a plurality of video frames included in the video data;
step S2, inputting each video frame into a head detection algorithm model in sequence, so as to detect a head rectangular area containing the head of a user in each video frame through the head detection algorithm model;
s3, inputting each head rectangular area into a recognition model to determine that a first rectangular area using the mobile equipment behavior exists in each head rectangular area through the recognition model;
s4, adjusting the focal length of the fisheye camera to amplify the first rectangular region to obtain a corresponding second rectangular region;
s5, inputting the second rectangular area into a mobile equipment detection model and a hand detection model;
step S6, judging the interval distance between the mobile device and the user head in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model;
step S7, judging the overlapping area between the area where the mobile equipment is located and the area where the human hand is located under the condition that the interval distance is smaller than a distance threshold value;
and S8, judging that the user of the second rectangular area has the action of using the mobile equipment under the condition that the overlapping area is larger than an overlapping threshold value.
2. The method according to claim 1, wherein step S2 comprises:
sequentially inputting N continuous video frames of the same fisheye camera into a head detection algorithm model to detect a head rectangular region containing a user head in each video frame through the head detection algorithm model;
the method further comprises the steps of:
steps S3 to S8 are performed for each of N consecutive video frames;
in the case where the overlapping area of the N consecutive video frames is greater than the overlapping threshold, it is determined that a user included in the N consecutive video frames has a behavior of using the mobile device.
3. The method according to claim 1, wherein step S4 comprises:
determining first region coordinates of the first rectangular region;
determining a region center point of the first rectangular region according to the first region coordinates;
and adjusting the focal length of the fisheye camera by the center point of the area so as to enlarge the first rectangular area and obtain a corresponding second rectangular area.
4. A method according to claim 3, wherein step S4 comprises:
determining the region height of the first rectangular region according to the first region coordinates;
acquiring an initial focal length and picture height pixels of the fisheye camera;
determining the zoom multiple of the fisheye camera according to the picture height pixels and the region height;
determining a target zoom focal length of the fisheye camera according to the zoom multiple and the initial focal length;
and adjusting the focal length of the fisheye camera to the target zoom focal length so as to enlarge the first rectangular area to obtain a corresponding second rectangular area.
5. The method according to claim 4, wherein the method further comprises:
determining a second region coordinate corresponding to the enlarged first rectangular region according to the zoom multiple and the first region coordinate;
and determining the rectangular area corresponding to the second area coordinate as a second rectangular area.
6. The method according to claim 1 or 2, characterized in that the method further comprises:
determining that no user exists to use the mobile device if any of the following is satisfied:
inputting each head rectangular region into a recognition model, wherein the recognition model does not detect the existence of a mobile device using behavior in any head rectangular region;
the mobile equipment is not detected in the second rectangular area by the mobile equipment detection model, and/or the human hand is not detected in the second rectangular area by the hand detection model;
the separation distance is greater than or equal to a distance threshold;
the overlap area is greater than or equal to an overlap threshold.
7. The method according to claim 1, wherein the method further comprises:
and triggering an alarm mechanism in the case that the user of the second rectangular area is judged to have the action of using the mobile device.
8. An apparatus for identifying behavior using a mobile device, comprising:
the data acquisition module is used for acquiring video data of the fisheye camera to obtain a plurality of video frames included in the video data; adjusting the focal length of the fisheye camera to amplify the first rectangular region to obtain a corresponding second rectangular region;
the first global judging module is used for inputting each video frame into the head detection algorithm model in sequence so as to detect a head rectangular area containing the head of the user in each video frame through the head detection algorithm model;
the second global judging module is used for inputting each head rectangular area into the recognition model so as to determine that a first rectangular area using the mobile equipment behavior exists in each head rectangular area through the recognition model;
the first local judging module is used for inputting the second rectangular area into a mobile equipment detection model and a hand detection model; judging the interval distance between the mobile device and the head of the user in the second rectangular area under the condition that the mobile device and the human hand are simultaneously included in the second rectangular area through the mobile device detection model and the hand detection model;
the second local judging module is used for judging the overlapping area between the area where the mobile equipment is located and the area where the human hand is located under the condition that the interval distance is smaller than a distance threshold value;
and the behavior prediction module is used for judging that the user of the second rectangular area has the behavior of using the mobile equipment under the condition that the overlapping area is larger than an overlapping threshold value.
9. A machine-readable storage medium having instructions stored thereon, which when executed by a processor cause the processor to be configured to perform the method of identifying usage mobile device behavior according to any of claims 1 to 7.
10. A processor configured to perform the method of identifying usage mobile device behavior according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210923201.3A CN115294649B (en) | 2022-08-02 | 2022-08-02 | Method, apparatus, storage medium and processor for identifying behavior using mobile device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210923201.3A CN115294649B (en) | 2022-08-02 | 2022-08-02 | Method, apparatus, storage medium and processor for identifying behavior using mobile device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115294649A CN115294649A (en) | 2022-11-04 |
| CN115294649B true CN115294649B (en) | 2023-05-12 |
Family
ID=83826850
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210923201.3A Active CN115294649B (en) | 2022-08-02 | 2022-08-02 | Method, apparatus, storage medium and processor for identifying behavior using mobile device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115294649B (en) |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114719767A (en) * | 2022-03-30 | 2022-07-08 | 中国工商银行股份有限公司 | Distance detection method and device, storage medium and electronic equipment |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10997730B2 (en) * | 2019-08-21 | 2021-05-04 | XNOR.ai, Inc. | Detection of moment of perception |
| CN114170682A (en) * | 2021-12-01 | 2022-03-11 | 北京计算机技术及应用研究所 | Method for detecting illegal call answering and calling behaviors in restricted area |
-
2022
- 2022-08-02 CN CN202210923201.3A patent/CN115294649B/en active Active
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114719767A (en) * | 2022-03-30 | 2022-07-08 | 中国工商银行股份有限公司 | Distance detection method and device, storage medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115294649A (en) | 2022-11-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR101216271B1 (en) | camera tampering detection | |
| CN106878670B (en) | A kind of method for processing video frequency and device | |
| EP2405393B1 (en) | Device, method and program for creating information for object position estimation | |
| CN111654700B (en) | Privacy mask processing method and device, electronic equipment and monitoring system | |
| CN109766755A (en) | Face identification method and Related product | |
| CN113379999B (en) | Fire detection method, device, electronic equipment and storage medium | |
| KR20120113014A (en) | Image recognition apparatus and vison monitoring method thereof | |
| CN114882003A (en) | Method, medium and computing device for detecting shooting pose change of camera | |
| US9607210B2 (en) | Video surveillance system and method for fraud detection | |
| CN115294649B (en) | Method, apparatus, storage medium and processor for identifying behavior using mobile device | |
| CN111582024B (en) | Video stream processing method, device, computer equipment and storage medium | |
| KR20110104457A (en) | Remote security control system using cloud computing | |
| US20180197000A1 (en) | Image processing device and image processing system | |
| EP2833321A1 (en) | Information processing device, information processing method and program | |
| CN114187666B (en) | Identification method and system for watching mobile phone while walking | |
| CN112906651B (en) | Target detection method and device | |
| CN114898279A (en) | Object detection method and device, computer equipment and storage medium | |
| CN114387544A (en) | High-altitude parabolic detection method and system, electronic equipment and storage medium | |
| US20110234912A1 (en) | Image activity detection method and apparatus | |
| CN114219938A (en) | Region-of-interest acquisition method | |
| CN112419638A (en) | Method and device for acquiring alarm video | |
| WO2023276050A1 (en) | Image processing system and imaging device | |
| JP2018128936A (en) | Image processing program, image processing method, and image processing system | |
| CN112288774B (en) | Mobile detection method, mobile detection device, electronic equipment and storage medium | |
| CN117095505A (en) | Flame detection alarm method for edge AI equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20230423 Address after: Room C, No. 888, Huanhu West 2nd Road, Nanhui new town, Pudong New Area, Shanghai Applicant after: Shanghai Yuchuang Intelligent Technology Co.,Ltd. Address before: Floor 4, Unit 2, Building 5, West Zhigu District, No. 360, Wuke West Fifth Road, Wuhou District, Chengdu, Sichuan 610000 Applicant before: Shanghai Yuchuang Intelligent Technology Co.,Ltd. Chengdu Branch |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |