CN116206356A - Behavior recognition device and method and electronic equipment - Google Patents

Abstract

Embodiments of the present application provide a behavior recognition device and method, and electronic equipment. The method includes: detecting an object in an image to obtain an object detection frame; using a lightweight network and performing pose estimation based on the object detection frame to obtain multiple key points of the object; wherein, the lightweight The backbone network of the level network is a MobileNet network structure, and the lightweight network further includes an upsampling module connected to the MobileNet network structure; and the behavior of the object is identified based on the multiple key points. In this way, the speed of posture estimation can be accelerated, not only can the accuracy of behavior recognition results be improved, but also behavior recognition can be performed in real time.

Description

Behavior recognition device and method and electronic equipment

technical field

The embodiment of the present application relates to the technical field of image detection.

Background technique

Recent advances in artificial intelligence and deep learning technologies have made image-based behavior recognition a reality. Behavior recognition technology can recognize complex behaviors composed of multiple actions or movements. The object frame can be detected by the object detection module, and multiple key points can be detected by the pose estimation module to recognize the behavior of the object.

It should be noted that the above introduction to the technical background is only for the convenience of a clear and complete description of the technical solutions of the present application, and for the convenience of those skilled in the art to understand and explain, not just because these solutions are described in the background technology part of the application. It is considered that the above technical solutions are well known to those skilled in the art.

Contents of the invention

However, the inventors found that the pose estimation module is a time-consuming part. If the number of detected objects increases, the time spent on pose estimation will be greatly increased, and real-time recognition cannot be realized. Therefore, it is difficult to apply to real-time requirements such as embedded devices. higher occasions.

Aiming at at least one of the above technical problems, the embodiments of the present application provide a behavior recognition device and method as well as electronic equipment, expecting to improve the speed of behavior recognition under the premise of ensuring the accuracy of behavior recognition results.

According to an aspect of an embodiment of the present application, a behavior recognition device is provided, including:

A detection unit, which detects an object in the image to obtain an object detection frame;

An estimation unit, which uses a lightweight network and performs pose estimation based on the object detection frame to obtain multiple key points of the object; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the lightweight The magnitude network also includes an upsampling module connected to the MobileNet network structure; and

A recognition unit, which recognizes the behavior of the object based on the plurality of key points.

According to another aspect of the embodiment of the present application, a behavior recognition method is provided, including:

Detect objects in the image to obtain object detection frames;

Use a lightweight network and perform pose estimation based on the object detection frame to obtain multiple key points of the object; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the lightweight network is also Including an upsampling module connected to the MobileNet network structure; and

Recognize the behavior of the object based on the multiple key points.

According to another aspect of the embodiments of the present application, there is provided an electronic device, including a memory and a processor, the memory stores a computer program, and the processor is configured to execute the computer program to implement the aforementioned behavior recognition methods.

One of the beneficial effects of the embodiments of the present application is: using a lightweight network and performing pose estimation based on the object detection frame to obtain multiple key points of the object; wherein, the backbone network of the lightweight network is a MobileNet network structure, and The lightweight network also includes an upsampling module connected to the MobileNet network structure. In this way, the speed of posture estimation can be accelerated, not only can the accuracy of behavior recognition results be improved, but also behavior recognition can be performed in real time.

With reference to the following descriptions and drawings, specific implementation manners of the embodiments of the present application are disclosed in detail, indicating how the principles of the embodiments of the present application can be adopted. It should be understood that the embodiments of the present application are not limited thereby in scope. Embodiments of the present application encompass many changes, modifications and equivalents within the spirit and scope of the appended claims.

Description of drawings

The included drawings are used to provide a further understanding of the embodiments of the present application, which constitute a part of the specification, are used to illustrate the implementation of the present application, and explain the principle of the present application together with the text description. Apparently, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can also obtain other implementation manners according to these drawings without creative efforts. In the attached picture:

FIG. 1 is a schematic diagram of a framework of behavior recognition in an embodiment of the present application;

FIG. 2 is a schematic diagram of a framework of a CPN;

Fig. 3 is a schematic diagram of the behavior recognition method of the embodiment of the present application;

FIG. 4 is a schematic diagram of a lightweight network according to an embodiment of the present application;

Fig. 5 is a schematic diagram of Block in the lightweight network of Fig. 4;

Fig. 6 is a schematic diagram of SeModule in the Block of Fig. 5;

FIG. 7 is an example diagram of behavior recognition in the embodiment of the present application;

Fig. 8 is a schematic diagram of a behavior recognition device according to an embodiment of the present application;

Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application.

Detailed ways

The foregoing and other features of the embodiments of the present application will become apparent from the following description, with reference to the accompanying drawings. In the description and drawings, specific embodiments of the present application are specifically disclosed, which indicate some embodiments in which the principles of the embodiments of the present application can be adopted. It should be understood that the present application is not limited to the described embodiments, on the contrary , The embodiments of the present application include all modifications, variations and equivalents falling within the scope of the appended claims.

In this embodiment of the application, the terms "first", "second", etc. are used to distinguish different elements from the title, but do not indicate the spatial arrangement or time order of these elements, and these elements should not be referred to by these terms restricted. The term "and/or" includes any and all combinations of one or more of the associated listed items. The terms "comprising", "including", "having" and the like refer to the presence of stated features, elements, elements or components, but do not exclude the presence or addition of one or more other features, elements, elements or components.

In the embodiments of the present application, the singular forms "a", "the", etc. include plural forms, which should be broadly understood as "one" or "a class" rather than limited to "one"; in addition, the term "all The above should be understood to include both the singular and the plural, unless the context clearly dictates otherwise. Furthermore, the term "based on" should be understood as "at least in part based on..." and the term "based on" should be understood as "at least in part based on...", unless the context clearly indicates otherwise.

Features described and/or illustrated with respect to one embodiment can be used in the same or similar manner in one or more other embodiments, in combination with, or instead of features in other embodiments . The term "comprising/comprising" when used herein refers to the presence of a feature, integer, step or component, but does not exclude the presence or addition of one or more other features, integers, steps or components.

FIG. 1 is a schematic diagram of a framework of behavior recognition according to an embodiment of the present application. As shown in FIG. 1 , for an input image, the object detection module 101 can be used for detection to obtain an object detection frame; then the pose estimation module 102 can be used for pose estimation to obtain key points of one or more objects. The feature calculation module 103 can be used to perform feature extraction, and then the behavior of the object can be identified through the lightweight classifier 104 . When performing pose estimation, a neural network model or the like can be used, for example, a cascaded pyramid network (CPN, Cascaded Pyramid Network) model can be used.

Figure 2 is a schematic diagram of a framework of CPN. CPN can include GlobalNet and RefineNet; the backbone network of GlobalNet is ResNet. Among them, GolbalNet is responsible for the detection of key points of the network, and it has a better prediction effect on key points of parts that are easier to detect (such as eyes, arms, etc.), and the loss function used is L2 loss. A 1*1 convolution operation can be used on the feature map (featuremap) before each elem-sum operation. RefineNet corrects the results predicted by GolbalNet; GolbalNet has a large prediction error for key points that are occluded, invisible, or have complex backgrounds, and RefineNet can correct these key points. For specific content of models such as CPN, reference may also be made to related technologies.

However, CPN is a relatively heavyweight network structure. If the number of detected objects increases, the time required for pose estimation will be greatly increased, and real-time recognition cannot be realized. Therefore, it is difficult to apply to embedded devices and other occasions with high real-time requirements.

In this embodiment of the present application, the object to be detected may be a human body of various ages, for example, an elderly person, a child, or an elderly person and/or a caregiver, a child and/or a guardian. The present application is not limited thereto, and the object to be detected may be a human body with vital signs, or a robot without vital signs, or the like.

Embodiments of the first aspect

An embodiment of the present application provides a behavior recognition method. Fig. 3 is a schematic diagram of the behavior recognition method of the embodiment of the present application, as shown in Fig. 3, the method includes:

301. Detect an object in an image to obtain an object detection frame;

302. Use a lightweight network to perform pose estimation based on the object detection frame, and obtain multiple key points of the object; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the lightweight network also includes a MobileNet network a structurally connected upsampling module; and

303. Identify the behavior of the object based on multiple key points.

It should be noted that the above accompanying drawing 3 only schematically illustrates the embodiment of the present application, but the present application is not limited thereto. For example, the execution order of various operations can be appropriately adjusted, and some other operations can be added or some of them can be reduced. Those skilled in the art can make appropriate modifications according to the above content, and are not limited to the above description in FIG. 3 .

In some embodiments, the image with the object to be detected can be one or more frames of images in the video frame, that is, the image can be a dynamic image, but the application is not limited thereto. For one or more static images, the present application The application examples are also applicable.

In some embodiments, the lightweight network is generated by using MobileNet to replace the backbone network of GlobalNet in the cascaded pyramid network, and using an upsampling module to replace the RefineNet and pyramid in the cascaded pyramid network structure.

For example, using MobileNetv3 instead of Resnet-50backbone can use fewer parameters and reduce the memory footprint required during inference, thereby speeding up pose estimation. In addition, remove the RefineNet and pyramid structure of CPN, and directly use the upsampling module to process the output of MobileNetv3, which can further improve the speed of pose estimation.

In some embodiments, the lightweight network uses Globalnet with MobileNet as the backbone network to perform multiple down-sampling, and uses an up-sampling module to perform multiple up-sampling on the result of one down-sampling in the multiple down-sampling sampling. Thus, the upsampling module can be directly used to simplify the network structure, realize a lightweight network structure, and further improve the speed of pose estimation.

In some embodiments, the up-sampling module performs multiple up-sampling on the result of the lowest down-sampling in the multiple down-sampling. As a result, the bottom-level downsampling results can be used, which can not only realize a lightweight network structure, further improve the speed of pose estimation, but also further improve the accuracy of pose estimation.

Fig. 4 is a schematic diagram of the lightweight network of the embodiment of the present application. As shown in Fig. 4, for a 3xHxW input image, convolution "Conv2d, 16x3x3, 2" (as shown in 401) and batch normalization can be performed (BN, BatchNorm) "BN, hswish" (as shown in 402 ) operation.

As shown in Figure 4, MobileNetv3 can be used for downsampling (as shown in 403 to 406), such as "Block,3x16x16x16,1,Relu,None" "Block,3x16x64x24,2,Relu,None" "Block,3x24x72x24,1 ,Relu,None" and other operations (as shown in 403), "Block,5x24x72x40,2,Relu,Se" "Block,5x40x120x40,1,Relu,Se" and "Block,5x40x120x40,1,Relu,Se" and other operations ( As shown in 404), "Block,3x40x240x80,2,hswish,None" "Block,3x80x200x80,1,hswish,None" "Block,3x80x184x80,1,hswish,None" "Block,3x80x184x80,1,hswish,None" Wait for the operation (as shown in 405), "Block,3x80x480x112,1,hswish,Se" "Block,3x112x672x160,1,hswish,Se" "Block,5x112x672x160,1,hswish,Se" "Block,5x160x672x160,2,hswish ,Se" "Block,5x160x960x160,1,hswish,Se" and other operations (as shown in 406). Wherein, a linear correction unit (Relu, Rectified linear unit), hswish, Se, etc. may be used, and for specific meanings of these parameters, reference may be made to related technologies.

FIG. 5 is a schematic diagram of Blocks in the lightweight network of FIG. 4 . As shown in Figure 5, "Conv72x1x1,1" (as shown in 501), "BN, nolinear" (as shown in 502), "Conv 72x5x5,2" (as shown in 503), "BN, nolinear" can be performed (as shown in 504), "Conv 40x1x1,1" (as shown in 505), "BN" (as shown in 506) and other operations; in addition, "Conv 40x1x1,1" (as shown in 507) can also be performed, "BN" (shown in 508) and other operations. As shown in FIG. 5 , the Block may also include a SeModule module 509 .

FIG. 6 is a schematic diagram of the SeModule module 509 in the Block of FIG. 5 . As shown in Figure 6, "AdaptiveAvgPool" (as shown in 601), "Conv 10x1x1,1" (as shown in 602), "BN, Relu" (as shown in 603), "Conv 40x1x1,1" ( As shown in 604), "BN, hsigmoid" (as shown in 605) and other operations.

Figures 4 to 6 above schematically illustrate the use of MobileNetv3 to replace the Globalnet of Resnet, and the upsampling module of the embodiment of the present application will be described below.

As shown in FIG. 4 , the up-sampling module performs multiple up-sampling on the result of the bottom-level down-sampling (as shown in 406 ) among the multiple down-samplings. As shown in Figure 4, "ConvTranspose2d, 80x4x4, 2" (as shown in 407), "BN, Relu" (as shown in 408), "ConvTranspose2d, 40x4x4, 2" (as shown in 409), "BN ,Relu" (as shown in 410), "ConvTranspose2d,24x4x4,2" (as shown in 411), "BN, Relu" (as shown in 412), "Conv2d,17x1x1,1" (as shown in 413), etc. operate. As shown in Figure 4, a 17xHxW heatmap can be output.

4 to 6 above exemplarily illustrate the lightweight network of the present application, but the present application is not limited thereto.

Fig. 7 is an example diagram of behavior recognition according to the embodiment of the present application. For the sake of simplicity, only one object (human body) is marked with reference signs. As shown in Figure 7, through the behavior recognition of the embodiment of the present application, an object detection frame 701 can be generated for multiple objects, and a plurality of connected key points 702 can also be obtained, so that the behavior of the object can be recognized, not only The behavior of the object can be accurately recognized, and the speed of behavior recognition is fast, which can meet the real-time requirement.

Table 1 shows a comparison result between the CPN (represented by CPN-Resnet50) and the lightweight network of the embodiment of the present application (represented by MobileNet-transpose). As shown in Table 1, the lightweight network of the embodiment of the present application can significantly reduce memory usage and parameters.

Table 1

Model GPU memory (M) Weight parameter (M) CPN-Resnet50 2275 108.9 MobileNet-transpose 839 6.4

Table 2 shows the existing pose estimation model (represented by CPN-EfficientNet), the model of the backbone network of EfficientNet replaced by MobileNetv3 (represented by CPN-MobileNetv3) and the lightweight network of the embodiment of the present application (represented by MobileNet-transpose Indicates a comparison result of ).

Table 2

Model Weight parameter (M) FPS AP(0.5:0.95) AR(0.5:0.95) CPN-EfficientNet 7.5M 17.16 0.591 0.631 CPN-MobileNetv3 5.5M 20.91 0.556 0.602 MobileNet-transpose 6.4M 21.54 0.607 0.646

As shown in Table 2, although the speed of CPN-MobileNetv3 has increased compared with CPN-EfficientNet, the performance has declined. However, because the lightweight network MobileNet-transpose in the embodiment of the present application directly uses the upsampling module, it can not only ensure the accuracy of behavior recognition, improve system performance, but also increase the speed of behavior recognition.

The above only describes the steps or processes related to the present application, but the present application is not limited thereto. The behavior recognition method may also include other steps or processes, and for the specific content of these steps or processes, reference may be made to the prior art. In addition, the embodiments of the present application are only illustrated by taking some structures of the behavior recognition model as examples above, but the present application is not limited to these structures, and appropriate modifications can also be made to these structures, and the implementation of these modifications should be included in within the scope of the embodiments of this application.

The above embodiments only illustrate the embodiments of the present application as examples, but the present application is not limited thereto, and appropriate modifications can also be made on the basis of the above various embodiments. For example, each of the above-mentioned embodiments may be used alone, or one or more of the above-mentioned embodiments may be combined.

As can be seen from the above-mentioned embodiments, using a lightweight network and performing pose estimation based on the object detection frame, multiple key points of the object are obtained; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the lightweight network is also It includes an upsampling module connected with the MobileNet network structure. In this way, the speed of posture estimation can be accelerated, not only can the accuracy of behavior recognition results be improved, but also behavior recognition can be performed in real time.

Embodiments of the second aspect

The embodiment of the present application provides a behavior recognition device, and the same content as the embodiment of the first aspect will not be repeated.

FIG. 8 is a schematic diagram of a behavior recognition device according to an embodiment of the present application. As shown in FIG. 8, the behavior recognition device 800 includes:

A detection unit 801, which detects an object in the image to obtain an object detection frame;

An estimation unit 802, which uses a lightweight network and performs pose estimation based on the object detection frame to obtain multiple key points of the object; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the The lightweight network also includes an upsampling module connected to the MobileNet network structure; and

A recognition unit 803, which recognizes the behavior of the object based on the multiple key points.

In some embodiments, the lightweight network is generated by using MobileNet to replace the backbone network of the GlobalNet in the cascaded pyramid network, and using the up-sampling module to replace the RefineNet in the cascaded pyramid network and pyramid structures.

In some embodiments, the lightweight network uses Globalnet with MobileNet as the backbone network to perform multiple downsampling, and uses the upsampling module to perform multiple downsampling on the result of one downsampling in the multiple downsamplings. upsampling.

In some embodiments, the up-sampling module performs multiple up-sampling on the result of the lowest down-sampling in the multiple down-sampling.

It should be noted that the above only describes the components or modules related to the present application, but the present application is not limited thereto. The behavior recognition apparatus 800 may also include other components or modules, and for the specific content of these components or modules, reference may be made to related technologies.

For the sake of simplicity, FIG. 8 only exemplarily shows the connection relationship or signal direction among various components or modules, but it should be clear to those skilled in the art that various related technologies such as bus connection may be used. The foregoing components or modules may be implemented by hardware facilities such as processors and memories; this is not limited in this embodiment of the present application.

The above embodiments only illustrate the embodiments of the present application as examples, but the present application is not limited thereto, and appropriate modifications can also be made on the basis of the above various embodiments. For example, each of the above-mentioned embodiments may be used alone, or one or more of the above-mentioned embodiments may be combined.

As can be seen from the above-mentioned embodiments, a lightweight network is used to perform pose estimation based on object detection frames to obtain multiple key points of an object; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the lightweight The network also includes an upsampling module connected with the MobileNet network structure. In this way, the speed of posture estimation can be accelerated, not only can the accuracy of behavior recognition results be improved, but also behavior recognition can be performed in real time.

Embodiments of the third aspect

An embodiment of the present application provides an electronic device, including the behavior recognition apparatus 800 described in the embodiment of the second aspect, the content of which is incorporated herein. The electronic device may be, for example, a computer, a server, a workstation, a laptop computer, a smart phone, etc.; but this embodiment of the present application is not limited thereto.

Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 9 , an electronic device 900 may include: a processor (such as a central processing unit CPU) 910 and a memory 920 ; the memory 920 is coupled to the central processing unit 910 . The memory 920 can store various data; in addition, it also stores a program 921 for information processing, and executes the program 921 under the control of the processor 910 .

In some embodiments, the functions of the behavior recognition device 800 are integrated into the processor 910 for implementation. Wherein, the processor 910 is configured to implement the behavior recognition method described in the embodiment of the first aspect.

In some embodiments, the behavior recognition device 800 is configured separately from the processor 910 , for example, the behavior recognition device 800 can be configured as a chip connected to the processor 910 , and the functions of the behavior recognition device 800 are realized through the control of the processor 910 .

For example, the processor 910 is configured to perform the following control: detect an object in the image to obtain an object detection frame; use a lightweight network to perform pose estimation based on the object detection frame, and obtain multiple key points of the object point; wherein, the backbone network of the lightweight network is a MobileNet network structure, and the lightweight network also includes an upsampling module connected to the MobileNet network structure; and based on the multiple key points to the Object behavior is recognized.

In some embodiments, the lightweight network is generated by using MobileNet to replace the backbone network of the GlobalNet in the cascaded pyramid network, and using the up-sampling module to replace the RefineNet in the cascaded pyramid network and pyramid structures.

In some embodiments, the lightweight network uses Globalnet with MobileNet as the backbone network to perform multiple downsampling, and uses the upsampling module to perform multiple downsampling on the result of one downsampling in the multiple downsamplings. upsampling.

In some embodiments, the up-sampling module performs multiple up-sampling on the result of the lowest down-sampling in the multiple down-sampling.

In addition, as shown in FIG. 9 , the electronic device 900 may further include: an input/output (I/O) device 930 and a display 940 , etc.; where the functions of the above components are similar to those of the prior art, and will not be repeated here. It should be noted that the electronic device 900 does not necessarily include all the components shown in FIG. 9 ; in addition, the electronic device 900 may also include components not shown in FIG. 9 , and reference may be made to related technologies.

The embodiment of the present application also provides a computer-readable program, wherein when the program is executed in the electronic device, the program causes the computer to execute the behavior recognition method as described in the embodiment of the first aspect in the electronic device .

The embodiment of the present application also provides a storage medium storing a computer-readable program, wherein the computer-readable program enables the computer to execute the behavior recognition method as described in the embodiment of the first aspect in the electronic device.

The above devices and methods in this application can be implemented by hardware, or by combining hardware and software. The present application relates to a computer-readable program that, when executed by a logic component, enables the logic component to realize the above-mentioned device or constituent component, or enables the logic component to realize the above-mentioned various methods or steps. The present application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memories, and the like.

The method/device described in conjunction with the embodiments of the present application may be directly embodied as hardware, a software module executed by a processor, or a combination of both. For example, one or more of the functional block diagrams shown in the figure and/or one or more combinations of the functional block diagrams may correspond to each software module or each hardware module of the computer program flow. These software modules may respectively correspond to the steps shown in the figure. These hardware modules, for example, can be realized by solidifying these software modules by using a Field Programmable Gate Array (FPGA).

A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM or any other form of storage medium known in the art. A storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium, or it can be an integral part of the processor. The processor and storage medium can be located in the ASIC. The software module can be stored in the memory of the mobile terminal, or can be stored in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) adopts a large-capacity MEGA-SIM card or a large-capacity flash memory device, the software module can be stored in the MEGA-SIM card or large-capacity flash memory device.

One or more of the functional blocks described in the accompanying drawings and/or one or more combinations of the functional blocks can be implemented as a general-purpose processor, a digital signal processor (DSP) for performing the functions described in this application ), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or any suitable combination thereof. One or more of the functional blocks described in the drawings and/or one or more combinations of the functional blocks can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors processor, one or more microprocessors in communication with a DSP, or any other such configuration.

The present application has been described above in conjunction with specific implementation manners, but those skilled in the art should be clear that these descriptions are exemplary rather than limiting the protection scope of the present application. Those skilled in the art can make various variations and modifications to the application based on the principle of the application, and these variations and modifications are also within the scope of the application.

Claims (9)

1. A behavior recognition apparatus, the apparatus comprising:

a detection unit that detects an object in an image to obtain an object detection frame;

an estimation unit that obtains a plurality of key points of the object using a lightweight network and performing pose estimation based on the object detection frame; the backbone network of the lightweight network is a MobileNet network structure, and the lightweight network further comprises an up-sampling module connected with the MobileNet network structure; and

and an identification unit that identifies a behavior of the object based on the plurality of key points.

2. The apparatus of claim 1, wherein the lightweight network is generated by: a backbone network of GlobalNet in a cascading pyramid network is replaced with MobileNet and the upsampling module is used to replace RefineNet and pyramid structures in the cascading pyramid network.

3. The apparatus of claim 2, wherein the lightweight network uses Globalnet with MobileNet as backbone network to perform multiple downsampling, and wherein the upsampling module uses the upsampling module to upsample the result of one of the multiple downsampling.

4. The apparatus of claim 3, wherein the upsampling module upsamples the result of a bottommost downsampling of the plurality of downsampling a plurality of times.

5. A method of behavior recognition, the method comprising:

detecting an object in the image to obtain an object detection frame;

using a lightweight network and performing pose estimation based on the object detection frame to obtain a plurality of key points of the object; the backbone network of the lightweight network is a MobileNet network structure, and the lightweight network further comprises an up-sampling module connected with the MobileNet network structure; and

and identifying the behavior of the object based on the plurality of key points.

6. The method of claim 5, wherein the lightweight network is generated by: a backbone network of GlobalNet in a cascading pyramid network is replaced with MobileNet and the upsampling module is used to replace RefineNet and pyramid structures in the cascading pyramid network.

7. The method of claim 6, wherein the lightweight network uses a Globalnet with a MobileNet backbone network for multiple downsampling, and wherein the upsampling module upsamples the result of one of the multiple downsampling.

8. The method of claim 7, wherein the upsampling module upsamples the result of a bottommost downsampling of the plurality of downsampling a plurality of times.

9. An electronic device comprising a memory storing a computer program and a processor configured to execute the computer program to implement the behavior recognition method of any one of claims 5 to 8.

Images