WO2022213552A1

WO2022213552A1 - Systems and methods for helmet detection

Info

Publication number: WO2022213552A1
Application number: PCT/CN2021/119585
Authority: WO
Inventors: Shaofei ZHENG; Bangjie TANG; Huadong PAN; Jun Yin; Xingming Zhang
Original assignee: Zhejiang Dahua Technology Co Ltd
Current assignee: Zhejiang Dahua Technology Co Ltd
Priority date: 2021-04-07
Filing date: 2021-09-22
Publication date: 2022-10-13
Anticipated expiration: 2023-10-07
Also published as: CN113158851B; CN113158851A

Abstract

Systems and methods for helmet detection are disclosed. The systems may obtain an image associated with a target object. The systems may determine, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image. The systems may determine, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result. The intermediate detection result may be associated with a plurality of key points associated with the target object. The systems may determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

Description

SYSTEMS AND METHODS FOR HELMET DETECTION

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to Chinese Patent Application No. 202110373959. X filed on April 7, 2021, the contents of which are incorporated herein by reference in their entirety.

TECHNICAL FIELD

The present disclosure generally relates to computer vision technology, and in particular, to systems and methods for detecting whether a target object is wearing a helmet or not.

BACKGROUND

With the development of industrial technology, engineering construction (e.g., building construction, site production) becomes more and more extensive. In order to ensure safety during engineering construction, an object (e.g., a construction worker) who enters a construction site is usually required to wear a helmet. However, due to fluke and the discomfort caused by wearing the helmet, some objects are reluctant to wear helmets. Therefore, it is desirable to provide systems and methods for detecting whether the object who enters the construction site is wearing a helmet or not, thereby reducing the occurrence of safety accidents and improving work safety.

SUMMARY

An aspect of the present disclosure relates to a system for helmet detection. The system may include at least one storage device including a set of instructions and at least one processor in communication with the at least one storage device. When executing the set of instructions, the at least one processor may be directed to cause the system to implement operations. The operations may include obtaining an image associated with a target object. The operations may include determining, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image. The operations may include determining, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result. The intermediate detection result may be associated with a plurality of key points associated with the target object. The operations may further include determining a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

In some embodiments, the initial detection result may include position information of a head region of the target object and a determination result of whether there is a helmet in the head region of the target object.

In some embodiments, the initial detection result may further include a determination result of whether there is a hand region at least partially located within the head region of the target object.

In some embodiments, the plurality of key points may represent a shape feature of a target region associated with a head region of the target object.

In some embodiments, the plurality of key points may include a first point located at a top location of an edge of the target region, a second point located at a far-left location of the edge of the target region, a third point located at a far-right location of the edge of the target region, a fourth point located between the first point and the second point, and a fifth point located between the first point and the third point.

In some embodiments, the determining, using the key-point extraction model, the intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result may include determining, using the key-point extraction model, the plurality of key points associated with the target object in an image portion indicated by the at least portion of the initial detection result, and determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points. The intermediate detection result may include a determination result of whether there is a helmet in a head region of the target object.

In some embodiments, the determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points may include determining whether one or more of the plurality of key points are located at a target position of the image portion indicated by the at least portion of the initial detection result, and determining, in response to determining that one or more of the plurality of key points are located at the target position, the determination result that there is no helmet in the head region of the target object.

In some embodiments, the determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points may include in response to determining that no key point is located at the target position, dividing the plurality of key points into two groups of key points including a same key point, for each of the two groups of key points, determining a connection line from the same key point to a target point, and determining the determination result that there is a helmet in the head region of the target object in response to that connection lines corresponding to the two groups of key points satisfy a preset condition.

In some embodiments, the determining the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result may include determining, using a classification model, a second intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result, and determining the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result, the intermediate detection result, and the second intermediate detection result.

In some embodiments, the second intermediate detection result may include a determination result of whether there is a helmet in a head region of the target object and/or a color of the helmet.

In some embodiments, the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result may include determining the target detection result that the target object is wearing a helmet and a color of the helmet in response to that: all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in a head region of the target object, the initial detection result indicates there is no hand region at least partially located within the head region of the target object, and the second intermediate detection result indicates the color of the helmet.

In some embodiments, the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result may include determining, in response to that: all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in the head region of the target object, the initial detection result indicates there is a hand region at least partially located within the head region of the target object, and the second intermediate detection result indicates the color of the helmet, a target confidence level based on confidence levels corresponding to the initial detection result, the intermediate detection result, and the second intermediate detection result, and determining, in response to determining that the target confidence level is larger than or equal to a confidence level threshold, the target detection result that the target object is wearing a helmet and a color of the helmet.

In some embodiments, the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result may further include determining, in response to determining that the target confidence level is less than the confidence level threshold, the target detection result that the target object is not wearing a helmet and a color of the helmet.

In some embodiments, the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result may include determining the target detection result that the target object is not wearing a helmet in response to that: at least one of the initial detection result, the intermediate detection result, or the second intermediate detection result indicates that there is no helmet in a head region of the target object, or the second intermediate detection result does not indicate the color of the helmet.

A further aspect of the present disclosure relates to a method for helmet detection. The method may be implemented on a computing device including at least one processor, at least one storage medium, and a communication platform connected to a network. The method may include obtaining an image associated with a target object. The method may include determining, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image. The method may include determining, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result. The intermediate detection result may be associated with a plurality of key points associated with the target object. The method may further include determining a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

A still further aspect of the present disclosure relates to a system for helmet detection. The system may include an obtaining module and a determination module. The obtaining module may be configured to obtain an image associated with a target object. The determination module may be configured to determine, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image. The determination module may be configured to determine, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result. The intermediate detection result may be associated with a plurality of key points associated with the target object. The determination module may be configured to determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

A still further aspect of the present disclosure relates to a non-transitory computer readable medium including executable instructions. When the executable instructions are executed by at least one processor, the executable instructions may direct the at least one processor to perform a method. The method may include obtaining an image associated with a target object. The method may include determining, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image. The method may include determining, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result. The intermediate detection result may be associated with a plurality of key points associated with the target object. The method may further include determining a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

Additional features will be set forth in part in the description which follows, and in part will become apparent to those skilled in the art upon examination of the following and the accompanying drawings or may be learned by production or operation of the examples. The features of the present disclosure may be realized and attained by practice or use of various aspects of the methodologies, instrumentalities, and combinations set forth in the detailed examples discussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The present disclosure is further described in terms of exemplary embodiments. These exemplary embodiments are described in detail with reference to the drawings. These embodiments are non-limiting exemplary embodiments, in which like reference numerals represent similar structures throughout the several views of the drawings, and wherein:

FIG. 1 is a schematic diagram illustrating an exemplary helmet detection system according to some embodiments of the present disclosure;

FIG. 2 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure;

FIG. 3 is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure;

FIG. 4 is a flowchart illustrating an exemplary process for helmet detection according to some embodiments of the present disclosure;

FIGs. 5A-5C are schematic diagrams illustrating exemplary initial detection results according to some embodiments of the present disclosure;

FIGs. 6A-6C are schematic diagrams illustrating exemplary intermediate detection results according to some embodiments of the present disclosure;

FIG. 7 is a flowchart illustrating an exemplary process for determining an intermediate detection result associated with whether a target object is wearing a helmet or not according to some embodiments of the present disclosure;

FIGs. 8A and 8B are schematic diagrams illustrating exemplary operations for determining an intermediate detection result associated with whether a target object is wearing a helmet or not according to some embodiments of the present disclosure;

FIG. 9 is a flowchart illustrating an exemplary process for helmet detection according to some embodiments of the present disclosure;

FIG. 10 is a schematic diagram illustrating an exemplary second intermediate detection result according to some embodiments of the present disclosure;

FIG. 11 is a flowchart illustrating an exemplary process for helmet detection according to some embodiments of the present disclosure;

FIG. 12 is a flowchart illustrating an exemplary process for determining a detection model, a key-point extraction model, and/or a classification model according to some embodiments of the present disclosure; and

FIG. 13 is a schematic diagram illustrating exemplary marking information of a training sample according to some embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are set forth by way of examples in order to provide a thorough understanding of the relevant disclosure. However, it should be apparent to those skilled in the art that the present disclosure may be practiced without such details. In other instances, well-known methods, procedures, systems, components, and/or circuitry have been described at a relatively high level, without detail, in order to avoid unnecessarily obscuring aspects of the present disclosure. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown, but to be accorded the widest scope consistent with the claims.

It will be understood that the terms “system, ” “engine, ” “unit, ” “module, ” and/or “block” used herein are one method to distinguish different components, elements, parts, sections, or assemblies of different levels in ascending order. However, the terms may be displaced by other expressions if they may achieve the same purpose.

Generally, the words “module, ” “unit, ” or “block” used herein, refer to logic embodied in hardware or firmware, or to a collection of software instructions. A module, a unit, or a block described herein may be implemented as software and/or hardware and may be stored in any type of non-transitory computer-readable medium or other storage devices. In some embodiments, a software module/unit/block may be compiled and linked into an executable program. It will be appreciated that software modules can be callable from other modules/units/blocks or from themselves, and/or may be invoked in response to detected events or interrupts. Software modules/units/blocks configured for execution on computing devices (e.g., processor 220 illustrated in FIG. 2) may be provided on a computer-readable medium, such as a compact disc, a digital video disc, a flash drive, a magnetic disc, or any other tangible medium, or as a digital download (and can be originally stored in a compressed or installable format that needs installation, decompression, or decryption prior to execution) . Such software code may be stored, partially or fully, on a storage device of the executing computing device, for execution by the computing device. Software instructions may be embedded in firmware, such as an EPROM. It will be further appreciated that hardware modules (or units or blocks) may be included in connected logic components, such as gates and flip-flops, and/or can be included in programmable units, such as programmable gate arrays or processors. The modules (or units or blocks) or computing device functionality described herein may be implemented as software modules (or units or blocks) , but may be represented in hardware or firmware. In general, the modules (or units or blocks) described herein refer to logical modules (or units or blocks) that may be combined with other modules (or units or blocks) or divided into sub-modules (or sub-units or sub-blocks) despite their physical organization or storage.

It will be understood that when a unit, an engine, a module, or a block is referred to as being “on, ” “connected to, ” or “coupled to” another unit, engine, module, or block, it may be directly on, connected or coupled to, or communicate with the other unit, engine, module, or block, or an intervening unit, engine, module, or block may be present, unless the context clearly indicates otherwise. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items.

The terminology used herein is for the purposes of describing particular examples and embodiments only and is not intended to be limiting. As used herein, the singular forms “a, ” “an, ” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “include” and/or “comprise, ” when used in this disclosure, specify the presence of integers, devices, behaviors, stated features, steps, elements, operations, and/or components, but do not exclude the presence or addition of one or more other integers, devices, behaviors, features, steps, elements, operations, components, and/or groups thereof.

In addition, it should be understood that in the description of the present disclosure, the terms “first” , “second” , or the like, are only used for the purpose of differentiation, and cannot be interpreted as indicating or implying relative importance, nor can be understood as indicating or implying the order.

The flowcharts used in the present disclosure illustrate operations that systems implement according to some embodiments of the present disclosure. It is to be expressly understood, the operations of the flowcharts may be implemented not in order. Conversely, the operations may be implemented in an inverted order, or simultaneously. Moreover, one or more other operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

An aspect of the present disclosure relates to systems and methods for detecting whether a target object is wearing a helmet or not. The systems may obtain an image associated with a target object (e.g., a construction worker) . According to the image, the systems may determine an initial detection result associated with whether the target object is wearing a helmet or not using a detection model. According to at least portion of the initial detection result, the systems may determine an intermediate detection result associated with whether the target object is wearing a helmet or not using a key-point extraction model. The intermediate detection result may be associated with a plurality of key points associated with the target object. The plurality of key points may represent a shape feature of a target region (e.g., a helmet region) associated with a head region of the target object. Further, the systems may determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

According to the systems and methods of the present disclosure, the target detection result associated with whether the target object is wearing a helmet or not is determined based on the initial detection result and the intermediate detection result, which can avoid situations that objects use disguise for helmet detection and improve the accuracy of the helmet detection.

FIG. 1 is a schematic diagram illustrating an exemplary helmet detection system according to some embodiments of the present disclosure. In some embodiments, the helmet detection system 100 may be applied in various application scenarios where it is necessary to identify whether a target object is wearing a helmet or not. As shown, the helmet detection system 100 may include a server 110, a network 120, a detection device 130, and a storage device 140.

The server 110 may be a single server or a server group. The server group may be centralized or distributed (e.g., the server 110 may be a distributed system) . In some embodiments, the server 110 may be local or remote. For example, the server 110 may access information and/or data stored in the detection device 130 and/or the storage device 140 via the network 120. As another example, the server 110 may be directly connected to the detection device 130 and/or the storage device 140 to access stored information and/or data. In some embodiments, the server 110 may be implemented on a computing device 200 including one or more components illustrated in FIG. 2 of the present disclosure.

In some embodiments, the server 110 may include a processing device 112. The processing device 112 may process information and/or data relating to helmet detection to perform one or more functions described in the present disclosure. For example, the processing device 112 may obtain an image associated with a target object and determine an initial detection result associated with whether the target object is wearing a helmet or not based on the image. According to at least portion of the initial detection result, the processing device 112 may determine an intermediate detection result associated with whether the target object is wearing a helmet or not using a key-point extraction model. Further, the processing device 112 may determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

In some embodiment, the server 110 may be unnecessary and all or part of the functions of the server 110 may be implemented by other components (e.g., the detection device 130) of the helmet detection system 100. For example, the processing device 112 may be integrated into the detection device 130 and the functions (e.g., determining the target detection result associated with whether the target object is wearing a helmet or not) of the processing device 112 may be implemented by the detection device 130.

The network 120 may facilitate exchange of information and/or data for the helmet detection system 100. In some embodiments, one or more components (e.g., the server 110, the detection device 130, the storage device 140) of the helmet detection system 100 may transmit information and/or data to other component (s) of the helmet detection system 100 via the network 120. For example, the server 110 may obtain the image associated with the target object from the detection device 130 via the network 120. As another example, the server 110 may transmit the target detection result associated with whether the target object is wearing a helmet or not to the detection device 130 via the network 120. In some embodiments, one or more components (e.g., the server 110, the detection device 130, the storage device 140) of the helmet detection system 100 may communicate information and/or data with one or more external resources such as an external database of a third party, etc. For example, the server 110 may obtain a model (e.g., a detection model, a key-point extraction model, a classification model) associated with the helmet detection from a database of a vendor or manufacture (e.g., a manufacture of the detection device 130) that provides and/or updates the model. In some embodiments, the network 120 may be any type of wired or wireless network, or a combination thereof.

The detection device 130 may be configured to manage entry and/or exit of objects. Merely by way of example, the detection device 130 may be a turnstile, a gate machine, etc. In some embodiments, the detection device 130 may be in communication with the server 110 via the network 120. The server 110 may control the detection device 130 to open or close to allow an object to enter or exit. For example, when an object moves near the detection device 130, the server 110 may determine a target detection result associated with whether the target object is wearing a helmet or not. If the target object is wearing a helmet, the server 110 may control the detection device 130 to open to allow the object to enter; if the target object is not wearing a helmet, the server 110 may control the detection device 130 to keep closed.

In some embodiments, the detection device 130 may include at least one imaging device 130-1. The imaging device 130-1 may be configured to capture an image associated with an object moving near the detection device 130 and transmit the image to the server 110. In some embodiments, the imaging device 130-1 may include a camera, a video recorder, an image sensor, etc. The camera may include a gun camera, a dome camera, an integrated camera, a monocular camera, a binocular camera, a multi-view camera, or the like, or any combination thereof. The video recorder may include a PC Digital Video Recorder (DVR) , an embedded DVR, or the like, or any combination thereof. The image sensor may include a Charge Coupled Device (CCD) image sensor, a Complementary Metal Oxide Semiconductor (CMOS) image sensor, or the like, or any combination thereof. In some embodiments, the imaging device 130-1 may include a plurality of components each of which can capture an image. For example, the imaging device 130-1 may include a plurality of sub-cameras that can capture image simultaneously. In some embodiments, the imaging device 130-1 may transmit the captured image to one or more components (e.g., the server 110, the storage device 140) of the helmet detection system 100 via the network 120.

The storage device 140 may be configured to store data and/or instructions. The data and/or instructions may be obtained from, for example, the server 110 and/or any other component of the helmet detection system 100. In some embodiments, the storage device 140 may store data and/or instructions that the server 110 may execute or use to perform exemplary methods described in the present disclosure. In some embodiments, the storage device 140 may include a mass storage, a removable storage, a volatile read-and-write memory, a read-only memory (ROM) , or the like, or any combination thereof.

In some embodiments, the storage device 140 may be connected to the network 120 to communicate with one or more components (e.g., the server 110, the detection device 130, ) of the helmet detection system 100. One or more components of the helmet detection system 100 may access the data or instructions stored in the storage device 140 via the network 120. In some embodiments, the storage device 140 may be directly connected to or communicate with one or more components (e.g., the server 110, the detection device 130) of the helmet detection system 100. In some embodiments, the storage device 140 may be part of other components of the helmet detection system 100, such as the server 110, or the detection device 130.

In some embodiments, the helmet detection system 100 may further include a user device (not shown) configured to receive information and/or data from the server 110, the detection device 130, and/or the storage device 140, via the network 120. For example, the user device may receive the image captured by the imaging device 130-1 from the imaging device 130-1 via the network 120. As another example, the user device may receive the target detection result associated with whether the target object is wearing a helmet or not from the server 110 via the network 120. In some embodiments, the user device may provide a user interface via which a user may view information and/or input data and/or instructions to the helmet detection system 100. For example, the user may view the image captured by the imaging device 130-1 and/or the target detection result associated with whether the target object is wearing a helmet or not via the user interface. As another example, the user may input an instruction (e.g., an instruction to open or close the channel of the detection device 130) associated with the helmet detection via the user interface. In some embodiments, the user device may include a mobile phone, a computer, a wearable device, or the like, or any combination thereof.

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.

FIG. 2 is a schematic diagram illustrating exemplary hardware and/or software components of an exemplary computing device according to some embodiments of the present disclosure. In some embodiments, the server 110 may be implemented on the computing device 200. For example, the processing device 112 may be implemented on the computing device 200 and configured to perform functions of the processing device 112 disclosed in this disclosure.

The computing device 200 may be used to implement any component of the helmet detection system 100 as described herein. For example, the processing device 112 may be implemented on the computing device 200, via its hardware, software program, firmware, or a combination thereof. Although only one such computer is shown, for convenience, the computer functions relating to helmet detection as described herein may be implemented in a distributed fashion on a number of similar platforms to distribute the processing load.

The computing device 200, for example, may include COM ports 250 connected to and from a network connected thereto to facilitate data communications. The computing device 200 may also include a processor (e.g., a processor 220) , in the form of one or more processors (e.g., logic circuits) , for executing program instructions. For example, the processor 220 may include interface circuits and processing circuits therein. The interface circuits may be configured to receive electronic signals from a bus 210, wherein the electronic signals encode structured data and/or instructions for the processing circuits to process. The processing circuits may conduct logic calculations, and then determine a conclusion, a result, and/or an instruction encoded as electronic signals. Then the interface circuits may send out the electronic signals from the processing circuits via the bus 210.

The computing device 200 may further include program storage and data storage of different forms including, for example, a disk 270, a read-only memory (ROM) 230, or a random-access memory (RAM) 240, for storing various data files to be processed and/or transmitted by the computing device 200. The computing device 200 may also include program instructions stored in the ROM 230, RAM 240, and/or another type of non-transitory storage medium to be executed by the processor 220. The methods and/or processes of the present disclosure may be implemented as the program instructions. The computing device 200 may also include an I/O component 260, supporting input/output between the computing device 200 and other components. The computing device 200 may also receive programming and data via network communications.

Merely for illustration, only one processor is illustrated in FIG. 2. Multiple processors 220 are also contemplated; thus, operations and/or method steps performed by one processor 220 as described in the present disclosure may also be jointly or separately performed by the multiple processors. For example, if in the present disclosure the processor 220 of the computing device 200 executes both step A and step B, it should be understood that step A and step B may also be performed by two different processors 220 jointly or separately in the computing device 200 (e.g., a first processor executes step A and a second processor executes step B, or the first and second processors jointly execute steps A and B) .

FIG. 3 is a block diagram illustrating an exemplary processing device according to some embodiments of the present disclosure. The processing device 112 may include an obtaining module 310, a determination module 320, and a training module 330.

The obtaining module 310 may be configured to obtain an image associated with a target object. More descriptions regarding the obtaining of the image associated with the target object may be found elsewhere in the present disclosure, for example, operation 410 in FIG. 4 and relevant descriptions thereof.

The determination module 320 may be configured to determine, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image associated with the target object. More descriptions regarding the determining of the initial detection result may be found elsewhere in the present disclosure, for example, operation 420 in FIG. 4 and relevant descriptions thereof. The determination module 320 may be configured to determine, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result. More descriptions regarding the determining of the intermediate detection result may be found elsewhere in the present disclosure, for example, operation 430 in FIG. 4 and relevant descriptions thereof. Further, the determination module 320 may be configured to determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result. More descriptions regarding the determining of the target detection result may be found elsewhere in the present disclosure, for example, operation 440 in FIG. 4 and relevant descriptions thereof.

The training module 330 may be configured to determine/obtain a detection model, a key-point extraction model, and/or a classification model. For example, the training module 330 may be configured to obtain a plurality of training samples. Further, the training module 330 may be configured to determine at least one of a detection model, a key-point extraction model, or a classification model by training at least one of a preliminary detection model, a preliminary key-point extraction model, and a preliminary classification model based on the plurality of marked training samples. More descriptions regarding the determining/obtaining of the detection model, the key-point extraction model, and/or the classification model may be found elsewhere in the present disclosure, for example, FIG. 12 and relevant descriptions thereof.

The modules in the processing device 112 may be connected to or communicate with each other via a wired connection or a wireless connection. The wired connection may include a metal cable, an optical cable, a hybrid cable, or the like, or any combination thereof. The wireless connection may include a Local Area Network (LAN) , a Wide Area Network (WAN) , a Bluetooth, a ZigBee, a Near Field Communication (NFC) , or the like, or any combination thereof.

It should be noted that the above description regarding the processing device 112 is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations and modifications may be made under the teachings of the present disclosure. In some embodiments, two or more of the modules may be combined as a single module, and any one of the modules may be divided into two or more units.

For example, the determination module 320 and the training module 330 may be combined as a single module which may both determine the target detection result and obtain at least one of the detection model, the key-point extraction model, or the classification model. As another example, the determination module 320 may be divided into a first determination unit configured to determine the initial detection result, a second determination unit configured to determine the intermediate detection result, and a third determination unit configured to determine the target detection result.

In some embodiments, the processing device 112 may include one or more additional modules. For example, the processing device 112 may also include a transmission module (not shown) configured to transmit signals (e.g., electrical signals, electromagnetic signals) to one or more components (e.g., the detection device 130) of the helmet detection system 100. As another example, the processing device 112 may include a storage module (not shown) used to store information and/or data (e.g., the image associated with the target object, the initial detection result, the intermediate detection result, the target detection result) associated with the helmet detection.

In some embodiments, the training module 330 may be implemented on a separate device (e.g., a processing device independent from the server 110) . In some embodiments, the training module 330 may be unnecessary and at least one of the detection model, the key-point extraction model, or the classification model may be obtained from a storage device (e.g., the storage device 140, an external storage device) disclosed elsewhere in the present disclosure.

FIG. 4 is a flowchart illustrating an exemplary process for helmet detection according to some embodiments of the present disclosure. In some embodiments, process 400 may be executed by the helmet detection system 100. For example, the process 400 may be implemented as a set of instructions (e.g., an application) stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) . In some embodiments, the processing device 112 (e.g., the processor 220 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 400. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 400 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 400 illustrated in FIG. 4 and described below is not intended to be limiting.

In 410, the processing device 112 (e.g., the obtaining module 310 illustrated in FIG. 3) (e.g., the interface circuits and/or processing circuits of the processor 220) may obtain an image associated with a target object.

In some embodiments, the target object may include a biological object. For example, the target object may be a person (e.g., a construction worker, a manager, visitor) entering or exiting a construction site.

In some embodiments, the processing device 112 may direct the detection device 130 (e.g., the imaging device 130-1) to capture the image associated with the target object. In some embodiments, the processing device 112 may obtain or determine the image from a video associated with the target object captured by the detection device 130 (e.g., the imaging device 130-1) . For example, the processing device 112 may perform a framing operation on the video to obtain the image from the video. In some embodiments, the image may be previously acquired by the detection device 130 (e.g., the imaging device 130-1) and stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) . The processing device 112 may obtain the image from the storage device via a network (e.g., the network 120) . In some embodiments, the processing device 112 may obtain the image from a third party (e.g., a search engine, a social media, a news media, a map website) .

In some embodiments, the processing device 112 (or the detection device 130) may determine the image associated with the target object based on an initial image (e.g., an initial image captured by the detection device 130) . In some embodiments, the processing device 112 may determine a count of object (s) in the initial image and determine whether the count is larger than 1. In response to determining that the count of the object (s) in the initial image is equal to 1, the processing device 112 may directly designate the initial image as the image associated with the target object. In response to determining that the count of the object (s) in the initial image is larger than 1, the processing device 112 may determine a head region (e.g., a region including the head of the object and an accessory (e.g., a helmet) worn on the head of the object) or an upper body region of each object in the initial image. The processing device 112 may designate an object with a largest head region or a largest upper body region as the target object and extract an image portion including the head region or an image portion including the upper body region from the initial image.

In 420, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image associated with the target object.

In some embodiments, the detection model may be pre-trained and stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) disclosed elsewhere in the present disclosure. The processing device 112 may retrieve the detection model from the storage device. In some embodiments, the detection model may include a machine learning model, for example, a neural network model. The neural network model may include a multi-layer perceptron (MLP) model, a deep neural network (DNN) model, a convolutional neural network (CNN) model, a deep convolutional encoder-decoder (DCED) network model, a generative adversarial network (GAN) model, or the like, or any combination thereof. In some embodiments, the detection model may be trained based on a plurality of training samples. More descriptions regarding the detection model may be found elsewhere in the present disclosure (e.g., FIG. 12 and the description thereof) .

In some embodiments, the processing device 112 may input the image associated with the target object into the detection model and determine the initial detection result associated with whether the target object is wearing a helmet or not based on an output of the detection model.

In some embodiments, the initial detection result may include a determination result of whether there is a head region of the target object in the image. In some embodiments, when there is no head region in the image, the processing device 112 may directly determine a result that “the target object is not wearing a helmet. ” In some embodiments, when there is a head region in the image, the initial detection result may also include an extracted image (or image portion) of the head region of the target object.

In some embodiments, the initial detection result may include position information of the head region of the target object and/or a determination result of whether there is a helmet in the head region of the target object. The position information of the head region of the target object may include coordinate information (e.g., coordinates in a target coordinate system (e.g., a coordinate system 540 illustrated in FIG. 5A) ) of pixels in the head region of the target object, an area of the head region, etc. In some embodiments, the initial detection result may also include a confidence level corresponding to the determination result, which indicates a credibility or a probability of the determination result. In some embodiments, the confidence level may be expressed as a value between 0 and 1. The larger the value is, the higher the confidence level may be. For example, it is assumed that there is a helmet in the head region of the target object, the determination result may be 1 and the confidence level corresponding to the determination result may be a value greater than 0 and less than or equal to 1; it is assumed that there is no helmet in the head region of the target object, the determination result may be 0 and the confidence level corresponding to the determination result may be 0.

In some embodiments, the initial detection result may further include a determination result of whether there is a hand region at least partially located within the head region of the target object. In some embodiments, the initial detection result may also include position information of the hand region. The position information of the hand region may include coordinate information of pixels in the hand region, an area of the hand region, etc. In some embodiments, the initial detection result may also include a confidence level corresponding to the determination result, which indicates a credibility or a probability of the determination result.

In some embodiments, the processing device 112 may determine a head region and a hand region of the target object in the image of the target object by using the detection model. When an overlap region between the head region and the hand region is larger than or equal to a preset value, an output of the detection model may include a determination result that there is a hand region at least partially located within the head region of the target object; when the overlap region between the head region and the hand region of the target object is less than the preset value, the output of the detection model may include a determination result that there is no hand region at least partially located within the head region of the target object. In some embodiments, when there is a hand region at least partially located within the head region of the target object, the determination result may be 1 and the confidence level corresponding to the determination result may be a value greater than 0 and less than or equal to 1; when there is no hand region at least partially located within the head region of the target object, the determination result may be 0 and the confidence level corresponding to the determination result may be 0. More descriptions of the initial detection result may be found elsewhere in the present disclosure (e.g., FIGs. 5A-C and the description thereof) .

In 430, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result.

In some embodiments, the key-point extraction model may be pre-trained and stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) disclosed elsewhere in the present disclosure. The processing device 112 may retrieve the key-point extraction model from the storage device. In some embodiments, the key-point extraction model may include a machine learning model, for example, a neural network model. In some embodiments, the key-point extraction model may be the same as or different from the detection model.

In some embodiments, the key-point extraction model may be connected to the detection model. For example, the key-point extraction model and the detection model may be two neural network models connected to each other. In some embodiments, the output of the detection model may be used as an input of the key-point extraction model. For example, the processing device 112 may input the initial detection result or a portion (e.g., an image portion of the head region) thereof into the key-point extraction model and determine the intermediate detection result associated with whether the target object is wearing a helmet or not based on an output of the key-point extraction model.

In some embodiments, the key-point extraction model may be trained separately or jointly with the detection model. More descriptions regarding the key-point extraction model may be found elsewhere in the present disclosure (e.g., FIG. 12 and the description thereof) .

In some embodiments, the intermediate detection result may be associated with a plurality of key points associated with the target object. In some embodiments, the processing device 112 may determine, using the key-point extraction model, the plurality of key points associated with the target object in an image portion (e.g., an image portion of the head region) indicated by the at least portion of the initial detection result. The processing device 112 may input the initial detection result or a portion thereof into the key-point extraction model and determine the plurality of key points in the image portion of the head region based on an output of the key-point extraction model.

In some embodiments, the plurality of key points may represent a shape feature of a target region associated with the head region of the target object. In some embodiments, the target region may be a region where an accessory (e.g., a helmet) may be worn on the head of the target object, for example, a region in a vicinity of and above a forehead of the targe object. In some embodiments, the plurality of key points may be edge points of the target region or points in a vicinity of the edge of the target region. In some embodiments, the plurality of key points may include a first point located at a top location of the edge of the target region, a second point located at a far-left location of the edge of the target region, a third point located at a far-right location of the edge of the target region, a fourth point located between the first point and the second point, and a fifth point located between the first point and the third point. More descriptions regarding the plurality of key points may be found elsewhere in the present disclosure (e.g., FIGs. 6A and 6B and the descriptions thereof) .

Further, according to the plurality of key points, the processing device 112 may determine the intermediate detection result associated with whether the target object is wearing a helmet or not. Since a shape of an actual helmet is different from that of a disguise (e.g., a paper, a headband) and the plurality of key points represents the shape feature of the target region (e.g., a helmet region) associated with the head region of the target object, the processing device 112 can determine the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points accurately. In some embodiments, the intermediate detection result may include a determination result of whether there is a helmet in a head region of the target object and/or a confidence level corresponding to the determination result. More descriptions of the intermediate detection result may be found elsewhere in the present disclosure (e.g., FIGs. 6-8 and the descriptions thereof) .

In 440, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.

In some embodiments, when both the initial detection result and the intermediate detection result indicate that there is a helmet in the head region of the target object and the initial detection result indicates that there is no hand region at least partially located within the head region of the target object, the processing device 112 may determine the target detection result that the target object is wearing a helmet. Further, the processing device 112 may also determine a target confidence level corresponding to the target detection result, for example, an average or a weighted average of corresponding confidence levels corresponding to the initial detection result and the intermediate detection result. For example, the processing device 112 may determine the target confidence level according to formula (1) below:

tar_conf= (α*det_head_conf+β*K_hat_type_conf) /2 (1)

where tar_conf refers to the target confidence level, det_head_conf refers to a confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the initial detection result, K_hat_type_conf refers to a confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the intermediate detection result, α refers to a weight of det_head_conf, and β refers to a weight of K_hat_type_conf.

In some embodiments, a weight (e.g., α) of the confidence level corresponding to the initial detection result may be equal to a weight (e.g., β) of the confidence level corresponding to the intermediate detection result. For example, the weight (e.g., α) of the confidence level corresponding to the initial detection result and the weight (e.g., β) of the confidence level corresponding to the intermediate detection result may be equal to 1. In some embodiments, since the accuracy of the initial detection result may be greater than the intermediate detection result, the weight (e.g., α) of the confidence level corresponding to the initial detection result may be larger than or equal to the weight (e.g., β) of the confidence level corresponding to the intermediate detection result.

In some embodiments, when both the initial detection result and the intermediate detection result indicate that there is a helmet in the head region of the target object and the initial detection result indicates that there is a hand region at least partially located within the head region of the target object, the processing device 112 may determine a target confidence level based on confidence levels corresponding to the initial detection result and the intermediate detection result and determine the target detection result based on the target confidence level, for example, by comparing the target confidence level with a confidence level threshold.

For example, the processing device 112 may determine the target confidence level according to formula (2) below:

tar_conf= (α*det_head_conf+β*K_hat_type_conf) /2-γ*det_hand_conf (2) ,

where tar_conf refers to the target confidence level, det_head_conf refers to the confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the initial detection result, K_hat_type_conf refers to the confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the intermediate detection result, det_hand_conf refers to the confidence level corresponding to the determination result of whether there is a hand region at least partially located within the head region of the target object in the initial detection result, α refers to a weight of det_head_conf, β refers to a weight of K_hat_type_conf, and γ refers to a weight of det_hand_conf. In some embodiments, γ may be a default setting (e.g., 0.5) of the helmet detection system 100 or may be adjustable under different situations. default.

Further, the processing device 112 may determine whether the target confidence level is larger than a confidence level threshold. The confidence level threshold may be a default setting (e.g., 0.5) of the helmet detection system 100 or may be adjustable under different situations. In response to determining that the target confidence level is larger than or equal to the confidence level threshold, the processing device 112 may determine the target detection result that the target object is wearing a helmet. In response to determining that the target confidence level is less than the confidence level threshold, the processing device 112 may determine the target detection result that the target object is not wearing a helmet.

According to the embodiment above, since a disguise (e.g., a paper, a headband) usually requires hand support, when the initial detection result indicates that there is a hand region at least partially located within the head region of the target object, the target detection result is determined based on the target confidence level, which can improve the accuracy of the helmet detection.

In some embodiments, when the initial detection result or the intermediate detection result indicates that there is no helmet in the head region of the target object, the processing device 112 may determine the target detection result that the target object is not wearing a helmet.

In some embodiments, the processing device 112 may also determine, using a classification model, a second intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result. Further, the processing device 112 may determine the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result. More descriptions may be found elsewhere in the present disclosure (e.g., FIG. 9 and the description thereof) .

It should be noted that the above description is merely provided for the purposes of illustration, and not intended to limit the scope of the present disclosure. For persons having ordinary skills in the art, multiple variations or modifications may be made under the teachings of the present disclosure. However, those variations and modifications do not depart from the scope of the present disclosure.

For example, one or more other optional operations (e.g., a storing operation, a transmitting operation) may be added elsewhere in the process 400. In the storing operation, the processing device 112 may store information and/or data (e.g., the image associated with the target object, the initial detection result, the intermediate detection result, the target detection result) associated with the helmet detection in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) disclosed elsewhere in the present disclosure. In the transmitting operation, the processing device 112 may transmit the target detection result associated with whether the target object is wearing a helmet or not to other components (e.g., the detection device 130) of the helmet detection system 100.

FIGs. 5A-5C are schematic diagrams illustrating exemplary initial detection results according to some embodiments of the present disclosure.

As illustrated in FIGs. 5A-5C, an image A of a target object 510, an image B of a target object 520, and an image C of a target object 530 are inputted into the detection model and the detection model outputs corresponding initial detection result (s) .

As described in connection with operation 420, the initial detection result may include a determination result of whether there is a head region of the target object in the image, an image portion of the head region of the target object, position information of the head region of the target object, a determination result of whether there is a helmet in the head region of the target object, a confidence level corresponding to the determination result, a determination result of whether there is a hand region at least partially located within the head region of the target object, a confidence level corresponding to the determination result, position information of the hand region of the target object, etc.

As illustrated in FIG. 5A, the initial detection result includes a determination result that “there is a head region in the image A, ” an image portion A’ of the head region of the target object 510, position information ( “head_pos” ) of the head region, a determination result ( “det_head type = 1” ) that there is a helmet in the head region of the target object, a confidence level ( “det_head_conf = 0.9” ) corresponding to the determination result, a determination result ( “det_hand_type = 0” ) that there is no hand region at least partially located within the head region of the target object, and a confidence level ( “det_hand_conf = 0” ) corresponding to the determination result.

As illustrated in FIG. 5B, the initial detection result includes a determination result that “there is a head region in the image B, ” an image portion B’ of the head region of the target object 520, position information ( “head_pos” ) of the head region, a determination result ( “det_head type = 1” ) that there is a helmet in the head region of the target object, a confidence level ( “det_head_conf = 0.9” ) corresponding to the determination result, a determination result ( “det_hand_type = 1” ) that there is a hand region at least partially located within the head region of the target object, a confidence level ( “det_hand_conf = 0.9” ) corresponding to the determination result, and position information ( “hand_pos” ) of the hand region.

As illustrated in FIG. 5C, the initial detection result includes a determination result that “there is a head region in the image C, ” an image portion C’ of the head region of the target object 530, position information ( “head_pos” ) of the head region, a determination result ( “det_head type = 0” ) that there is no helmet in the head region of the target object, a confidence level ( “det_head_conf = 0” ) corresponding to the determination result, a determination result ( “det_hand_type = 0” ) that there is no hand region at least partially located within the head region of the target object, a confidence level ( “det_hand_conf = 0” ) corresponding to the determination result.

FIGs. 6A-6C are schematic diagrams illustrating exemplary intermediate detection results according to some embodiments of the present disclosure.

As illustrated in FIGs. 6A-6C, the image portion A’ illustrated in FIG. 5A, the image portion C’ illustrated in FIG. 5C, and an image portion D are inputted into the key-point extraction model and the key-point extraction model outputs corresponding intermediate detection result (s) .

As described in connection with operation 430, the intermediate detection result may be associated with a plurality of key points associated with the target object and include a determination result of whether there is a helmet in a head region of the target object, a confidence level corresponding to the determination result, etc.

As illustrated in FIGs. 6A-6C, the plurality of key points includes a first point T_key, a second point Dl_key, a third point Dr_key, a fourth point Ml_key, and a fifth point Mr_key. The first point T_key is located at a top location of an edge of a target region (e.g., a region in a vicinity of and above a forehead) ; the second point Dl_key is located at a far-left location of the edge of the target region; the third point Dr_key is located at a far-right location of the edge of the target region; the fourth point Ml_key is located at a location of the edge of the target region between the first point and the second point; the fifth point Mr_key is located at a location of the edge of the target region between the first point and the third point.

Further, as illustrated in FIG. 6A, the intermediate detection obtained based on the image portion A’ includes a determination result ( “K_hat_type = 1” ) that there is a helmet in a head region of the target object and a confidence level corresponding to the determination result, for example,

K_hat_type_conf= (T_key_conf+Ml_key_conf+Mr_key_conf+Dl_key_conf+Dr_key_conf) /5.

Detailed descriptions regarding determining the determination result can be found in FIG. 7 and the description thereof.

As illustrated in FIG. 6B, the intermediate detection obtained based on the image portion B’ includes a determination result ( “K_hat_type = 0” ) that there is no helmet in a head region of the target object and a confidence level corresponding to the determination result, for example,

As illustrated in FIG. 6C, the intermediate detection obtained based on the image portion D includes a determination result ( “K_hat_type = 0” ) that there is no helmet in a head region of the target object and a confidence level corresponding to the determination result, for example,

FIG. 7 is a flowchart illustrating an exemplary process for determining an intermediate detection result associated with whether a target object is wearing a helmet or not according to some embodiments of the present disclosure. In some embodiments, process 700 may be executed by the helmet detection system 100. For example, the process 700 may be implemented as a set of instructions (e.g., an application) stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) . In some embodiments, the processing device 112 (e.g., the processor 220 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 700. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 700 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 700 illustrated in FIG. 7 and described below is not intended to be limiting.

In 710, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine whether one or more of a plurality of key points are located at a target position of an image portion indicated by the at least portion of an initial detection result.

As described in connection with FIG. 4, the image portion indicated by the at least portion of the initial detection result may be an image portion (e.g., the image portion A’ , the image portion B’ , and the image portion C’ illustrated in FIGs. 5A-5C) of the head region of the target object. In some embodiments, the target position may be an upper left corner (e.g., the origin of the coordinate system 540 illustrated in FIG. 5A) of the image portion.

In 720, in response to determining that one or more of the plurality of key points are located at the target position, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a determination result that there is no helmet in the head region of the target object.

In some embodiments, the processing device 112 may also determine a confidence level corresponding to the determination result based on the plurality of key points and corresponding confidence levels. Specifically, the processing device 112 may determine the confidence level corresponding to the determination result based on the one or more of the plurality of key points that are located at the target position. For example, when a count of the key point (s) located at the target position is equal to 1, the processing device 112 may designate a confidence level corresponding to the key point as the confidence level corresponding to the determination result. As another example, when the count of the key point (s) located at the target position is larger than 1, the processing device 112 may determine an average of confidence levels corresponding to the key points located at the target position as the confidence level corresponding to the determination result.

In 730, in response to determining that no key point is located at the target position, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may divide the plurality of key points into two groups of key points including a same key point.

As described in connection with FIG. 4, the plurality of key points may include a first point located at a top location of the edge of the target region, a second point located at a far-left location of the edge of the target region, a third point located at a far-right location of the edge of the target region, a fourth point located between the first point and the second point, and a fifth point located between the first point and the third point. In some embodiments, the same key point may be the first point. Accordingly, the two groups of key points may include a first group including the first point, the second point, and the fourth point and a second group including the first point, the third point, and the fifth point.

In 740, for each of the two groups of key points, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a connection line from the same key point to a target point.

In some embodiments, for the first group including the first point, the second point, and the fourth point, the processing device 112 may designate the second point as the target point and determine a connection line (also referred to as a “first connection line, ” “L1” illustrated in FIG. 8A) from the first point (i.e., the same point) to the second point; for the second group including the first point, the third point, and the fifth point, the processing device 112 may designate the third point as the target point and determine a connection line (also referred to as a “second connection line, ” “L2” illustrated in FIG. 8A) from the first point (i.e., the same point) to the third point.

In some embodiments, for the first group including the first point, the second point, and the fourth point, the processing device 112 may designate the fourth point as the target point and determine a connection line (also referred to as a “third connection line” ) from the first point (i.e., the same point) to the fourth point; for the second group including the first point, the third point, and the fifth point, the processing device 112 may designate the fifth point as the target point and determine a connection line (also referred to as a “fourth connection line” ) from the first point (i.e., the same point) to the fifth point.

In 750, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a determination result that there is a helmet in the head region of the target object in response to that connection lines corresponding to the two groups of key points satisfy a preset condition.

In some embodiments, the preset condition may be that: absolute values of slopes of the first connection line (e.g., “L1” illustrated in FIG. 8A) and the second connection line (e.g., “L2” illustrated in FIG. 8A) are less than or equal to a first slope threshold, the fourth point is on an outside (e.g., left side) of the first connection line, and the fifth point is on an outside (e.g., right side) of the second connection line.

Merely by way of example, the processing device 112 may determine the first connection line as a formula (3) below:

where x _L1 and y _L1 refer to coordinates of points at the first connection line in a target coordinate system (e.g., a coordinate system 540 illustrated in FIG. 8A) , x1 and y1 refer to coordinates of the first point in the target coordinate system, x2 and y2 refer to coordinates of the second point in the target coordinate system.

Further, the processing device 112 may determine a virtual point on the first connection line corresponding to the fourth point according to formula (4) below:

where x4 and y4′ refer to coordinates of the virtual point in the target coordinate system.

Similarly, the processing device 112 may determine the second connection line as a formula (5) below:

where x _L2 and y _L2 refer to coordinates of points at the second connection line in the target coordinate system, x3 and y3 refer to coordinates of the third point in the target coordinate system.

Further, the processing device 112 may determine a virtual point on the second connection line corresponding to the fifth point according to formula (6) below:

where x5 and y5′ refer to coordinates of the virtual point in the target coordinate system.

If y4′>y4, y ₅′>y5,

and

the processing device 112 may determine that the first connection line and the second connection line satisfy the preset condition. x4 and y4 refer to coordinates of the fourth point in the target coordinate system, x5 and y5 refer to coordinates of the fifth point in the target coordinate system.

In some embodiments, the preset condition may be that: absolute values of slopes of the third connection line (e.g., “L3” illustrated in FIG. 8B) and the fourth connection line (e.g., “L4” illustrated in FIG. 8B) are less than or equal to a second slope threshold, the second point is on an inside (e.g., right side) of the third connection line, and the third point is on an inside (e.g., left side) of the fourth connection line. The processing device 112 may also determine the third connection line and the fourth connection line in a similar way with above, which is not repeated here.

In some embodiments, the first slope threshold and/or the second slope threshold may be set empirically or according to an actual need. For example, the first slope threshold and/or the second slope threshold may be determined based on a general helmet size. Since the general helmet size is different from a size of a head of the target object, when the preset condition that the absolute values of the slopes of the first connection line and the second connection line are less than or equal to the first slope threshold (or the absolute values of the slopes of the third connection line and the fourth connection line are less than or equal to a second slope threshold) is satisfied, the determination result that there is a helmet in the head region of the target object may be determined.

In some embodiments, the processing device 112 may also determine a confidence level corresponding to the determination result based on the plurality of key points and corresponding confidence levels. For example, the processing device 112 may determine an average of confidence levels corresponding to the plurality of key points as the confidence level corresponding to the determination result.

FIGs. 8A and 8B are schematic diagrams illustrating exemplary operations for determining an intermediate detection result associated with whether a target object is wearing a helmet or not according to some embodiments of the present disclosure.

As illustrated in FIGs. 8A and 8B, the plurality of key points include a first point T_key, a second point Dl_key, a third point Dr_key, a fourth point Ml_key, and a fifth point Mr_key. The processing device 112 may determine whether one or more of the first point T_key, the second point Dl_key, the third point Dr_key, the fourth point Ml_key, and the fifth point Mr_key are located at a target position (e.g., the origin of the coordinate system 540) of the image portion A’ . In response to determining that no key point is located at the target position, the processing device 112 may divide the key points into a first group including the first point T_key, the second point Dl_key, and the fourth point Ml_key and a second group including the first point T_key, the third point Dr_key, and the fifth point Mr_key.

In some embodiments, as illustrated in FIG. 8A, for the first group of key points, the processing device 112 may determine a first connection line L1 from the first point T_key to the second point Dl_key; for the second group of key points, the processing device 112 may determine a second connection line L2 from the first point T_key to the third point Dr_key. Further, as described in connection with operation 750, the processing device 112 may determine whether the first connection line L1 and the second connection line L2 satisfy a preset condition. In response to that first connection line L1 and the second connection line L2 satisfy the preset condition, the processing device 112 may determine a determination result that there is a helmet in the head region of the target object.

In some embodiments, as illustrated in FIG. 8B, for the first group of key points, the processing device 112 may determine a third connection line L3 from the first point T_key to the fourth point MI_key; for the second group of key points, the processing device 112 may determine a fourth connection line L4 from the first point T_key to the fifth point Mr_key. Further, as described in connection with operation 750, the processing device 112 may determine whether the third connection line L3 and the fourth connection line L4 satisfy a preset condition. In response to that the third connection line L3 and the fourth connection line L4 satisfy the preset condition, the processing device 112 may determine a determination result that there is a helmet in the head region of the target object.

FIG. 9 is a flowchart illustrating an exemplary process for helmet detection according to some embodiments of the present disclosure. In some embodiments, process 900 may be executed by the helmet detection system 100. For example, the process 900 may be implemented as a set of instructions (e.g., an application) stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) . In some embodiments, the processing device 112 (e.g., the processor 220 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 900. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 900 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 900 illustrated in FIG. 9 and described below is not intended to be limiting.

In 910, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine, using a classification model, a second intermediate detection result associated with whether a target object is wearing a helmet or not based on at least portion of the initial detection result.

In some embodiments, the classification model may be pre-trained and stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) disclosed elsewhere in the present disclosure. The processing device 112 may retrieve the classification model from the storage device. In some embodiments, the classification model may include a machine learning model, for example, a neural network model. In some embodiments, the classification model may be the same as or different from the detection model and/or the key-point extraction model.

In some embodiments, the classification model may be connected to the detection model. For example, the classification model and the detection model may be two neural network models connected to each other. In some embodiments, the output of the detection model may be used as an input of the classification model. For example, the processing device 112 may input the initial detection result or a portion (e.g., an image portion of the head region) thereof into the classification model and determine the second intermediate detection result based on an output of the classification model.

In some embodiments, the classification model may be trained separately or jointly with the detection model and/or a key-point extraction model. More descriptions regarding the classification model may be found elsewhere in the present disclosure (e.g., FIG. 12 and the description thereof) .

In some embodiments, the second intermediate detection result may include a determination result of whether there is a helmet in the head region of the target object, a confidence level corresponding to the determination result, a color of the helmet, and/or a confidence level corresponding to the color of the helmet.

In 920, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result, the intermediate detection result, and the second intermediate detection result.

In some embodiments, when all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in the head region of the target object, the initial detection result indicates that there is no hand region at least partially located within the head region of the target object, and the second intermediate detection result indicates that the color of the helmet, the processing device 112 may determine the target detection result that the target object is wearing a helmet and a color of the helmet. Further, the processing device 112 may also determine a confidence level corresponding to the target detection result, for example, an average or a weighted average of the corresponding confidence levels corresponding to the initial detection result, the intermediate detection result, and the second intermediate detection result. For example, the processing device 112 may determine the target confidence level according to formula (7) below:

tar_conf= (α*det_head_conf+β*K_hat_type_conf+δ*cls_hat_type_conf) /3 (7)

where tar_conf refers to the target confidence level, det_head_conf refers to a confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the initial detection result, K_hat_type_conf refers to a confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the intermediate detection result, cls_hat_type_conf refers to a confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the second intermediate detection result, α refers to a weight of det_head_conf, β refers to a weight of K_hat_type_conf, and δ refers to a weight of cls_hat_type_conf.

In some embodiments, when all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in the head region of the target object, the initial detection result indicates that there is a hand region at least partially located within the head region of the target object, and the second intermediate detection result indicates that the color of the helmet, the processing device 112 may determine a target confidence level based on confidence levels corresponding to the initial detection result, the intermediate detection result, the second intermediate detection result, and the determination result of whether there is a hand region at least partially located within the head region of the target object in the initial detection result, for example, by comparing the target confidence level with a confidence level threshold.

For example, the processing device 112 may determine the target confidence level according to formula (8) below:

tar_conf= (α*det_head_conf+β*K_hat_type_conf+δ*cls_hat_type_conf) /3-γ*det_hand_conf (8) ,

where tar_conf refers to the target confidence level, det_head_conf refers to the confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the initial detection result, K_hat_type_conf refers to the confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the intermediate detection result, cls_hat_type_conf refers to the confidence level corresponding to the determination result of whether there is a helmet in the head region of the target object in the second intermediate detection result, det_hand_conf refers to the confidence level corresponding to the determination result of whether there is a hand region at least partially located within the head region of the target object in the initial detection result, αrefers to a weight of det_head_conf, β refers to a weight of K_hat_type_conf, δ refers to a weight of cls_hat_type_conf, and γ refers to a weight of det_hand_conf.

In some embodiments, α may be equal to β and δ. For example, α, β, and δmay be equal to 1. In some embodiments, since the accuracy of the initial detection result may be greater than the intermediate detection result and/or second intermediate detection result, αmay be larger than β and/or δ. In some embodiments, since the accuracy of the second intermediate detection result may be greater than the intermediate detection result, δ may be larger than β. In some embodiments, γ may be a default setting (e.g., 0.5) of the helmet detection system 100 or may be adjustable under different situations. default.

Further, the processing device 112 may determine whether the target confidence level is larger than a confidence level threshold. The confidence level threshold may be a default setting (e.g., 0.5) of the helmet detection system 100 or may be adjustable under different situations. In response to determining that the target confidence level is larger than or equal to the confidence level threshold, the processing device 112 may determine the target detection result that the target object is wearing a helmet and the color of the helmet is yellow. In response to determining that the target confidence level is less than the confidence level threshold, the processing device 112 may determine the target detection result that the target object is not wearing a helmet.

In some embodiments, when at least one of the initial detection result, the intermediate detection result, and the second intermediate detection result indicates that there is no helmet in the head region of the target object, the processing device 112 may determine the target detection result that the target object is not wearing a helmet. Further, the processing device 112 may also determine the target confidence level corresponding to the target detection result, for example, as expressed in formula (7) above, the average or the weighted average of corresponding confidence levels corresponding to the initial detection result, the intermediate detection result, and the second intermediate detection result.

In some embodiments, when the second intermediate detection result does not indicate the color of the helmet (e.g., a color result in the second intermediate detection result is not within preset colors of the classification model) , the processing device 112 may determine the target detection result that the target object is not wearing a helmet. Further, the processing device 112 may designate a confidence level corresponding to the color of the helmet as the target confidence level corresponding to the target detection result.

FIG. 10 is a schematic diagram illustrating an exemplary second intermediate detection result according to some embodiments of the present disclosure.

As illustrated in FIG. 10, the image portion A’ illustrated in FIG. 5A is inputted into the classification model and the classification model outputs corresponding second intermediate result (s) .

As described in connection with operation 910, the second intermediate detection result may include a determination result of whether there is a helmet in the head region of the target object, a confidence level corresponding to the determination result, a color of the helmet, a confidence level corresponding to the color of the helmet, etc. As illustrated in FIG. 10, the second intermediate detection result includes a determination result ( “cls_hat_type = 1” ) that there is a helmet in the head region of the target object, a confidence level (cls_hat_type_conf = 0.9) corresponding to the determination result, a color (cls_hat_color = other color) of the helmet, and a confidence level (cls_hat_color_conf =0.8) corresponding to the color of the helmet.

FIG. 11 is a flowchart illustrating an exemplary process for helmet detection according to some embodiments of the present disclosure. In some embodiments, process 1100 may be executed by the helmet detection system 100. For example, the process 1100 may be implemented as a set of instructions (e.g., an application) stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) . In some embodiments, the processing device 112 (e.g., the processor 220 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 1100. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 1100 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 1100 illustrated in FIG. 11 and described below is not intended to be limiting.

In 1101, the processing device 112 (e.g., the obtaining module 310 illustrated in FIG. 3) (e.g., the interface circuits and/or processing circuits of the processor 220) may obtain an image associated with a target object. The image associated with the target object may be obtained in a similar manner as operation 410, and relevant descriptions are not repeated here.

In 1102, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image.

In 1103, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine whether there is a head region of the target object in the image based on the initial detection result. In some embodiments, if there is a head region of the target object in the image, the processing device 112 may proceed to perform operation 1104; if there is no head region of the target object in the image of the target object, the processing device 112 may proceed to perform operation 1112.

In 1104, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may extract an image portion of the head region of the target object based on the initial detection result.

In 1105, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a determination result associated with whether there is a hand region based on the initial detection result.

In 1106, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result.

In 1107, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine, using a classification model, a second intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result.

In 1108, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine whether the target object is wearing a helmet or not based at least in part on the initial detection result, the intermediate detection result, and the second intermediate detection result. Whether the target object is wearing a helmet may be determined in a similar manner as operation 920, and relevant descriptions are not repeated here. In some embodiments, if the target object is wearing a helmet, the processing device 112 may proceed to perform operation 1109; if the target object is not wearing a helmet, the processing device 112 may proceed to perform operation 1110.

In 1109, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a target detection result that the target object is wearing a helmet and a color of the helmet.

In 1110, the processing device 112 (e.g., the determination module 320 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine a target detection result that the target object is not wearing a helmet.

FIG. 12 is a flowchart illustrating an exemplary process for determining a detection model, a key-point extraction model, and/or a classification model according to some embodiments of the present disclosure. In some embodiments, process 1200 may be executed by the helmet detection system 100. For example, the process 1200 may be implemented as a set of instructions (e.g., an application) stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) . In some embodiments, the processing device 112 (e.g., the processor 220 of the computing device 200 and/or one or more modules illustrated in FIG. 3) may execute the set of instructions and may accordingly be directed to perform the process 1200. Alternatively, the process 1200 may be performed by a computing device of a system of a vendor that provides and/or maintains the detection model, the key-point extraction model, and/or the classification model. The operations of the illustrated process presented below are intended to be illustrative. In some embodiments, the process 1200 may be accomplished with one or more additional operations not described and/or without one or more of the operations discussed. Additionally, the order of the operations of process 1200 illustrated in FIG. 12 and described below is not intended to be limiting.

In 1210, the processing device 112 (e.g., the training module 330 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may obtain a plurality of training samples.

In some embodiments, each of the plurality of training samples may be a sample image including a sample object. For example, the sample image may be an image acquired by the detection device 130 (e.g., the imaging device 130-1) or other external imaging devices. In some embodiments, at least a portion of the plurality of training samples may be previously generated and stored in a storage device (e.g., the storage device 140, the ROM 230, and/or the RAM 240) disclosed elsewhere in the present disclosure or an external storage device. The processing device 112 may retrieve the plurality of training samples directly from the storage device or the external storage device. In some embodiments, at least a portion of the plurality of training samples may be generated by the processing device 112. Merely by way of example, the processing device 112 may direct the detection device 130 (e.g., the imaging device 130-1) to capture the sample image of at least one object.

In some embodiments, each of the plurality of training samples may be marked manually by a technician using an application that is set or loaded on the processing device 112 or an external processing device. In some embodiments, each of the plurality of training samples may be automatically marked by the processing device 112 or an external processing device.

In some embodiments, for each of the plurality of training samples, at least one of whether there is a head region of a sample object in the training sample, a position of the head region of the sample object, whether the sample object wears a helmet or not, a color of the helmet, whether there is a hand region at least partially located within the head region of the sample object, a position of the hand region, or positions of a plurality of key points associated with the sample object may be marked in the training sample. More descriptions of the marking of the plurality of training samples may be found elsewhere in the present disclosure (e.g., FIG. 13 and the description thereof) .

In some embodiments, the plurality of training samples may be augmented by moving the head region of the sample object. For example, for a training sample, after making the training sample, the technician may randomly adjust the position of the head region of the sample object in the training sample to obtain a new training sample.

In 1220, the processing device 112 (e.g., the training module 330 illustrated in FIG. 3) (e.g., the processing circuits of the processor 220) may determine at least one of a detection model, a key-point extraction model, or a classification model by training at least one of a preliminary detection model, a preliminary key-point extraction model, and a preliminary classification model based on the plurality of marked training samples.

In some embodiments, each of the preliminary detection model, the preliminary key-point extraction model, and the preliminary classification model may include a machine learning model, for example, a neural network model. As described in connection with FIG. 4, the neural network model may include a multi-layer perceptron (MLP) model, a deep neural network (DNN) model, a convolutional neural network (CNN) model, a deep convolutional encoder-decoder (DCED) network model, a generative adversarial network (GAN) model, or the like, or any combination thereof. In some embodiments, each of the preliminary detection model, the preliminary key-point extraction model, and the preliminary classification model may include at least one preliminary model parameter. The at least one preliminary model parameter may be set according to a default setting of the helmet detection system 100 or may be adjustable under different situations. Taking a convolutional neural network (CNN) model as an example, the at least one preliminary model parameter may include a count of convolutional layers, a count of kernels, a kernel size, a stride, a padding of each convolutional layer, or the like, or any combination thereof.

In some embodiments, the processing device 112 may train at least one of the preliminary detection model, the preliminary key-point extraction model, or the preliminary classification model iteratively until a termination condition is satisfied. In response to that the termination condition is satisfied, the at least one of the detection model, the key-point extraction model, or the classification model may be finalized. In some embodiments, the termination condition may relate to a value of a loss function. For example, the termination condition may be deemed satisfied if the value of the loss function is minimal or smaller than a predetermined threshold. As another example, the termination condition may be deemed satisfied if the value of the loss function converges. In some embodiments, “convergence” may refer to that the variation of the values of the loss function in two or more consecutive iterations is equal to or smaller than a predetermined threshold. In some embodiments, “convergence” may refer to that a difference between the value of the loss function and a target value is equal to or smaller than a predetermined threshold. In some embodiments, the termination condition may be deemed satisfied when a specified count of iterations have been performed in the training process.

In some embodiments, the preliminary detection model, the preliminary key-point extraction model, and the preliminary classification model may be trained jointly. In some embodiments, the preliminary key-point extraction model, and the preliminary classification model may be connected with the preliminary detection model, respectively. In each iteration, a marked training sample may be used as an input of the preliminary detection model or an intermediate detection model obtained in a previous iteration. The preliminary detection model or the intermediate detection model may generate an estimated initial detection result associated with whether the sample object is wearing a helmet or not. Merely by way of example, the estimated initial detection result may include position information of a head region of the sample object, a determination result of whether there is a helmet in the head region of the sample object, a determination result of whether there is a hand region at least partially located within the head region of the sample object, and/or position information of the hand region. At least portion of the estimated initial detection result may be used as an input of the preliminary key-point extraction model (or an intermediate key-point extraction model in the previous iteration) and the preliminary classification model (or an intermediate classification model in the previous iteration) , respectively. The preliminary key-point extraction model or the intermediate key-point extraction model may generate an estimated intermediate detection result associated with whether the sample object is wearing a helmet or not. Merely by way of example, the estimated intermediate detection result may include positions of a plurality of key points. The preliminary classification model or the intermediate classification model may generate an estimated second intermediate detection result associated with whether the sample object is wearing a helmet or not. Merely by way of example, the estimated second intermediate detection result may include a determination result of whether there is a helmet in a head region of the sample object and/or a color of the helmet.

In some embodiments, the preliminary detection model, the preliminary key-point extraction model, and the preliminary classification model may be trained separately. In each iteration, a marked training sample may be used as an input of the preliminary detection model (or an intermediate detection model) , the preliminary key-point extraction model (or an intermediate key-point extraction model) , and the preliminary classification model (or an intermediate classification model) , respectively. The preliminary detection model (or an intermediate detection model) , the preliminary key-point extraction model (or an intermediate key-point extraction model) , and the preliminary classification model (or an intermediate classification model) may generate the estimated initial detection result, the estimated intermediate detection result, and the estimated second intermediate detection result, respectively.

Further, values of model parameters of the preliminary detection model, the preliminary key-point extraction model, and the preliminary classification model may be updated by comparing the marked information in the marked training sample with the estimated initial detection result, the estimated intermediate detection result, and the estimated second intermediate detection result, respectively.

FIG. 13 is a schematic diagram illustrating exemplary marking of a training sample according to some embodiments of the present disclosure.

As described in connection with FIG. 12, for each of the plurality of training samples, at least one of whether there is a head region of a sample object in the training sample, a position of the head region of the sample object, whether the sample object wears a helmet or not, a color of the helmet, whether there is a hand region at least partially located within the head region of the sample object, a position of the hand region, or positions of a plurality of key points associated with the sample object may be marked in the training sample.

As illustrated in FIG. 13, when there is a head region of the sample object in the training sample, the technician may mark, in the training sample, a determination result (denoted as “determination result 1” ) of whether there is a head region of the sample object as 1; when there is no head region of the sample object in the training sample, the technician may mark the determination result as 0 in the training sample.

When the sample object wears a helmet in the training sample, the technician may mark, in the training sample, a determination result (denoted as “determination result 2” ) of whether the sample object wears a helmet or not as 1; when the sample object does not wear a helmet in the training sample, the technician may mark the determination result as 0 in the training sample.

For the color of the helmet, the technician may mark the color of the helmet in the training sample from a plurality of preset colors. The preset colors may be set by the technician according to an actual need. For example, the preset colors may include yellow, white, blue, and red. When the actual color of the helmet is within the plurality of preset colors, the technician may mark the color of the helmet as its actual color; when the actual color of the helmet is not within the plurality of preset colors, the technician may mark the color of the helmet as “other color. ”

When there is a hand region at least partially located within the head region of the sample object in the training sample, the technician may mark, in the training sample, a determination result (denoted as “determination result 3” ) of whether there is a hand region at least partially located within the head region of the sample object as 1; when there is no hand region at least partially located within the head region of the sample object in the training sample, the technician may mark the determination result as 0 in the training sample.

As illustrated in FIG. 13, the technician may mark the position of the head region of the sample object and the position of the hand region by boxing the head region of the sample object and the hand region in the application, respectively.

The technician may mark the positions of the plurality of key points by clicking on each key point in the application. In some embodiments, when the sample object wears a helmet in the training sample, the technician may designate a first point located at a top location of an edge of the helmet, a second point located at a far-left location of the edge of the helmet, a third point located at a far-right location of the edge of the helmet, a fourth point located at a location of the edge of the helmet between the first point and the second point, and a fifth point located at a location of the edge of the helmet between the first point and the third point as the positions of the plurality of key points; when the sample object does not wear a helmet in the training sample, the technician may designate an upper left corner of the head region as the positions of the plurality of key points.

Having thus described the basic concepts, it may be rather apparent to those skilled in the art after reading this detailed disclosure that the foregoing detailed disclosure is intended to be presented by way of example only and is not limiting. Various alterations, improvements, and modifications may occur and are intended to those skilled in the art, though not expressly stated herein. These alterations, improvements, and modifications are intended to be suggested by this disclosure, and are within the spirit and scope of the exemplary embodiments of this disclosure.

Moreover, certain terminology has been used to describe embodiments of the present disclosure. For example, the terms “one embodiment, ” “an embodiment, ” and/or “some embodiments” mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Therefore, it is emphasized and should be appreciated that two or more references to “an embodiment” or “one embodiment” or “an alternative embodiment” in various portions of this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined as suitable in one or more embodiments of the present disclosure.

Further, it will be appreciated by one skilled in the art, aspects of the present disclosure may be illustrated and described herein in any of a number of patentable classes or context including any new and useful process, machine, manufacture, or collocation of matter, or any new and useful improvement thereof. Accordingly, aspects of the present disclosure may be implemented entirely hardware, entirely software (including firmware, resident software, micro-code, etc. ) or combining software and hardware implementation that may all generally be referred to herein as a “unit, ” “module, ” or “system. ” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable media having computer-readable program code embodied thereon.

A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated signal may take any of a variety of forms, including electromagnetic, optical, or the like, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that may communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable signal medium may be transmitted using any appropriate medium, including wireless, wireline, optical fiber cable, RF, or the like, or any suitable combination of the foregoing.

Computer program code for carrying out operations for aspects of the present disclosure may be written in a combination of one or more programming languages, including an object oriented programming language such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB. NET, Python or the like, conventional procedural programming languages, such as the "C" programming language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages such as Python, Ruby, and Groovy, or other programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN) , or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider) or in a cloud computing environment or offered as a service such as a Software as a Service (SaaS) .

Furthermore, the recited order of processing elements or sequences, or the use of numbers, letters, or other designations thereof, are not intended to limit the claimed processes and methods to any order except as may be specified in the claims. Although the above disclosure discusses through various examples what is currently considered to be a variety of useful embodiments of the disclosure, it is to be understood that such detail is solely for that purpose, and that the appended claims are not limited to the disclosed embodiments, but, on the contrary, are intended to cover modifications and equivalent arrangements that are within the spirit and scope of the disclosed embodiments. For example, although the implementation of various components described above may be embodied in a hardware device, it may also be implemented as a software-only solution, e.g., an installation on an existing server or mobile device.

Similarly, it should be appreciated that in the foregoing description of embodiments of the present disclosure, various features are sometimes grouped together in a single embodiment, figure, or description thereof for the purpose of streamlining the disclosure aiding in the understanding of one or more of the various embodiments. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claimed subject matter requires more features than are expressly recited in each claim. Rather, claimed subject matter may lie in less than all features of a single foregoing disclosed embodiment.

Claims

A system, comprising:

at least one storage device including a set of instructions; and

at least one processor in communication with the at least one storage device, wherein when executing the set of instructions, the at least one processor is directed to perform operations including:

obtaining an image associated with a target object;

determining, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image;

determining, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result, the intermediate detection result being associated with a plurality of key points associated with the target object; and

determining a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.
The system of claim 1, wherein the initial detection result includes position information of a head region of the target object and a determination result of whether there is a helmet in the head region of the target object.
The system of claim 2, wherein the initial detection result further includes a determination result of whether there is a hand region at least partially located within the head region of the target object.
The system of any of claims 1-3, wherein the plurality of key points represent a shape feature of a target region associated with a head region of the target object.
The system of claim 4, wherein the plurality of key points include a first point located at a top location of an edge of the target region, a second point located at a far-left location of the edge of the target region, a third point located at a far-right location of the edge of the target region, a fourth point located between the first point and the second point, and a fifth point located between the first point and the third point.
The system of any of claims 1-5, wherein the determining, using the key-point extraction model, the intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result includes:

determining, using the key-point extraction model, the plurality of key points associated with the target object in an image portion indicated by the at least portion of the initial detection result; and

determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points, the intermediate detection result including a determination result of whether there is a helmet in a head region of the target object.
The system of claim 6, wherein the determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points includes:

determining whether one or more of the plurality of key points are located at a target position of the image portion indicated by the at least portion of the initial detection result; and

in response to determining that one or more of the plurality of key points are located at the target position, determining the determination result that there is no helmet in the head region of the target object.
The system of claim 7, wherein the determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points includes:

in response to determining that no key point is located at the target position,

dividing the plurality of key points into two groups of key points including a same key point;

for each of the two groups of key points, determining a connection line from the same key point to a target point; and

determining the determination result that there is a helmet in the head region of the target object in response to that connection lines corresponding to the two groups of key points satisfy a preset condition.
The system any of claims 1-8, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result includes:

determining, using a classification model, a second intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result; and

determining the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result, the intermediate detection result, and the second intermediate detection result.
The system of claim 9, wherein the second intermediate detection result includes at least one of a determination result of whether there is a helmet in a head region of the target object or a color of the helmet.
The system of claim 9, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result includes:

in response to that:

all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in a head region of the target object,

the initial detection result indicates there is no hand region at least partially located within the head region of the target object, and

the second intermediate detection result indicates the color of the helmet;

determining the target detection result that the target object is wearing a helmet and a color of the helmet.
The system of claim 9, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result includes:

in response to that:

all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in the head region of the target object,

the initial detection result indicates there is a hand region at least partially located within the head region of the target object, and

the second intermediate detection result indicates the color of the helmet;

determining a target confidence level based on confidence levels corresponding to the initial detection result, the intermediate detection result, and the second intermediate detection result;

in response to determining that the target confidence level is larger than or equal to a confidence level threshold, determining the target detection result that the target object is wearing a helmet and a color of the helmet.
The system of claim 12, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result further includes:

in response to determining that the target confidence level is less than the confidence level threshold, determining the target detection result that the target object is not wearing a helmet and a color of the helmet.
The system of claim 9, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result includes:

in response to that:

at least one of the initial detection result, the intermediate detection result, or the second intermediate detection result indicates that there is no helmet in a head region of the target object; or

the second intermediate detection result does not indicate the color of the helmet; determining the target detection result that the target object is not wearing a helmet.
A method implemented on a computing device including at least one processor, at least one storage medium, and a communication platform connected to a network, the method comprising:

obtaining an image associated with a target object;

determining, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image;

determining, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result, the intermediate detection result being associated with a plurality of key points associated with the target object; and

determining a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.
The method of claim 15, wherein the initial detection result includes position information of a head region of the target object and a determination result of whether there is a helmet in the head region of the target object.
The method of claim 16, wherein the initial detection result further includes a determination result of whether there is a hand region at least partially located within the head region of the target object.
The method of any of claims 15-17, wherein the plurality of key points represent a shape feature of a target region associated with a head region of the target object.
The method of claim 18, wherein the plurality of key points include a first point located at a top location of an edge of the target region, a second point located at a far-left location of the edge of the target region, a third point located at a far-right location of the edge of the target region, a fourth point located between the first point and the second point, and a fifth point located between the first point and the third point.
The method of any of claims 15-19, wherein the determining, using the key-point extraction model, the intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result includes:

determining, using the key-point extraction model, the plurality of key points associated with the target object in an image portion indicated by the at least portion of the initial detection result; and

determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points, the intermediate detection result including a determination result of whether there is a helmet in a head region of the target object.
The method of claim 20, wherein the determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points includes:

determining whether one or more of the plurality of key points are located at a target position of the image portion indicated by the at least portion of the initial detection result; and

in response to determining that one or more of the plurality of key points are located at the target position, determining the determination result that there is no helmet in the head region of the target object.
The method of claim 21, wherein the determining the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points includes:

in response to determining that no key point is located at the target position,

dividing the plurality of key points into two groups of key points including a same key point;

for each of the two groups of key points, determining a connection line from the same key point to a target point; and

determining the determination result that there is a helmet in the head region of the target object in response to that connection lines corresponding to the two groups of key points satisfy a preset condition.
The method any of claims 15-22, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result includes:

determining, using a classification model, a second intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result; and

determining the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result, the intermediate detection result, and the second intermediate detection result.
The method of claim 23, wherein the second intermediate detection result includes at least one of a determination result of whether there is a helmet in a head region of the target object or a color of the helmet.
The method of claim 23, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result includes:

in response to that:

all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in a head region of the target object,

the initial detection result indicates there is no hand region at least partially located within the head region of the target object, and

the second intermediate detection result indicates the color of the helmet;

determining the target detection result that the target object is wearing a helmet and a color of the helmet.
The method of claim 23, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result includes:

in response to that:

all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in the head region of the target object,

the initial detection result indicates there is a hand region at least partially located within the head region of the target object, and

the second intermediate detection result indicates the color of the helmet;

determining a target confidence level based on confidence levels corresponding to the initial detection result, the intermediate detection result, and the second intermediate detection result;

in response to determining that the target confidence level is larger than or equal to a confidence level threshold, determining the target detection result that the target object is wearing a helmet and a color of the helmet.
The method of claim 26, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result further includes:

in response to determining that the target confidence level is less than the confidence level threshold, determining the target detection result that the target object is not wearing a helmet and a color of the helmet.
The method of claim 23, wherein the determining the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result includes:

in response to that:

at least one of the initial detection result, the intermediate detection result, or the second intermediate detection result indicates that there is no helmet in a head region of the target object; or

the second intermediate detection result does not indicate the color of the helmet; determining the target detection result that the target object is not wearing a helmet.
A system, comprising

an obtaining module configured to obtain an image associated with a target object; and

a determination module configured to:

determine, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image;

determine, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result, the intermediate detection result being associated with a plurality of key points associated with the target object; and

determine a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.
The system of claim 29, wherein the initial detection result includes position information of a head region of the target object and a determination result of whether there is a helmet in the head region of the target object.
The system of claim 30, wherein the initial detection result further includes a determination result of whether there is a hand region at least partially located within the head region of the target object.
The system of any of claims 29-31, wherein the plurality of key points represent a shape feature of a target region associated with a head region of the target object.
The system of claim 32, wherein the plurality of key points include a first point located at a top location of an edge of the target region, a second point located at a far-left location of the edge of the target region, a third point located at a far-right location of the edge of the target region, a fourth point located between the first point and the second point, and a fifth point located between the first point and the third point.
The system of any of claims 29-33, wherein to determine, using the key-point extraction model, the intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result, the determination module is further configured to:

determine, using the key-point extraction model, the plurality of key points associated with the target object in an image portion indicated by the at least portion of the initial detection result; and

determine the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points, the intermediate detection result including a determination result of whether there is a helmet in a head region of the target object.
The system of claim 34, wherein to determine the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points, the determination module is further configured to:

determine whether one or more of the plurality of key points are located at a target position of the image portion indicated by the at least portion of the initial detection result; and

in response to determining that one or more of the plurality of key points are located at the target position, determine the determination result that there is no helmet in the head region of the target object.
The system of claim 35, wherein to determine the intermediate detection result associated with whether the target object is wearing a helmet or not based on the plurality of key points, the determination module is further configured to:

in response to determining that no key point is located at the target position,

divide the plurality of key points into two groups of key points including a same key point;

for each of the two groups of key points, determine a connection line from the same key point to a target point; and

determine the determination result that there is a helmet in the head region of the target object in response to that connection lines corresponding to the two groups of key points satisfy a preset condition.
The system any of claims 29-36, wherein to determine the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result, the determination module is further configured to:

determine, using a classification model, a second intermediate detection result associated with whether the target object is wearing a helmet or not based on the at least portion of the initial detection result; and

determine the target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result, the intermediate detection result, and the second intermediate detection result.
The system of claim 37, wherein the second intermediate detection result includes at least one of a determination result of whether there is a helmet in a head region of the target object or a color of the helmet.
The system of claim 37, wherein to determine the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result, the determination module is further configured to:

in response to that:

all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in a head region of the target object,

the initial detection result indicates there is no hand region at least partially located within the head region of the target object, and

the second intermediate detection result indicates the color of the helmet;

determine the target detection result that the target object is wearing a helmet and a color of the helmet.
The system of claim 37, wherein to determine the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result, the determination module is further configured to:

in response to that:

all the initial detection result, the intermediate detection result, and the second intermediate detection result indicate that there is a helmet in the head region of the target object,

the initial detection result indicates there is a hand region at least partially located within the head region of the target object, and

the second intermediate detection result indicates the color of the helmet;

determine a target confidence level based on confidence levels corresponding to the initial detection result, the intermediate detection result, and the second intermediate detection result;

in response to determining that the target confidence level is larger than or equal to a confidence level threshold, determine the target detection result that the target object is wearing a helmet and a color of the helmet.
The system of claim 40, wherein to determine the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result further, the determination module is further configured to:

in response to determining that the target confidence level is less than the confidence level threshold, determine the target detection result that the target object is not wearing a helmet and a color of the helmet.
The system of claim 37, wherein to determine the target detection result associated with whether the target object is wearing a helmet or not based on the initial detection result, the intermediate detection result, and the second intermediate detection result, the determination module is further configured to:

in response to that:

at least one of the initial detection result, the intermediate detection result, or the second intermediate detection result indicates that there is no helmet in a head region of the target object; or

the second intermediate detection result does not indicate the color of the helmet; determine the target detection result that the target object is not wearing a helmet.
A non-transitory computer readable medium, comprising executable instructions that, when executed by at least one processor, direct the at least one processor to perform a method, the method comprising:

obtaining an image associated with a target object;

determining, using a detection model, an initial detection result associated with whether the target object is wearing a helmet or not based on the image;

determining, using a key-point extraction model, an intermediate detection result associated with whether the target object is wearing a helmet or not based on at least portion of the initial detection result, the intermediate detection result being associated with a plurality of key points associated with the target object; and

determining a target detection result associated with whether the target object is wearing a helmet or not based at least in part on the initial detection result and the intermediate detection result.