CN115546516B

CN115546516B - Personnel gear gathering method, device, computer equipment and storage medium

Info

Publication number: CN115546516B
Application number: CN202211200033.1A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Beijing Real AI Technology Co Ltd
Current assignee: Beijing Real AI Technology Co Ltd
Priority date: 2022-09-29
Filing date: 2022-09-29
Publication date: 2024-08-06
Anticipated expiration: 2042-09-29
Also published as: CN115546516A

Abstract

The embodiment of the application discloses a personnel gear gathering method, a device, computer equipment and a storage medium. The method comprises the following steps: acquiring a first personnel profile and a second personnel profile, wherein the first personnel profile comprises a first keyframe set, the first keyframe set comprises first keyframes of at least one image feature type, and the second personnel profile comprises a second keyframe set, and the second keyframe set comprises second keyframes of at least one image feature type; performing file similarity calculation on the first key frame set and the second key frame set according to the image feature type of the first key frame and the image feature type of the second key frame to obtain a target similarity value; and if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file. By implementing the method provided by the embodiment of the application, the personnel gear gathering effect can be improved.

Description

Personnel gear gathering method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method and apparatus for personnel gear gathering, a computer device, and a storage medium.

Background

Along with the continuous development of security monitoring, more and more face cameras and security cameras are applied to daily life, so that the track of people in snapshot is tracked across time and region to become an important subject in the security field, and therefore, a personnel filing technology appears, and personnel filing refers to filing of a large number of face photos according to individuals.

The key point of personnel file gathering is how to judge whether two files belong to the same person, if so, the two files are combined into the same file.

In the prior art, only a face is used for capturing photos when files are gathered, only one face is used for capturing photos when files are gathered to serve as key frames of files, a large number of human photos generated by a public security camera are not utilized, so that images of personnel files are not abundant, partial characteristic information of archives personnel can be lost when only one face is used for capturing photos to serve as key frames of files, accuracy rate and recall rate of file gathering results are reduced, a situation of multiple files of one person can occur, and file gathering effect is still to be improved.

Disclosure of Invention

The embodiment of the application provides a personnel file gathering method, a device, computer equipment and a storage medium, which can improve the personnel file gathering effect.

In a first aspect, an embodiment of the present application provides a method for personnel gear gathering, including:

acquiring a first personnel profile and a second personnel profile, wherein the first personnel profile comprises a first keyframe set, the first keyframe set comprises first keyframes of at least one image feature type, and the second personnel profile comprises a second keyframe set, and the second keyframe set comprises second keyframes of at least one image feature type;

performing file similarity calculation on the first key frame set and the second key frame set according to the image feature type of the first key frame and the image feature type of the second key frame to obtain a target similarity value;

And if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file.

In a second aspect, an embodiment of the present application further provides a personal gear gathering device, including:

The system comprises a transceiver module, a first personal archive and a second personal archive, wherein the first personal archive comprises a first keyframe set, the first keyframe set comprises first keyframes of at least one image feature type, the second personal archive comprises a second keyframe set, and the second keyframe set comprises second keyframes of at least one image feature type;

The processing module is used for carrying out file similarity calculation on the first key frame set and the second key frame set according to the image feature type of the first key frame and the image feature type of the second key frame to obtain a target similarity value; and if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file.

In some embodiments, the processing module is specifically configured to, when executing the step of obtaining the target similarity value by performing file similarity calculation on the first keyframe set and the second keyframe set according to the image feature type of the first keyframe and the image feature type of the second keyframe:

Determining a first target key frame and a second target key frame according to a corresponding relation between preset image feature types and priorities, wherein the first target key frame is a key frame with the highest image feature type priority in the first key frame which is not subjected to similarity calculation in the first key frame set, the second target key frame is a key frame with the highest image feature type priority in the second key frame which is not subjected to similarity calculation in the second key frame set, and the image feature types of the first target key frame and the second target key frame are the same;

performing file similarity calculation on the first target key frame and the second target key frame to obtain candidate similarity values;

And determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold value.

In some embodiments, the processing module is specifically configured to, when executing the step of determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold value:

if the comparison result shows that the candidate similarity value is greater than or equal to the preset threshold value, determining the candidate similarity value as the target similarity value;

If the comparison result shows that the candidate similarity value is smaller than the preset threshold value, determining whether other key frames which are not subjected to similarity calculation exist in the first key frame set and the second key frame set;

if other key frames which are not subjected to similarity calculation exist, returning to execute the step of determining the first target key frame and the second target key frame according to the corresponding relation between the preset image feature type and the priority;

And if no other key frames which are not subjected to similarity calculation exist, determining the candidate similarity as the target similarity value.

In some embodiments, after executing the step of aggregating the first personnel profile and the second personnel profile to obtain a third personnel profile, the processing module is further configured to:

determining candidate key frames of each image feature type in the third personnel file according to the first key frame of each image feature type and the second key frame of each image feature type;

For the candidate key frames of each image feature type, if the number of the candidate key frames is smaller than or equal to a preset key frame number threshold, determining the candidate key frames as third key frames of the corresponding image feature types;

And if the number of the candidate key frames is larger than the key frame number threshold, selecting a target number of candidate key frames from the candidate key frames as the third key frames of the corresponding image feature types, wherein the target number corresponds to the key frame number threshold.

In some embodiments, the processing module is specifically configured to, when executing the step of selecting a target number of candidate key frames from the candidate key frames as the third key frames of the corresponding image feature types:

for each candidate key frame, determining the sum of similarity between the candidate key frame and a plurality of target personnel images, wherein the target personnel images are personnel images with the same characteristic type as the candidate key frame images in the third personnel file;

and selecting the target number of candidate key frames with the largest sum of similarity from the candidate key frames as the third key frames of the corresponding image feature types according to the sum of the similarity of the candidate key frames.

And generating a personnel track corresponding to the third personnel file according to the space-time information of each personnel image in the third personnel file.

In some embodiments, the at least one image feature type includes at least one of a full face image type, a mask-worn face image type, and a human image type.

In a third aspect, an embodiment of the present application further provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the method when executing the computer program.

In a fourth aspect, embodiments of the present application also provide a computer readable storage medium storing a computer program comprising program instructions which, when executed by a processor, implement the above-described method.

Compared with the prior art, in the embodiment of the application, since the first personnel file and the second personnel file can comprise the key frames with various image feature types, the key frames with different image types can represent the features of different states or different body parts of the personnel, and when the personnel are in the file gathering, the key frames with various image feature types are combined for file gathering, so that the loss of the features of the personnel can be reduced, the file gathering is more comprehensive, the accuracy and recall rate of file gathering results are further improved, the occurrence of the conditions of multiple files of one person is reduced, and the personnel file gathering effect is improved.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an application scenario of a personnel file gathering method provided by an embodiment of the present application;

FIG. 2 is a schematic flow chart of a personnel accumulation method according to an embodiment of the present application;

FIG. 3 is a schematic sub-flowchart of a personnel accumulation method according to an embodiment of the present application;

FIG. 4 is a flowchart of a method for personnel accumulation according to another embodiment of the present application;

FIG. 5 is a schematic block diagram of a personnel accumulation assembly provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a server according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of a terminal according to an embodiment of the present application;

fig. 8 is a schematic structural diagram of a server according to an embodiment of the present application.

Detailed Description

The terms first, second and the like in the description and in the claims of embodiments of the application and in the above-described figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments described herein may be implemented in other sequences than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those explicitly listed but may include other steps or modules not expressly listed or inherent to such process, method, article, or apparatus, such that the partitioning of modules by embodiments of the application is only one logical partitioning, may be implemented with additional partitioning, such as a plurality of modules may be combined or integrated in another system, or some features may be omitted, or not implemented, and further, such that the coupling or direct coupling or communication connection between modules may be via some interfaces, indirect coupling or communication connection between modules may be electrical or otherwise similar, none of which are limited in embodiments of the application. The modules or sub-modules described as separate components may or may not be physically separate, may or may not be physical modules, or may be distributed in a plurality of circuit modules, and some or all of the modules may be selected according to actual needs to achieve the purposes of the embodiment of the present application.

The embodiment of the application provides a personnel file gathering method, a device, computer equipment and a storage medium, wherein an execution main body of the personnel file gathering method can be the personnel file gathering device provided by the embodiment of the application or the computer equipment integrated with the personnel file gathering device, wherein the personnel file gathering device can be realized in a hardware or software mode, and the computer equipment can be a terminal or a server.

When the computer device is a server, the server may be an independent physical server, or may be a server cluster or a distributed system formed by a plurality of physical servers, or may be a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, basic cloud computing services such as big data and artificial intelligence platforms, and the like.

When the computer device is a terminal, the terminal may include: smart phones, tablet computers, notebook computers, desktop computers, smart televisions, smart speakers, personal digital assistants (english: personal DIGITAL ASSISTANT, abbreviated to PDA), desktop computers, smart watches, and the like, which carry multimedia data processing functions (e.g., video data playing functions, music data playing functions), but are not limited thereto.

The scheme of the embodiment of the application can be realized based on an artificial intelligence technology, and particularly relates to the technical field of computer vision in the artificial intelligence technology and the fields of cloud computing, cloud storage, databases and the like in the cloud technology, and the technical fields are respectively described below.

Artificial intelligence (ARTIFICIAL INTELLIGENCE, AI) is the theory, method, technique, and application system that simulates, extends, and extends human intelligence using a digital computer or a machine controlled by a digital computer, perceives the environment, obtains knowledge, and uses the knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.

The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.

Computer Vision (CV) is a science of studying how to "look" a machine, and more specifically, to replace human eyes with a camera and a Computer to perform machine Vision such as recognition, tracking and measurement on a target, and further perform graphic processing to make the Computer process into an image more suitable for human eyes to observe or transmit to an instrument to detect. As a scientific discipline, computer vision research-related theory and technology has attempted to build artificial intelligence systems that can acquire information from images or multidimensional data. Computer vision techniques typically include image processing, people profiling, image semantic understanding, image retrieval, OCR, video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D techniques, virtual reality, augmented reality, synchronous positioning, map construction, and other techniques, as well as common people profiling, fingerprint recognition, and other biometric recognition techniques.

With research and advancement of artificial intelligence technology, research and application of artificial intelligence technology is being developed in various fields, such as common smart home, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned, automatic driving, unmanned aerial vehicles, robots, smart medical treatment, smart customer service, etc., and it is believed that with the development of technology, artificial intelligence technology will be applied in more fields and with increasing importance value.

The scheme of the embodiment of the application can be realized based on cloud technology, and particularly relates to the technical fields of cloud computing, cloud storage, databases and the like in the cloud technology, and the technical fields are respectively described below.

Cloud technology (Cloud technology) refers to a hosting technology for integrating hardware, software, network and other series resources in a wide area network or a local area network to realize calculation, storage, processing and sharing of data. Cloud technology (Cloud technology) is based on the general terms of network technology, information technology, integration technology, management platform technology, application technology and the like applied by Cloud computing business models, and can form a resource pool, so that the Cloud computing business model is flexible and convenient as required. Cloud computing technology will become an important support. Background services of technical networking systems require a significant amount of computing, storage resources, such as video websites, image-like websites, and more portals. Along with the high development and application of the internet industry, each article possibly has an own identification mark in the future, the identification mark needs to be transmitted to a background system for logic processing, data with different levels can be processed separately, and various industry data needs strong system rear shield support and can be realized only through cloud computing. According to the embodiment of the application, the identification result can be stored through the cloud technology.

Cloud storage (cloud storage) is a new concept that extends and develops in the concept of cloud computing, and a distributed cloud storage system (hereinafter referred to as a storage system for short) refers to a storage system that integrates a large number of storage devices (storage devices are also referred to as storage nodes) of various types in a network to work cooperatively through application software or application interfaces through functions such as cluster application, grid technology, and a distributed storage file system, so as to provide data storage and service access functions for the outside. In the embodiment of the application, the information such as network configuration and the like can be stored in the storage system, so that the server can conveniently call the information.

At present, the storage method of the storage system is as follows: when creating logical volumes, each logical volume is allocated a physical storage space, which may be a disk composition of a certain storage device or of several storage devices. The client stores data on a certain logical volume, that is, the data is stored on a file system, the file system divides the data into a plurality of parts, each part is an object, the object not only contains the data but also contains additional information such as a data Identification (ID) and the like, the file system writes each object into a physical storage space of the logical volume, and the file system records storage position information of each object, so that when the client requests to access the data, the file system can enable the client to access the data according to the storage position information of each object.

The process of allocating physical storage space for the logical volume by the storage system specifically includes: physical storage space is divided into stripes in advance according to the set of capacity measures for objects stored on a logical volume (which measures tend to have a large margin with respect to the capacity of the object actually to be stored) and redundant array of independent disks (RAID, redundant Array ofIndependent Disk), and a logical volume can be understood as a stripe, whereby physical storage space is allocated for the logical volume.

The Database (Database), which can be considered as an electronic filing cabinet, is a place for storing electronic files, and users can perform operations such as adding, inquiring, updating, deleting and the like on the data in the files. A "database" is a collection of data stored together in a manner that can be shared with multiple users, with as little redundancy as possible, independent of the application.

The Database management system (DBMS for short, english: database MANAGEMENT SYSTEM) is a computer software system designed for managing databases, and generally has basic functions of storage, interception, security assurance, backup and the like. The database management system may classify according to the database model it supports, e.g., relational, XML (Extensible Markup Language ); or by the type of computer supported, e.g., server cluster, mobile phone; or by the query language used, e.g., SQL (structured query language ), XQuery; or by performance impact emphasis, such as maximum scale, maximum speed of operation; or other classification schemes. Regardless of the manner of classification used, some DBMSs are able to support multiple query languages across categories, for example, simultaneously. In the embodiment of the application, the identification result can be stored in the database management system, so that the server can conveniently call.

It should be specifically noted that, the terminal according to the embodiment of the present application may be a device that provides voice and/or data connectivity to a service terminal, a handheld device with a wireless connection function, or other processing device connected to a wireless modem. Such as mobile telephones (or "cellular" telephones) and computers with mobile terminals, which can be portable, pocket, hand-held, computer-built-in or car-mounted mobile devices, for example, which exchange voice and/or data with radio access networks. For example, personal communication services (english: personal Communication Service, english: PCS) telephones, cordless telephones, session Initiation Protocol (SIP) phones, wireless local loop (Wireless Local Loop, english: WLL) stations, personal digital assistants (english: personal DIGITAL ASSISTANT, english: PDA) and the like.

In some embodiments, the embodiment of the present application may be applied to a personnel filing system 1 as shown in fig. 1, where the personnel filing system 1 includes a server 10 and at least one image capturing device 20, the image capturing device 20 sends a captured first personnel file to the server 10, where the personnel file includes one image or includes multiple post-filing images, and then the server 10 performs personnel filing processing on the first personnel file and a second personnel file in a memory, where a specific filing method includes: the server 10 obtains a first person profile comprising a first set of key frames comprising first key frames of at least one image feature type and a second person profile comprising a second set of key frames comprising second key frames of at least one image feature type; performing file similarity calculation on the first key frame set and the second key frame set according to the image feature type of the first key frame and the image feature type of the second key frame to obtain a target similarity value; and if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file.

In some embodiments, the first personnel file and the second personnel file are the personnel files sent to the server 10 by the one or more image capturing devices 20; in other embodiments, the first personnel file may be a personnel file sent by the image capturing device 20 to the server 10, and the second personnel file is a file stored in the server 10 and required to be aggregated with the first personnel file; in still other embodiments, the first personnel file and the second personnel file are both internally stored in the server 10, and two personnel files currently need to be aggregated; in this embodiment, the first personnel file and the second personnel file include one image or multiple files, and in this embodiment, the first personnel file and the second personnel file are two files that currently need to be processed for file aggregation, and specific acquisition paths of the first personnel file and the second personnel file are not limited herein.

The technical scheme of the application will be described in detail below with reference to several embodiments.

Referring to fig. 2, a personnel gear gathering method provided by an embodiment of the present application is described below, where the embodiment of the present application includes:

201. A first person profile is obtained, the first person profile comprising a first set of keyframes comprising first keyframes of at least one image feature type, and a second person profile comprising a second set of keyframes comprising second keyframes of at least one image feature type.

It should be noted that, in the present embodiment, the key frames (including the first key frame and the second key frame) are the personnel images which are selected from the corresponding personnel files and have good quality and can represent the files, and when the two personnel files need to be processed for file aggregation, only the key frames of the two personnel files are needed to determine whether the two files can be processed for file aggregation, so as to reduce the calculation amount of file aggregation.

The image acquisition device in this embodiment includes a face camera, a public security camera and other various types of cameras, where the face camera is used for capturing face images of a person, the public security camera can capture the body images of the person, the face camera and the public security camera can be disposed at the same position or at different positions, and the invention is not limited in this regard.

In some embodiments, in order to improve the quality of the image for image capture, the server in this embodiment filters the acquired image before capturing the image, filters the image that fails (for example, the pixel is relatively low) in advance, or the image capturing device first determines the quality of the acquired image before sending the image to the server, and discards the image if the quality fails, without sending the image to the server.

In this embodiment, the at least one image feature type includes at least one of a full face image type, a mask-wearing face image type, and a human body image type. At this time, the first person file and the second person file include images of at least one image feature type of the whole face image type, the mask-wearing face image type and the human body image type.

If the acquired image contains a face image and a human body image of a person, if a face (a complete face image or a mask-wearing face image) in the image is clear, dividing the image feature type of the image into a complete face image type or a mask-wearing face image type according to whether the face is worn by a mask or not; if the face image in the image is not clear and the human body image is clear, at the moment, the image characteristic type of the image is determined as the human body image type.

202. And performing file similarity calculation on the first key frame set and the second key frame set according to the image feature type of the first key frame and the image feature type of the second key frame to obtain a target similarity value.

In some embodiments, when only one target person image is included in the first person profile, the step of calculating the target similarity is as follows:

At this time, the target personnel image is a first key frame corresponding to the first personnel file, the target image feature type of the first key frame is firstly determined, then a second key frame corresponding to the target image feature type is extracted from the second key frame set, file similarity calculation is performed on the first key frame and the second key frame, specifically, if a plurality of second key frames corresponding to the target image feature type exist, similarity calculation is required to be performed on the plurality of second key frames respectively, a similarity average value is calculated, and the similarity average value is determined to be the target similarity value.

It should be noted that, at the initial stage of the second personnel file gathering, the second personnel file may lack a second key frame corresponding to the feature type of the target image, at this time, when judging whether the target personnel image belongs to an image in the image of the second key frame, whether the second personnel file has an associated feature corresponding to the target personnel image needs to be judged according to the existing feature association technology, if so, whether the target personnel image belongs to the second personnel file is judged according to the associated feature, if so, the target personnel image is added into the second personnel file, and the target personnel image is determined as a key frame of the feature type of the target image of the second personnel file.

For example, if the target image feature type of the target person image is a human body image type and the second person file does not collect an image of the human body image type temporarily, at this time, further detecting whether a complete image (for example, a complete face image including a human body image or a mask-wearing face image) similar to the spatial and temporal information of the target person image exists in the second person file, then performing similarity calculation on the target person image and the human body image in the complete image to obtain a target similarity value, and finally determining whether the target person image belongs to the second person file according to the target similarity value.

In some embodiments, in order to avoid that the image feature type corresponding to the image feature type of the first keyframe cannot be found in the second personnel file, in this embodiment, the user may set at least one second keyframe for each image feature type in the second personnel file in advance, where the second keyframe set includes second keyframes of multiple image feature types.

In some embodiments, if the target image feature type of the target person image is a complete face image type, at this time, the target person image is preferentially compared with a key frame of the complete face image type in the second person file, if the comparison fails, the target person image is further compared with a key frame of a mask-wearing face image type in the second person file, if the comparison fails, it is indicated that the target person image does not belong to the second person file, otherwise, the target person image belongs to the second person file. Similarly, if the target image feature type of the target person image is the mask-wearing face image type, at this time, the target person image is preferentially compared with the key frames of the mask-wearing face image type in the second person file, if the comparison fails, the target person image is further compared with the key frames of the complete face image type in the second person file, if the comparison fails, the target person image is not the second person file, and if the target image feature type of the target person image is the human body image type, at this time, the target person image can only be compared with the key frames of the human body image type in the second person file.

In some embodiments, when the first person profile includes a plurality of images and the first person profile includes a plurality of image feature types, the step of calculating the target similarity is as follows:

Specifically, in some embodiments, the server sets different priorities for different image feature types, and in this embodiment, the file similarity calculation is preferentially performed on the key frames of the image feature types with high priorities, and at this time, referring to fig. 3, the specific steps of performing the file similarity calculation on the first key frame set and the second key frame set are as follows:

2021. And determining a first target key frame and a second target key frame according to the corresponding relation between the preset image feature type and the priority.

Specifically, in this embodiment, the priority of the full face image type is higher than the priority of the Dai Kouzhao face image type, and the priority of the face image type of the mask is higher than the priority of the human image type.

The first target key frame is a key frame with the highest image feature type priority in the first key frame which is not subjected to similarity calculation in the first key frame set, the second target key frame is a key frame with the highest image feature type priority in the second key frame which is not subjected to similarity calculation in the second key frame set, and the image feature types of the first target key frame and the second target key frame are the same.

2022. And performing file similarity calculation on the first target key frame and the second target key frame to obtain candidate similarity values.

For example, a first face key frame of a complete face image type is obtained in a first key frame set, a second face key frame of the complete face image type is obtained in a second key frame set, and then file similarity calculation is performed on the first face key frame and the second face key frame to obtain candidate similarity values.

2023. And determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold value.

Specifically, if the comparison result is that the candidate similarity value is greater than or equal to the preset threshold value, determining the candidate similarity value as the target similarity value; if the comparison result shows that the candidate similarity value is smaller than the preset threshold value, determining whether other key frames which are not subjected to similarity calculation exist in the first key frame set and the second key frame set; if there are other key frames not subjected to the similarity calculation, returning to step 2021; and if no other key frames which are not subjected to similarity calculation exist, determining the candidate similarity as the target similarity value.

For example, in this embodiment, first, a first face key frame of a complete face image type is obtained in a first key frame set, and a second face key frame of a complete face image type is obtained in a second key frame set, then file similarity calculation is performed on the first face key frame and the second face key frame to obtain candidate similarity values, then whether the candidate similarity values are greater than or equal to the preset threshold value is judged, if yes, the candidate similarity values are determined to be target similarity values, if no, the image feature types which are not calculated are further obtained according to a preset priority order, in this embodiment, file similarity calculation is performed on the first face mask key frame and the second mask key frame of the mask face image type, which are obtained from the first key frame set and the second key frame set, then whether the candidate similarity values are greater than or equal to the preset threshold value is judged, if yes, the candidate similarity values are determined to be target similarity values, if no, the candidate similarity values are determined to be target similarity values, then the candidate similarity values are obtained from the first face mask key frame and the first mask key frame of the first mask type, and the second mask type are obtained, and if no candidate similarity values are obtained from the first face image type, the first face image type is obtained from the first mask key frame set, and the first mask type is directly, and the first mask type is calculated.

203. And if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file.

In this embodiment, if the target similarity value is greater than or equal to the preset threshold, it is indicated that the first personnel file and the second personnel file belong to the same personnel file, and at this time, the first personnel file and the second personnel file are subjected to file gathering processing to obtain a third personnel file.

When the target similarity value is smaller than the preset threshold value, the first personnel file and the second personnel file are not the same person file, and aggregation processing of the first personnel file and the second personnel file is not needed.

If the first personnel file is a single personnel image obtained from the image collecting device (for convenience of reading, when the first personnel file is a single personnel image, the first personnel file is hereinafter referred to as a target personnel image), for example, a single Zhang Zhua of the image collecting device photographs, and the second personnel file is a file of a certain person maintained in the server, the embodiment needs to determine whether the target personnel image belongs to the second personnel file, if the target personnel image does not belong to the second personnel file, the target personnel image is not added to the second personnel file, and then the other personnel files in the server are continuously polled until the files which can be added are found, and if all the personnel files in the server are polled, a new personnel file is created in the server according to the target personnel image.

Similarly, if the first person profile includes a plurality of aggregated images, if no enrollable profile is found in the server, then a new person profile is created based on the first person profile.

Further, in this embodiment, top K key frames are dynamically maintained for each type of key frame of the image feature type, where the value of Top K may be 2, or may be another value, for example, 5, and the specific value is not limited herein. At this time, in some embodiments, after the first personnel file and the second personnel file are subjected to the file gathering process to obtain a third personnel file, the method further includes:

a. And determining candidate key frames of each image feature type in the third personnel file according to the first key frame of each image feature type and the second key frame of each image feature type.

For example, in the first personnel file, there are 2 first key frames of the complete face image type, 2 first key frames of the mask-wearing face image type, and 2 first key frames of the human body image type; in the second personnel file, 2 second key frames of the complete face image type, 2 second key frames of the mask-wearing face image type and 2 second key frames of the human body image type are arranged; at this time, in the third personnel file, there are 4 candidate key frames of the whole face image type, 4 candidate key frames of the mask-wearing face image type, and 4 candidate key frames of the human body image type.

B. and determining the candidate key frames of the image feature types as third key frames of the corresponding image feature types if the number of the candidate key frames is smaller than or equal to a preset key frame number threshold value.

For example, for the candidate key frames of the complete face image type, if the number of candidate key frames of the complete face image type is less than or equal to the threshold number of key frames, the candidate key frames do not need to be screened, and the obtained candidate key frames are directly determined to be the third key frames of the corresponding image feature type.

The threshold of the number of key frames in this embodiment is the maximum value of the key frames of each type to be maintained, and at this time, the threshold of the number of key frames is Top K. If the value of Top K is 2, at this time, if the number of the obtained candidate key frames is less than or equal to 2, at this time, the obtained candidate key frames are directly determined as the third key frames corresponding to the image feature types.

C. And if the number of the candidate key frames is larger than the key frame number threshold, selecting a target number of candidate key frames from the candidate key frames as the third key frames of the corresponding image feature types, wherein the target number corresponds to the key frame number threshold.

For example, for candidate key frames of the full face image type, if the number of candidate key frames of the full face image type is greater than a key frame number threshold, further filtering of the candidate key frames is required.

In some embodiments, the specific screening method is as follows:

for each candidate key frame, determining the sum of similarity between the candidate key frame and a plurality of target personnel images, wherein the target personnel images are personnel images with the same characteristic type as the candidate key frame images in the third personnel file; and then selecting the target number of candidate key frames with the largest sum of similarity from the candidate key frames according to the sum of the similarity of the candidate key frames, and taking the target number of candidate key frames with the largest sum of similarity as the third key frame of the corresponding image feature type.

For example, for candidate key frames of the whole face image type, if the number of candidate key frames is 4 and the threshold of the number of key frames is 2, at this time, 2 third key frames serving as third person files need to be selected from the 4 candidate key frames, specifically, the steps are to respectively determine the sum of the similarity between each candidate key frame and the person images of the whole face image type in the third person files, obtain the sum of the similarity corresponding to each candidate key frame, and then select the 2 candidate key frames with the largest sum of the similarity as the third key frames. And further, the dynamic maintenance of the number of key frames is realized.

The screening method for the face image type of the wearer mask and the candidate key frames for the human body image type is similar to the screening method for the candidate key frames for the whole face image type, and detailed description is omitted here.

In some embodiments, after the first personnel file and the second personnel file are subjected to a file gathering process to obtain a third personnel file, the method further includes: and generating a personnel track corresponding to the third personnel file according to the space-time information of each personnel image in the third personnel file.

Therefore, the personnel track can be generated according to the data after the file gathering, and the recall rate of the image is high because the embodiment comprises various image feature types in the third personnel file, so that the accuracy of the generated personnel track is also high.

In some embodiments, in order to further understand the personnel document gathering method provided in this embodiment, please refer to fig. 4, the image acquisition device sends a frame of complete face snapshot to the server, at this time, the server performs similarity calculation with a second personnel file in the server according to the frame of complete face snapshot, at this time, K in Top K is taken as 2, where the darkened partial image in fig. 4 is a key frame image, first, it is determined that the image feature type of the frame of complete face snapshot is a complete face image type, then, file similarity calculation is performed on the key frame of the frame of complete face snapshot and the complete face image type in the second file, so as to obtain a target similarity result, if the target similarity result is greater than or equal to a preset threshold, the frame of complete face snapshot is put into the second file, so as to obtain a third file, and meanwhile, key frames of Top K are dynamically maintained.

In summary, in the embodiment of the present application, since the first personnel file and the second personnel file may include key frames with multiple image feature types, the key frames with different image types may represent the features of different states or different body parts of the personnel, and when the files are gathered, the key frames with multiple image feature types are combined to perform file gathering, so that the loss of the features of the personnel of the files can be reduced, the file gathering is more comprehensive, further the accuracy and recall rate of the file gathering result are improved, the occurrence of multiple files of one person is reduced, and the file gathering effect of the personnel is improved.

Fig. 5 is a schematic block diagram of a personal gear gathering device provided by an embodiment of the present application. As shown in FIG. 5, the application further provides a personnel gear gathering device corresponding to the personnel gear gathering method. The personal profile-gathering device comprises a unit for executing the personal profile-gathering method, and the device can be configured in a terminal or a server. Specifically, referring to fig. 5, the personal focusing device 500 includes a transceiver module 501 and a processing module 502, wherein:

A transceiver module 501, configured to obtain a first personnel profile and a second personnel profile, where the first personnel profile includes a first keyframe set, the first keyframe set includes a first keyframe of at least one image feature type, and the second personnel profile includes a second keyframe set, and the second keyframe set includes a second keyframe of at least one image feature type;

the processing module 502 is configured to perform file similarity calculation on the first keyframe set and the second keyframe set according to the image feature type of the first keyframe and the image feature type of the second keyframe, so as to obtain a target similarity value; and if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file.

In some embodiments, when the processing module 502 performs the step of performing file similarity calculation on the first keyframe set and the second keyframe set according to the image feature type of the first keyframe and the image feature type of the second keyframe to obtain a target similarity value, the processing module is specifically configured to:

In some embodiments, when the step of determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold is performed, the processing module 502 is specifically configured to:

In some embodiments, after executing the step of aggregating the first personnel profile and the second personnel profile to obtain a third personnel profile, the processing module 502 is further configured to:

In some embodiments, the processing module 502 is specifically configured to, when executing the step of selecting a target number of candidate keyframes from the candidate keyframes as the third keyframes of the corresponding image feature types:

It should be noted that, it is clearly understood by those skilled in the art that the specific implementation process of the personnel gear gathering device and each unit may refer to the corresponding description in the foregoing method embodiment, and for convenience and brevity of description, the description is omitted herein.

The personnel file collecting device in the embodiment of the application is described above from the point of view of the modularized functional entity, and the personnel file collecting device in the embodiment of the application is described below from the point of view of hardware processing.

It should be noted that, in each embodiment of the present application (including each embodiment shown in fig. 5), the entity devices corresponding to all the transceiver modules may be transceivers, and the entity devices corresponding to all the processing modules may be processors. When one of the devices has the structure shown in fig. 5, the processor, the transceiver and the memory implement the same or similar functions as the transceiver module and the processing module provided in the device embodiment of the device, and the memory in fig. 6 stores a computer program to be invoked when the processor executes the personnel gear gathering method.

The apparatus shown in fig. 5 may have a structure as shown in fig. 6, and when the apparatus shown in fig. 5 has a structure as shown in fig. 6, the processor in fig. 6 can implement the same or similar functions as the processing module provided by the apparatus embodiment corresponding to the apparatus, and the transceiver in fig. 6 can implement the same or similar functions as the transceiver module provided by the apparatus embodiment corresponding to the apparatus, and the memory in fig. 6 stores a computer program that needs to be invoked when the processor executes the personnel accumulation method. In the embodiment of the present application shown in fig. 5, the entity device corresponding to the transceiver module may be an input/output interface, and the entity device corresponding to the processing module may be a processor.

The embodiment of the present application further provides another terminal device, as shown in fig. 7, for convenience of explanation, only the portion relevant to the embodiment of the present application is shown, and specific technical details are not disclosed, please refer to the method portion of the embodiment of the present application. The terminal device may be any terminal device including a mobile phone, a tablet personal computer, a personal digital assistant (english: personal DigitalAssistant, english: PDA), a sales terminal (english: point ofSales, english: POS), a vehicle-mounted computer, and the like, taking the mobile phone as an example of the terminal device:

Fig. 7 is a block diagram showing a part of the structure of a mobile phone related to a terminal device provided by an embodiment of the present application. Referring to fig. 7, the mobile phone includes: radio Frequency (RF) circuit 610, memory 620, input unit 630, display unit 640, sensor 650, audio circuit 660, wireless fidelity (WIRELESS FIDELITY, wi-Fi) module 670, processor 680, and power supply 690. It will be appreciated by those skilled in the art that the handset construction shown in fig. 7 is not limiting of the handset and may include more or fewer components than shown, or may combine certain components, or a different arrangement of components.

The following describes the components of the mobile phone in detail with reference to fig. 7:

the RF circuit 610 may be configured to receive and transmit signals during a message or a call, and in particular, receive downlink information of a base station and process the downlink information with the processor 680; in addition, the data of the design uplink is sent to the base station. Generally, RF circuit 610 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (English full name: low NoiseAmplifier, english short name: LNA), a duplexer, and the like. In addition, the RF circuitry 610 may also communicate with networks and other devices via wireless communications. The wireless communication may use any communication standard or protocol, including but not limited to global system for mobile communications (Global System ofMobile communication, GSM), general packet Radio Service (GENERAL PACKET Radio Service, GPRS), code division multiple access (Code Division MultipleAccess, CDMA), wideband code division multiple access (Wideband Code Division Multiple Access, WCDMA), long term evolution (Long Term Evolution, LTE), email, short message Service (Short MESSAGING SERVICE, SMS), etc.

The memory 620 may be used to store software programs and modules, and the processor 680 may perform various functional applications and data processing of the cellular phone by executing the software programs and modules stored in the memory 620. The memory 620 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, application programs required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, phonebook, etc.) created according to the use of the handset, etc. In addition, memory 620 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device.

The input unit 630 may be used to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the handset. In particular, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, also referred to as a touch screen, may collect touch operations thereon or thereabout by a user (e.g., operations of the user on the touch panel 631 or thereabout using any suitable object or accessory such as a finger, a stylus, etc.), and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 631 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch azimuth of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device and converts it into touch point coordinates, which are then sent to the processor 680 and can receive commands from the processor 680 and execute them. In addition, the touch panel 631 may be implemented in various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. The input unit 630 may include other input devices 632 in addition to the touch panel 631. In particular, other input devices 632 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, mouse, joystick, etc.

The display unit 640 may be used to display information input by a user or information provided to the user and various menus of the mobile phone. The display unit 640 may include a display panel 641, and optionally, the display panel 641 may be configured in the form of a Liquid crystal display (full name: liquid CRYSTAL DISPLAY, abbreviated name: LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 631 may cover the display panel 641, and when the touch panel 631 detects a touch operation thereon or thereabout, the touch panel 631 is transferred to the processor 680 to determine the type of the touch event, and then the processor 680 provides a corresponding visual output on the display panel 641 according to the type of the touch event. Although in fig. 7, the touch panel 631 and the display panel 641 are two independent components to implement the input and input functions of the mobile phone, in some embodiments, the touch panel 631 and the display panel 641 may be integrated to implement the input and output functions of the mobile phone.

The handset may also include at least one sensor 650, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 641 according to the brightness of ambient light, and a proximity sensor that may turn off the display panel 641 and/or the backlight when the mobile phone is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the acceleration in all directions (generally three axes), and can detect the gravity and direction when stationary, and can be used for applications of recognizing the gesture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer gesture calibration), vibration recognition related functions (such as pedometer and knocking), and the like; other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc. that may also be configured with the handset are not described in detail herein.

Audio circuitry 660, speaker 661, microphone 662 may provide an audio interface between a user and the handset. The audio circuit 660 may transmit the received electrical signal converted from audio data to the speaker 661, and the electrical signal is converted into a sound signal by the speaker 661 to be output; on the other hand, microphone 662 converts the collected sound signals into electrical signals, which are received by audio circuit 660 and converted into audio data, which are processed by audio data output processor 680 for transmission to, for example, another cell phone via RF circuit 610, or which are output to memory 620 for further processing.

Wi-Fi belongs to a short-distance wireless transmission technology, and a mobile phone can help a user to send and receive e-mails, browse web pages, access streaming media and the like through a Wi-Fi module 670, so that wireless broadband Internet access is provided for the user. Although fig. 7 shows Wi-Fi module 670, it is understood that it does not belong to the necessary constitution of the mobile phone, and can be omitted entirely as needed within the scope of not changing the essence of the application.

Processor 680 is a control center of the handset, connects various parts of the entire handset using various interfaces and lines, and performs various functions and processes of the handset by running or executing software programs and/or modules stored in memory 620, and invoking data stored in memory 620, thereby performing overall monitoring of the handset. Optionally, processor 680 may include one or more processing modules; preferably, the processor 680 may integrate an application processor that primarily handles operating systems, user interfaces, applications, etc., with a modem processor that primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 680.

The handset further includes a power supply 690 (e.g., a battery) for powering the various components, which may be logically connected to the processor 680 through a power management system so as to perform functions such as managing charging, discharging, and power consumption by the power management system.

Although not shown, the mobile phone may further include a camera, a bluetooth module, etc., which will not be described herein.

In the embodiment of the present application, the processor 680 included in the mobile phone further has a flowchart for controlling and executing the personnel file gathering method shown in fig. 2.

Fig. 8 is a schematic diagram of a server structure according to an embodiment of the present application, where the server 720 may have a relatively large difference due to different configurations or performances, and may include one or more central processing units (english: central processing units, english: CPU) 722 (e.g., one or more processors) and a memory 732, and one or more storage mediums 730 (e.g., one or more mass storage devices) storing application programs 742 or data 744. Wherein memory 732 and storage medium 730 may be transitory or persistent. The program stored in the storage medium 730 may include one or more modules (not shown), each of which may include a series of instruction operations on a server. Still further, the central processor 722 may be configured to communicate with the storage medium 730 and execute a series of instruction operations on the storage medium 730 on the server 720.

The Server 720 may also include one or more power supplies 726, one or more wired or wireless network interfaces 750, one or more input/output interfaces 758, and/or one or more operating systems 741, such as Windows Server, mac OS X, unix, linux, freeBSD, and the like.

The steps performed by the server in the above embodiments may be based on the structure of the server 720 shown in fig. 8. The steps of the server shown in fig. 2 in the above embodiment may be based on the server structure shown in fig. 8, for example. For example, the processor 722 may perform the following operations by invoking instructions in the memory 732:

In the foregoing embodiments, the descriptions of the embodiments are emphasized, and for parts of one embodiment that are not described in detail, reference may be made to related descriptions of other embodiments.

It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the systems, apparatuses and modules described above may refer to the corresponding processes in the foregoing method embodiments, which are not repeated herein.

In the embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are merely illustrative, and for example, the division of the modules is merely a logical function division, and there may be additional divisions when actually implemented, for example, multiple modules or components may be combined or integrated into another system, or some features may be omitted or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be an indirect coupling or communication connection via some interfaces, devices or modules, which may be in electrical, mechanical, or other forms.

The modules described as separate components may or may not be physically separate, and components shown as modules may or may not be physical modules, i.e., may be located in one place, or may be distributed over a plurality of network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as a stand-alone product, may be stored in a computer readable storage medium.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product.

The computer program product includes one or more computer instructions. When the computer program is loaded and executed on a computer, the flow or functions according to the embodiments of the present application are fully or partially produced. The computer may be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by a wired (e.g., coaxial cable, fiber optic, digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer readable storage medium may be any available medium that can be stored by a computer or a data storage device such as a server, data center, etc. that contains an integration of one or more available media. The usable medium may be a magnetic medium (e.g., floppy disk, hard disk, tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk Solid STATE DISK (SSD)), etc.

The above description has been made in detail on the technical solutions provided by the embodiments of the present application, and specific examples are applied in the embodiments of the present application to illustrate the principles and implementation manners of the embodiments of the present application, where the above description of the embodiments is only for helping to understand the methods and core ideas of the embodiments of the present application; meanwhile, as for those skilled in the art, according to the idea of the embodiment of the present application, there are various changes in the specific implementation and application scope, and in summary, the present disclosure should not be construed as limiting the embodiment of the present application.

Claims

1. A personal profiling method, comprising:

If the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file;

The first personnel file and the second personnel file are two personnel files which are required to be aggregated at present; the first personnel file and the second personnel file comprise one image or a plurality of files; the first key frame is a good-quality personnel image which is selected from the first personnel files and can represent the files; the second key frame is a person image which is selected from the second person files and can represent the files with good quality;

performing file similarity calculation on the first keyframe set and the second keyframe set according to the image feature type of the first keyframe and the image feature type of the second keyframe to obtain a target similarity value, including:

Determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold value;

Wherein the first person profile includes a plurality of images and the first person profile includes a plurality of image feature types.

2. The method according to claim 1, wherein the determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold value includes:

3. The method of claim 1, wherein after the first personnel profile and the second personnel profile are subjected to a filing process to obtain a third personnel profile, the method further comprises:

4. A method according to claim 3, wherein said selecting a target number of candidate key frames from said candidate key frames as said third key frame for a corresponding image feature type comprises:

5. The method of claim 1, wherein after the first personnel profile and the second personnel profile are subjected to a filing process to obtain a third personnel profile, the method further comprises:

6. The method of any one of claims 1 to 5, wherein the at least one image feature type comprises at least one of a full face image type, a mask-worn face image type, and a human image type.

7. A personal gear gathering device, comprising:

The processing module is used for carrying out file similarity calculation on the first key frame set and the second key frame set according to the image feature type of the first key frame and the image feature type of the second key frame to obtain a target similarity value; if the target similarity value is greater than or equal to a preset threshold value, performing file aggregation processing on the first personnel file and the second personnel file to obtain a third personnel file;

The processing module is specifically configured to, when executing the step of performing file similarity calculation on the first keyframe set and the second keyframe set according to the image feature type of the first keyframe and the image feature type of the second keyframe to obtain a target similarity value:

8. The personal profile control device according to claim 7, wherein the processing module is configured to, when executing the step of determining the target similarity value according to the comparison result of the candidate similarity value and the preset threshold value:

9. The personal profiling apparatus as defined in claim 7, wherein the processing module is further configured to, after performing the step of profiling the first personal profile and the second personal profile to obtain a third personal profile:

10. The people mover of claim 9, wherein the processing module is configured to, when executing the step of selecting a target number of candidate keyframes from the candidate keyframes as the third keyframes for the corresponding image feature types:

11. The personal profiling apparatus as defined in claim 7, wherein the processing module is further configured to, after performing the step of profiling the first personal profile and the second personal profile to obtain a third personal profile:

12. The personal organizer of any one of claims 7 to 11, wherein the at least one image feature type comprises at least one of a full face image type, a mask-worn face image type, and a human image type.

13. A computer device, characterized in that it comprises a memory on which a computer program is stored and a processor which, when executing the computer program, implements the method according to any of claims 1-6.

14. A computer readable storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, can implement the method of any of claims 1-6.

15. A computer program product comprising instructions, characterized in that the computer program product comprises program instructions which, when run on a computer or a processor, cause the computer or the processor to perform the method of any of claims 1 to 6.