CN112101305B

CN112101305B - Multi-path image processing method and device and electronic equipment

Info

Publication number: CN112101305B
Application number: CN202011240209.7A
Authority: CN
Inventors: 郑东; 彭观海; 赵拯; 赵五岳
Original assignee: Universal Ubiquitous Technology Co ltd
Current assignee: Universal Ubiquitous Technology Co ltd
Priority date: 2020-05-12
Filing date: 2020-11-09
Publication date: 2021-02-12
Anticipated expiration: 2040-11-09
Also published as: CN112101305A; CN111597953A

Abstract

The embodiment of the disclosure provides a multi-channel image processing method, a multi-channel image processing device and electronic equipment, and belongs to the technical field of image processing, wherein the method comprises the following steps: acquiring initial single-frame images collected by multiple cameras in a preset area at the same time; splicing all the initial single-frame images into a frame of first integral image; performing pedestrian detection on the first overall image to obtain pedestrian information of at least one target object in the first overall image, wherein the pedestrian information at least comprises position coordinate information of a human body and a human head area of the target object in the first overall image; determining a target single-frame image corresponding to the target object according to the position coordinate information of the target object in the first overall image; and mapping the target object and the pedestrian information thereof back to the corresponding target single-frame image so as to track the target object according to the target single-frame image and the continuous frame images thereof. Therefore, computing resources are fully utilized, computing efficiency and recognition accuracy are improved, and time consumption of face recognition is reduced.

Description

Multi-path image processing method and device and electronic equipment

Technical Field

The present disclosure relates to the field of image processing technologies, and in particular, to a method and an apparatus for processing multiple paths of images, and an electronic device.

Background

In the prior art, in order to effectively identify the information of personnel entering a designated area within a specific time period, a non-inductive attendance checking device is installed in the designated area. The non-sensory attendance checking equipment is characterized in that common image information is obtained through a camera, key part coordinate information of all people in a picture is obtained through neural network calculation according to image data, then the human face quality information is analyzed in real time in the tracking process according to the tracking information of the people in key part information pictures in front and back frame images, and a human face image meeting requirements is screened for extracting human face characteristic information. According to the face characteristics of the people entering the designated area, the information of the people approaching the area can be effectively identified.

For the situation that a plurality of faces appear in the same picture, the quality of the faces is evaluated one by one through the conventional method, and the performance of the whole system is seriously restricted by the method for extracting the features of the effective faces, even the omission of effective personnel information is caused; in actual use, when a plurality of camera images are transmitted to a face recognition system for processing, the surplus of computing resources occurs, and the technical problems of the waste of the computing resources and the serious time consumption are caused by analyzing schemes one by one.

Therefore, the existing multi-path face recognition scheme has the technical problems of waste of computing resources and serious time consumption.

Disclosure of Invention

In view of the above, embodiments of the present disclosure provide a method, an apparatus, and an electronic device for processing multiple paths of images, which at least partially solve the problems in the prior art.

In a first aspect, an embodiment of the present disclosure provides a multi-path image processing method, including:

acquiring initial single-frame images collected by multiple cameras in a preset area at the same time;

splicing all the initial single-frame images into a frame of first integral image;

performing pedestrian detection on the first overall image to obtain pedestrian information of at least one target object in the first overall image, wherein the pedestrian information at least comprises position coordinate information of a human body area and a human head area of the target object in the first overall image;

determining a target single-frame image corresponding to the target object according to the position coordinate information of the human body area and the human head area of the target object in the first overall image, wherein the target single-frame image is any one of all initial single-frame images;

and mapping the target object and the pedestrian information thereof back to a corresponding target single-frame image so as to track the target object according to the target single-frame image and the continuous frame images thereof.

According to a specific implementation manner of the embodiment of the present disclosure, the step of stitching all the initial single-frame images into a frame of the first overall image includes:

determining the current number of all the initial single-frame images and the original size of each initial single-frame image;

determining a splicing scheme according to a preset splicing rule and the current number and the original size of all the initial single-frame images;

and according to the determined splicing scheme, splicing all the initial single-frame images into the first whole image.

According to a specific implementation manner of the embodiment of the present disclosure, the difference values between the original sizes of all the initial single-frame images are within a preset range;

the step of determining the stitching scheme according to the preset stitching rule and the current number and the original size of all the initial single-frame images comprises the following steps:

judging whether the current number of all the initial single-frame images is a complete square number or not;

if the current number of the initial single-frame images is a complete square number, determining a splicing scheme that the square root of the current number is used as the number of rows and columns, and uniformly splicing all the initial single-frame images into a square matrix;

and if the current number of the initial single-frame images is not the complete square number, determining a splicing scheme that the current number is matched with the minimum filling number to form a target complete square number, taking the square root of the target complete square number as the number of rows and columns, and uniformly splicing all the initial single-frame images into a square matrix.

According to a specific implementation manner of the embodiment of the present disclosure, before the step of determining the target single-frame image corresponding to the target object according to the position coordinate information of the target object in the first overall image, the method further includes:

generating a splicing mapping relation between each initial single-frame image and a corresponding pixel region in the first overall image;

the step of determining a target single-frame image corresponding to the target object according to the position coordinate information of the target object in the first overall image includes:

determining a target pixel area according to the position coordinate information of the target object in the first overall image;

determining an initial single-frame image corresponding to the target pixel area according to the splicing mapping relation;

and taking the initial single-frame image corresponding to the target pixel area as a target single-frame image corresponding to the target object.

According to a specific implementation manner of the embodiment of the present disclosure, the step of mapping the target object and the pedestrian information thereof back to the corresponding target single-frame image to perform tracking processing on the target object according to the target single-frame image and the continuous frame images thereof includes:

determining an associated frame image in continuous multi-frame images collected by a target camera according to the target object and the pedestrian information thereof, wherein the target camera is a camera for collecting a single frame image of the target, and the associated frame image is an image containing the corresponding pixel characteristics of the target object;

obtaining the face quality score of the target object in each associated frame image;

and storing or iteratively updating the face image of the target object in a preset storage space according to the face quality scores of the target object in all the associated frame images.

According to a specific implementation manner of the embodiment of the present disclosure, the step of obtaining the face quality score of the target object in each associated frame image includes:

determining all target objects contained in the associated frame image;

splicing the face pixel areas of all the target objects into a second integral picture;

and inputting the second overall image into a face quality recognition model to obtain the face quality score of each target object in the associated frame image.

all the initial single-frame images are subjected to scaling processing, and the scaled initial single-frame images are spliced into the first integral image;

the step of splicing all the face pixel regions of the target object into a second overall picture comprises the following steps:

screening out candidate target objects with the sizes of the face pixel regions meeting a preset range;

carrying out scaling adjustment and filling adjustment on the face pixel regions corresponding to all the candidate target objects so as to enable the face pixel regions corresponding to all the candidate target objects to be the same in size;

and splicing the face pixel areas of the candidate target objects with the same size into the second integral image.

According to a specific implementation manner of the embodiment of the present disclosure, the step of storing or iteratively updating the face image of the target object in a preset storage space according to the face quality scores of the target object in all the associated frame images includes:

judging whether a reference face image of the target object is stored in a preset storage space or not;

if the reference face image of the target object is not stored in the preset storage space, storing the associated frame image as the reference face image of the target object into the preset storage space;

if the reference face image of the target object is stored in the preset storage space, judging whether the face quality score of the reference face image of the target object is smaller than the face quality score in the associated frame image;

and if the face quality score of the reference face image of the target object is smaller than the face quality score in the associated frame image, storing the face image of the target object in the associated frame image as a new reference face image of the target object in the preset storage space.

In a second aspect, an embodiment of the present disclosure provides a multi-path image processing apparatus, including:

the acquisition module is used for acquiring initial single-frame images acquired by multiple cameras in a preset area at the same time;

the splicing module is used for splicing all the initial single-frame images into a first integral image;

the detection module is used for carrying out pedestrian detection on the first overall image so as to obtain pedestrian information of at least one target object in the first overall image, wherein the pedestrian information at least comprises position coordinate information of a human body area and a human head area of the target object in the first overall image;

a determining module, configured to determine a target single-frame image corresponding to the target object according to position coordinate information of a human body region and a head region of the target object in the first overall image, where the target single-frame image is any one of all initial single-frame images;

and the mapping module is used for mapping the target object and the pedestrian information thereof back to a corresponding target single-frame image so as to track the target object according to the target single-frame image and the continuous frame images thereof.

According to a specific implementation manner of the embodiment of the present disclosure, the splicing module is configured to:

the splicing module is used for:

the determining module is configured to:

According to a specific implementation manner of the embodiment of the present disclosure, the determining module is further configured to:

the device further comprises:

the quality analysis module is used for obtaining the face quality score of the target object in each associated frame image;

and the iteration updating module is used for storing or iteratively updating the face image of the target object in a preset storage space according to the face quality scores of the target object in all the associated frame images.

According to a specific implementation manner of the embodiment of the present disclosure, the quality analysis module is configured to:

determining all target objects contained in the associated frame image;

According to a specific implementation manner of the embodiment of the present disclosure, the iterative update module is configured to:

In a third aspect, an embodiment of the present disclosure further provides an electronic device, where the electronic device includes:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of processing multiple images in any of the implementations of the first aspect or the first aspect.

In a fourth aspect, the disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute the multi-path image processing method in the first aspect or any implementation manner of the first aspect.

In a fifth aspect, the present disclosure also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions that, when executed by a computer, cause the computer to perform the multi-path image processing method in the foregoing first aspect or any implementation manner of the first aspect.

The multi-path image processing method and the device in the embodiment of the disclosure comprise the following steps: acquiring initial single-frame images collected by multiple cameras in a preset area at the same time; splicing all the initial single-frame images into a frame of first integral image; performing pedestrian detection on the first overall image to obtain pedestrian information of at least one target object in the first overall image, wherein the pedestrian information at least comprises position coordinate information of a human body area and a human head area of the target object in the first overall image; determining a target single-frame image corresponding to the target object according to the position coordinate information of the human body area and the human head area of the target object in the first overall image, wherein the target single-frame image is any one of all initial single-frame images; and mapping the target object and the pedestrian information thereof back to a corresponding target single-frame image so as to track the target object according to the target single-frame image and the continuous frame images thereof. According to the scheme, the initial single-frame images collected by the multiple paths of cameras are spliced into the first integral image, the target objects in the initial single-frame images can be identified and the information can be extracted at the same time, and the target objects are mapped back to the original image. Therefore, computing resources are fully utilized, resource waste is avoided, computing efficiency and recognition accuracy are improved, and time consumption of face recognition is reduced.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings needed to be used in the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present disclosure, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.

Fig. 1 is a schematic flowchart of a multi-path image processing method according to an embodiment of the disclosure;

fig. 2a to fig. 2c are schematic diagrams of a first overall image according to a multi-path image processing method provided in the embodiment of the disclosure;

fig. 3 is a partial schematic flow chart of another multi-path image processing method according to an embodiment of the disclosure;

fig. 4a and 4b are schematic diagrams of a second overall image according to a multi-path image processing method provided by the embodiment of the disclosure;

fig. 5 is a schematic structural diagram of a multi-path image processing apparatus according to an embodiment of the disclosure;

fig. 6 is a schematic view of an electronic device provided in an embodiment of the present disclosure.

Detailed Description

The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

The embodiments of the present disclosure are described below with specific examples, and other advantages and effects of the present disclosure will be readily apparent to those skilled in the art from the disclosure in the specification. It is to be understood that the described embodiments are merely illustrative of some, and not restrictive, of the embodiments of the disclosure. The disclosure may be embodied or carried out in various other specific embodiments, and various modifications and changes may be made in the details within the description without departing from the spirit of the disclosure. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

It is noted that various aspects of the embodiments are described below within the scope of the appended claims. It should be apparent that the aspects described herein may be embodied in a wide variety of forms and that any specific structure and/or function described herein is merely illustrative. Based on the disclosure, one skilled in the art should appreciate that one aspect described herein may be implemented independently of any other aspects and that two or more of these aspects may be combined in various ways. For example, an apparatus may be implemented and/or a method practiced using any number of the aspects set forth herein. Additionally, such an apparatus may be implemented and/or such a method may be practiced using other structure and/or functionality in addition to one or more of the aspects set forth herein.

It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present disclosure, and the drawings only show the components related to the present disclosure rather than the number, shape and size of the components in actual implementation, and the type, amount and ratio of the components in actual implementation may be changed arbitrarily, and the layout of the components may be more complicated.

In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects may be practiced without these specific details.

The embodiment of the disclosure provides a multipath image processing method. The multi-path image processing method provided by the embodiment can be executed by a computing device, the computing device can be implemented as software, or implemented as a combination of software and hardware, and the computing device can be integrated in a server, a terminal device and the like.

Referring to fig. 1, a flow chart of a multi-path image processing method according to an embodiment of the present disclosure is schematically shown. As shown in fig. 1, the method mainly comprises the following steps:

s101, acquiring initial single-frame images collected by multiple paths of cameras in a preset area at the same time;

the multi-path image processing method is applied to a scene that a plurality of paths of cameras are installed in the same preset area to acquire and process images. The method comprises the steps of installing multiple paths of cameras with different visual angles at different positions of a preset area, collecting images of different areas in the preset area respectively, and splicing the collected multiple paths of images by using the scheme provided by the embodiment to perform centralized processing so as to improve the efficiency of image processing.

Firstly, acquiring a frame of original image acquired by a plurality of cameras in a preset area at a certain moment, and defining the frame of original image as an initial single-frame image, wherein the number of the initial single-frame images is consistent with that of the cameras in the preset area.

S102, splicing all the initial single-frame images into a first integral image;

as shown in fig. 2a to 2c, all the initial single-frame images obtained in the above steps (i.e., T1, T2, T3 and T4 shown in the figure) are spliced into a whole image, which is defined as a first whole image. Thus, the pixel characteristics of the initial multi-frame image are concentrated and simultaneously embodied in the first whole image of the frame.

According to a specific implementation manner of the embodiment of the present disclosure, as shown in fig. 3, the step of splicing all the initial single-frame images into a first overall image may specifically include:

s301, determining the current number of all the initial single-frame images and the original size of each initial single-frame image;

when image splicing is carried out, the number of initial single-frame images and the original size of each initial single-frame image need to be referred to, so that a splicing scheme can be determined subsequently, and the size of the spliced first overall image meets the size required by subsequent processing.

The number of the initial single-frame images is the same as the number of the multiple cameras from which the images are derived, that is, one camera provides one initial single-frame image, and the size of each initial single-frame image is consistent with the attribute parameters of the camera, the sizes of all the acquired initial single-frame images may be completely the same or different, and the width and height sizes of the initial single-frame images may not be consistent, for example, the image resolution of the initial single-frame image, that is, the width and height are 1920 × 1080, and the like, without limitation.

S302, determining a splicing scheme according to a preset splicing rule and the current number and the original size of all the initial single-frame images;

the electronic equipment is internally pre-stored with preset splicing rules for limiting how to splice a plurality of initial single-frame images into a first integral image. Considering that the first overall image needs to be input into the convolutional neural network model during subsequent analysis, the first overall image after splicing is limited to be square as much as possible, namely the number of the images arranged on the rows is equal, and the total size of the rows and the columns of the first overall image is relatively close.

preprocessing all initial single-frame images, wherein the preprocessing comprises equal scaling processing and/or filling processing;

according to the formula

And, in addition,

calculating the optimal solution of the number of rows and columns when all the initial single-frame images are arrayed, wherein width represents the width of each initial single-frame image after preprocessing, height represents the height of each initial single-frame image after preprocessing, cols represents the number of the initial single-frame images arranged in the row direction when the initial single-frame images are spliced into a first overall image, N represents the number of all the initial single-frame images, and rows represents the number of the initial single-frame images arranged in the column direction when the initial single-frame images are spliced into the array;

and determining a splicing scheme according to the identification information of all the initial single-frame images and the optimal solution of the array-arranged row and column numbers.

In this embodiment, the difference between the original sizes of all the initial single-frame images is limited within the preset range, so that when the splicing scheme is confirmed, only the number of the initial single-frame images needs to be considered, and the influence of the original sizes of different initial single-frame images on the splicing scheme does not need to be considered.

For example, as shown in fig. 2a, if there are 2 initial single-frame images, they may be directly stitched up and down, or two filler images may be added on the right side. As shown in fig. 2b, if there are 3 initial single frame images, 3 is not a perfect square number, and 3 can be matched with the minimum padding number 1 to form the target perfect square number 4, with the square root 2 of 4 as the number of rows and columns. As shown in fig. 2c, if there are 4 initial single-frame images, 4 is a complete square number, the square root 2 of 4 can be directly used as the number of rows and columns, and the 4 initial single-frame images can be spliced into a square matrix. As shown in fig. 2b, for the case that the minimum filling number needs to be matched, the filling image corresponding to the minimum filling number (T0 shown in the figure) is added so that all the images are arranged in a square matrix.

S303, splicing all the initial single-frame images into the first whole image according to the determined splicing scheme.

After the splicing scheme is determined according to the steps, all the initial single-frame images can be spliced into one initial single-frame image according to the determined splicing scheme.

In addition, considering that the size of a single initial single-frame image is large, when the splicing is carried out, all the initial single-frame images can be firstly subjected to reduction processing to a certain degree, so that the size of the spliced first overall image is not too large, and the later analysis processing is facilitated.

S103, carrying out pedestrian detection on the first overall image to obtain pedestrian information of at least one target object in the first overall image, wherein the pedestrian information at least comprises position coordinate information of a human body area and a human head area of the target object in the first overall image;

after the initial single-frame images are spliced into the whole first whole image, the personnel features on each initial single-frame image are displayed on the first whole image in a concentrated mode.

After all the initial single-frame images are spliced into a first integral image according to the steps, the pedestrian detection can be carried out on the first integral image, so that all the personnel characteristics in the first integral image can be obtained at the same time, and the personnel capable of being detected on the first integral image are defined as the target object. The first overall image is input into the face recognition model, so that the computation resources of the face recognition model can be effectively utilized to simultaneously extract pedestrian information of all target objects in the first overall image, such as face feature information and position coordinate information, and body information, posture information and the like can be included without limitation.

S104, determining a target single-frame image corresponding to the target object according to the position coordinate information of the human body area and the human head area of the target object in the first overall image, wherein the target single-frame image is any one of all initial single-frame images;

after all the target objects are obtained through pedestrian detection in the first overall image, the process of mapping the target objects back to the original single-frame image can be started. When a target object is detected in the first overall image, the acquired pedestrian information of the target object comprises position coordinate information of the target object in the first overall image, and the source initial single-frame image corresponding to the target object can be found according to the position coordinate information and is defined as a target single-frame image.

Optionally, according to a specific implementation manner of the embodiment of the present disclosure, before the step of determining the target single-frame image corresponding to the target object according to the position coordinate information of the target object in the first overall image, the method may further include:

In this embodiment, the target single-frame image corresponding to each target object is searched for according to the stitching mapping relationship between the initial single-frame image and the pixel region, which is generated when all the initial single-frame images are stitched into the first whole image.

And S105, mapping the target object and the pedestrian information thereof back to a corresponding target single-frame image so as to track the target object according to the target single-frame image and the continuous frame images thereof.

After all target objects are identified according to the steps and the target single-frame image corresponding to each target object is determined, the target object and the pedestrian information of each target object can be mapped back to the corresponding target single-frame image, and then the target object is tracked on the target single-frame image and the continuous frame images of the target object by combining the pedestrian information of the target object.

According to the scheme provided by the embodiment of the disclosure, the initial single-frame images acquired by the multiple paths of cameras are spliced into the first integral image, so that the target objects in the initial single-frame images can be identified and the information can be extracted at the same time, and then the target objects are mapped back to the original image. Therefore, computing resources are fully utilized, resource waste is avoided, computing efficiency and identification accuracy are improved, and time consumed by pedestrian detection is reduced.

According to a specific implementation manner of the embodiment of the present disclosure, the step of mapping the target object and the pedestrian information thereof back to the corresponding target single-frame image so as to perform the tracking processing on the target object according to the target single-frame image and the continuous frame images thereof in S105 may include:

The electronic equipment is internally provided with a preset storage space for storing and identifying the face image of the detected target object, and the preset storage space is used for storing the face image with higher face quality score, so that more detailed characteristic information can be conveniently obtained. The camera which acquires the target object is defined as a target camera, the target camera continuously acquires multi-frame images including a target single-frame image, and in the continuous multi-frame images, the target object appears to disappear, and an image which can detect the pixel characteristics corresponding to the target object is defined as an associated frame image.

Determining the face quality score of the target object in each associated frame image, wherein the higher the face quality score is, the higher the feature detail degree of the corresponding face image is, and storing or iteratively updating the face image in the preset storage space by using the face quality score in each associated frame image.

Further, according to a specific implementation manner of the embodiment of the present disclosure, the step of obtaining the face quality score of the target object in each associated frame image includes:

determining all target objects contained in the associated frame image;

As shown in fig. 4a and 4b, each associated frame includes a plurality of target objects, a face pixel region corresponding to each target object is extracted, the face pixel regions of all the target objects are spliced into a frame of a second overall image, and a face quality recognition model is input to obtain a face quality score of the face pixel region corresponding to each target object.

Therefore, the face quality scores of the face pixel regions corresponding to the target objects can be calculated simultaneously, calculation resources are fully utilized, resource waste is avoided, and the calculation efficiency and accuracy of the face quality scores are improved.

Further, according to a specific implementation manner of the embodiment of the present disclosure, the step of splicing all the initial single-frame images into a first overall image includes:

The present embodiment further defines the process flow of stitching the first whole image and the second whole image. Specifically, before all the initial single-frame images are spliced into the first overall image, all the initial single-frame images are subjected to scaling processing, particularly to reduction processing, and then all the initial single-frame images after being subjected to equal-scale reduction are spliced into the first overall image. Correspondingly, when the corresponding target single-frame image is mapped back, corresponding equal-scale amplification processing needs to be performed again to ensure that the original pixel characteristics of each target object are not changed.

When the second integral image is spliced, size constraint needs to be carried out on the face pixel area, namely, a candidate target object with the size of the face pixel area meeting the preset range is screened out, so that a small face, an overlarge face or a shielded face is filtered out. And then, scaling the screened face to a specified size in an equal proportion, filling edges in part of face pixel regions to ensure that the sizes of the face pixel regions after scaling adjustment and filling adjustment are the same, and splicing to obtain the effect shown in fig. 4a and 4 b.

In addition, according to another specific implementation manner of the embodiment of the present disclosure, the step of storing or iteratively updating the face image of the target object in a preset storage space according to the face quality scores of the target object in all the associated frame images may specifically include:

In this embodiment, it is limited to the iterative update process, and it is first determined whether new storage needs to be added or whether the stored image needs to be iteratively updated. Specifically, the face image of the target object stored in the storage space is defined as a reference face image. And if the reference face image is not stored in advance, storing the face image corresponding to the target object in the current associated frame into the preset space. If the reference face image is stored in advance, judging whether the face quality score of the current reference face image is smaller than the face quality score of the image in the associated frame, and if so, storing the face image of the target object in the associated frame with higher face quality score as a new reference face image. Of course, considering that there may be a plurality of reference face storage images, at this time, the face quality scores of the face image to be added and the cached reference face image may be compared and sorted, an image with a high face quality score is located in front of an image with a low score, and when the total number of the reference face storage images exceeds the total number of the reference face storage images, the face image with a low face quality score exceeding the total number of the reference face storage images needs to be deleted.

Taking a face detection snapshot project of a hisi3559 platform as an example, the time consumed for pedestrian detection of a single road is about 15ms, and the time consumed for judging the quality of a single face is 1.2 ms; splicing the pictures for 3ms, and detecting the spliced face for about 17 ms; the human face quality is obtained after the human faces are spliced together, and the analysis takes about 11ms after 16 human faces are spliced. Thus, when there are at most 4 videos, each video is generally about 13 people, and the comparison is as follows:

the original protocol total time =4 (lane) × 15 (ms/lane) +4 (lane) × 13 (person/lane) = 1.2 (ms/person) =122.4 ms;

the total time consumption of the scheme is =3 (ms/time) × 1 (time, 4 paths of videos are spliced to the whole graph 1) +17 (ms/time) × 1 (time, pedestrian detection is carried out on the whole graph 1) +3 (ms/time) × 4 (time, each path of video needs face splicing) +4 (path) × 11 (ms/path) =76 ms;

roughly estimated, performance can be improved by nearly 40%. Moreover, when the method is adopted, the more the number of people is, the more the performance improvement is more obvious compared with the original scheme.

To sum up, the multi-channel image processing method provided by the embodiment of the present disclosure splices a plurality of channels of video into one frame, and detects a target area for the spliced frame; and splicing all the target areas detected in the spliced picture together according to rules, and uniformly outputting the quality results of the targets. The method has the advantages of fully utilizing computing resources, avoiding resource waste, improving computing efficiency and identification accuracy and reducing time consumption of face identification.

Corresponding to the above method embodiment, referring to fig. 5, the disclosed embodiment further provides a multi-path image processing apparatus 50, including:

the acquiring module 501 is configured to acquire an initial single-frame image acquired by multiple cameras in a preset area at the same time;

a stitching module 502, configured to stitch all the initial single-frame images into a first whole image;

a detection module 503, configured to perform pedestrian detection on the first whole image to obtain pedestrian information of at least one target object in the first whole image, where the pedestrian information at least includes position coordinate information of a human body region and a human head region of the target object in the first whole image;

a determining module 504, configured to determine a target single-frame image corresponding to the target object according to position coordinate information of a human body region and a head region of the target object in the first overall image, where the target single-frame image is any one of all initial single-frame images;

and the mapping module 505 is configured to map the target object and the pedestrian information thereof back to a corresponding target single-frame image, so as to perform tracking processing on the target object according to the target single-frame image and continuous frame images thereof.

According to a specific implementation manner of the embodiment of the present disclosure, the splicing module 502 is configured to:

the splicing module is used for:

the determination module is to:

the device further comprises:

determining all target objects contained in the associated frame image;

The apparatus shown in fig. 5 may correspondingly execute the content in the above method embodiment, and details of the part not described in detail in this embodiment refer to the content described in the above method embodiment, which is not described again here.

Referring to fig. 6, an embodiment of the present disclosure also provides an electronic device 60, including:

at least one processor; and the number of the first and second groups,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of multiplexed image processing in the method embodiments described above.

The disclosed embodiments also provide a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the multi-pass image processing method in the aforementioned method embodiments.

The disclosed embodiments also provide a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, cause the computer to perform the multi-pass image processing method in the aforementioned method embodiments.

Referring now to FIG. 6, a schematic diagram of an electronic device 60 suitable for use in implementing embodiments of the present disclosure is shown. The electronic devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., car navigation terminals), and the like, and fixed terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.

As shown in fig. 6, the electronic device 60 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data necessary for the operation of the electronic apparatus 60 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Generally, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touch pad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, tape, hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 60 to communicate with other devices wirelessly or by wire to exchange data. While the figures illustrate an electronic device 60 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 609, or may be installed from the storage means 608, or may be installed from the ROM 602. The computer program, when executed by the processing device 601, performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

The computer readable medium may be embodied in the electronic device; or may exist separately without being assembled into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, enable the electronic device to implement the schemes provided by the method embodiments.

Alternatively, the computer readable medium carries one or more programs, which when executed by the electronic device, enable the electronic device to implement the schemes provided by the method embodiments.

Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C + +, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The units described in the embodiments of the present disclosure may be implemented by software or hardware. Where the name of a unit does not in some cases constitute a limitation of the unit itself, for example, the first retrieving unit may also be described as a "unit for retrieving at least two internet protocol addresses".

It should be understood that portions of the present disclosure may be implemented in hardware, software, firmware, or a combination thereof.

The above description is only for the specific embodiments of the present disclosure, but the scope of the present disclosure is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present disclosure should be covered within the scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the protection scope of the claims.

Claims

1. A method of multi-pass image processing, comprising:

stitching all the initial single-frame images into a frame of a first whole image comprises:

according to a preset splicing rule and the current number and the original size of all the initial single-frame images, determining a splicing scheme comprises the following steps:

according to the formula

And, in addition,

determining a splicing scheme according to the identification information of all the initial single-frame images and the optimal solution of the array-arranged row and column numbers;

splicing all the initial single-frame images into the first whole image according to the determined splicing scheme;

2. The method according to claim 1, wherein before the step of determining the target single-frame image corresponding to the target object according to the position coordinate information of the target object in the first whole image, the method further comprises:

3. The method according to any one of claims 1 to 2, wherein the step of mapping the target object and its pedestrian information back to a corresponding target single-frame image to perform tracking processing on the target object according to the target single-frame image and its continuous frame images comprises:

4. The method of claim 3, wherein the step of obtaining the face quality score of the target object in each associated frame image comprises:

determining all target objects contained in the associated frame image;

5. The method of claim 4, wherein said step of stitching all of said initial single frame images into a frame of a first whole image comprises:

6. The method according to claim 5, wherein the step of storing or iteratively updating the face image of the target object in a preset storage space according to the face quality scores of the target object in all the associated frame images comprises:

7. A multi-path image processing apparatus, comprising:

the splicing module is used for splicing all the initial single-frame images into a first integral image, and comprises:

according to the formula

And, in addition,

calculating the optimal solution of the number of rows and columns when all the initial single-frame images are arrayed, wherein width represents the width of each initial single-frame image after preprocessing, height represents the height of each initial single-frame image after preprocessing, cols represents the number of the initial single-frame images arranged in the row direction when the initial single-frame images are spliced into a first overall image, N represents the number of all the initial single-frame images, and rows represents the initial single-frame images arranged in the column direction when the initial single-frame images are spliced into the arrayThe number of images;

8. An electronic device, characterized in that the electronic device comprises:

at least one processor; and the number of the first and second groups,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the multi-pass image processing method of any one of the preceding claims 1-6.