CN113435346A

CN113435346A - Image processing method, image processing device, electronic equipment and computer storage medium

Info

Publication number: CN113435346A
Application number: CN202110728996.8A
Authority: CN
Inventors: 孙贺然
Original assignee: Beijing Sensetime Technology Development Co Ltd
Current assignee: Beijing Sensetime Technology Development Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2021-09-24

Abstract

The embodiment discloses an image processing method, an image processing device, electronic equipment and a computer storage medium, wherein the method comprises the following steps: acquiring an image of the motion field: carrying out human body detection on the image of the motion field to obtain at least one human body image; aiming at each human body image in at least one human body image, identifying multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image; determining human body identity information in the human body image according to the obtained identification result; and processing the images of the motion field according to the human identity information of each human image in the images of the motion field.

Description

Image processing method, image processing device, electronic equipment and computer storage medium

Technical Field

The present disclosure relates to computer vision processing technologies, and in particular, to an image processing method and apparatus, an electronic device, and a computer storage medium.

Background

In the related art, in order to analyze the motion data of the Person in the motion field, the Person in the motion field needs to be identified, and although the image of the motion field may be identified by using a pedestrian-identification (ReID) or a face recognition method in the related art, the accuracy of identifying the Person is low when the image acquisition device, such as a camera, is far away from the Person in the motion field.

Disclosure of Invention

The embodiment of the disclosure provides a technical scheme for image processing.

The embodiment of the present disclosure provides an image processing method, including:

acquiring an image of a motion field;

carrying out human body detection on the image of the motion field to obtain at least one human body image;

aiming at each human body image in the at least one human body image, identifying multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image respectively; determining human body identity information in the human body image according to the obtained identification result;

and processing the images of the motion field according to the human identity information of each human image in the images of the motion field.

In some embodiments, the plurality of attributes includes clothing color and text content; the identifying the multiple attributes of the human body image to obtain the identification results respectively corresponding to the multiple attributes in the human body image comprises the following steps:

carrying out clothing color identification on the human body image to obtain clothing color information of the human body image; and performing text recognition on the human body image to obtain text information in the human body image.

In some embodiments, the identifying, for each human body image in the at least one human body image, multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image respectively includes:

and under the condition that the quality score of a first human body image in the at least one human body image is lower than a quality score threshold value, identifying the multiple attributes of the first human body image to obtain identification results corresponding to the multiple attributes in the first human body image respectively.

In some embodiments, the method further comprises:

and under the condition that the number of pixel points of a first target object in the first human body image is smaller than a set number threshold, determining that the quality score of the first human body image is lower than the quality score threshold.

In some embodiments, the first target object represents at least part of a human body.

In some embodiments, the method further comprises:

acquiring depth information corresponding to the first human body image; determining the distance between the human body in the first human body image and image acquisition equipment for acquiring the first human body image according to the depth information corresponding to the first human body image;

determining that a quality score of the first human body image is below the quality score threshold if the distance is greater than a set distance.

In some embodiments, the method further comprises:

processing each human body image in the at least one human body image to obtain a processing result of each human body image;

the processing the image of the motion field according to the human identity information of each human image in the image of the motion field includes:

and processing the images of the motion field according to the processing result of each human body image and the human body identity information in each human body image.

In some embodiments, the processing result of each human body image includes at least one of position information and behavior information of a human body in each human body image;

the processing of each human body image in the at least one human body image to obtain a processing result of each human body image includes at least one of the following items:

analyzing the human body position of each human body image to obtain the position information of the human body in each human body image;

and performing behavior analysis on each human body image to obtain behavior information of the human body in each human body image.

In some embodiments, the processing the image of the motion field according to the processing result of each human body image and the human body identity information in each human body image includes:

and obtaining motion data corresponding to the human identity information in the images of the motion field according to the processing result of each human image and the human identity information in each human image.

In some embodiments, the playing field is a football field; the processing result of each human body image comprises position information and behavior information of the human body

The obtaining of the motion data corresponding to the human body identity information in the image of the motion field according to the processing result of each human body image and the human body identity information in each human body image includes:

and obtaining motion data corresponding to the human identity information in the images of the football field according to the position information and the behavior information of the human body in each human image and the human identity information in each human image.

In some embodiments, the method further comprises:

detecting a second target object in the image of the motion field to obtain the second target object in the image of the motion field; analyzing the second target object, and determining an analysis result of the second target object in the image of the motion field; the second target object comprises at least one of a moving object and a motion field keypoint;

the processing the image of the motion field according to the processing result of each human body image and the human body identity information in each human body image comprises:

and processing the image of the motion field according to the processing result of each human body image, the analysis result of the second target object in the image of the motion field and the human body identity information in each human body image.

In some embodiments, the processing the image of the motion field according to the processing result of each human body image, the analysis result of the second target object in the image of the motion field, and the human body identity information in each human body image includes:

and extracting an image with preset conditions from the motion field image according to the processing result of each human body image, the analysis result of the second target object in the image of the motion field and the human body identity information in each human body image.

An embodiment of the present disclosure further provides an image processing apparatus, including:

the acquisition module is used for acquiring an image of a motion field;

the detection module is used for carrying out human body detection on the image of the motion field to obtain at least one human body image;

the first processing module is used for identifying multiple attributes of each human body image in the at least one human body image to obtain identification results corresponding to the multiple attributes in the human body image; determining human body identity information in the human body image according to the obtained identification result;

and the second processing module is used for processing the images of the motion field according to the human identity information of each human image in the images of the motion field.

The disclosed embodiments also provide an electronic device comprising a processor and a memory for storing a computer program capable of running on the processor; wherein the processor is configured to run the computer program to perform any one of the image processing methods described above.

The disclosed embodiments also provide a computer storage medium having a computer program stored thereon, which when executed by a processor implements any of the image processing methods described above.

In the image processing method, the image processing device, the electronic device and the computer storage medium provided by the embodiment of the disclosure, first, an image of a motion field is acquired; then, carrying out human body detection on the image of the motion field to obtain at least one human body image; aiming at each human body image in at least one human body image, identifying multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image; determining human body identity information in the human body image according to the obtained identification result; and finally, processing the image of the motion field according to the human identity information of each human image in the image of the motion field.

It can be seen that in the embodiment of the present disclosure, the identification of the human identity can be performed based on multiple attributes in the human image, and even under the condition that the distance between the image acquisition device and the human body is far, the identification of the human identity is favorably and accurately identified according to the multiple attributes in the human image, and further, the processing precision of the image of the sports field is favorably improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the disclosure.

FIG. 1 is a flow chart of an image processing method of an embodiment of the present disclosure;

FIG. 2A is a schematic diagram of data associated with a soccer game presented on an interface of an electronic device in an embodiment of the present disclosure;

FIG. 2B is a schematic diagram of another data associated with a soccer game presented on an interface of an electronic device in an embodiment of the present disclosure;

FIG. 3 is a schematic diagram of a human detection box in a football field image presented on an interface of an electronic device in an embodiment of the disclosure;

fig. 4 is a schematic diagram of key points of a football pitch in a football pitch image presented on an interface of an electronic device in an embodiment of the present disclosure;

FIG. 5A is a schematic diagram of statistics of a football game presented on an interface of an electronic device in an embodiment of the disclosure;

FIG. 5B is a schematic diagram of highlights and strategic attack data presented on an interface of an electronic device in an embodiment of the disclosure;

FIG. 6A is a schematic view of person movement data presented on an interface of an electronic device in an embodiment of the disclosure;

FIG. 6B is another schematic diagram of person movement data presented on an interface of an electronic device in an embodiment of the disclosure;

fig. 7 is a schematic interface diagram illustrating sharing of a processing result of a soccer field image according to an embodiment of the present disclosure;

fig. 8 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present disclosure;

fig. 9 is a schematic structural diagram of an electronic device according to an embodiment of the disclosure.

Detailed Description

The present disclosure will be described in further detail below with reference to the accompanying drawings and examples. It is to be understood that the examples provided herein are merely illustrative of the present disclosure and are not intended to limit the present disclosure. In addition, the embodiments provided below are some embodiments for implementing the disclosure, not all embodiments for implementing the disclosure, and the technical solutions described in the embodiments of the disclosure may be implemented in any combination without conflict.

It should be noted that, in the embodiments of the present disclosure, the terms "comprises," "comprising," or any other variation thereof are intended to cover a non-exclusive inclusion, so that a method or apparatus including a series of elements includes not only the explicitly recited elements but also other elements not explicitly listed or inherent to the method or apparatus. Without further limitation, the use of the phrase "including a. -. said." does not exclude the presence of other elements (e.g., steps in a method or elements in a device, such as portions of circuitry, processors, programs, software, etc.) in the method or device in which the element is included.

For example, the image processing method provided by the embodiment of the present disclosure includes a series of steps, but the image processing method provided by the embodiment of the present disclosure is not limited to the described steps, and similarly, the image processing apparatus provided by the embodiment of the present disclosure includes a series of modules, but the apparatus provided by the embodiment of the present disclosure is not limited to include the explicitly described modules, and may also include modules that are required to be configured for acquiring relevant information or performing processing based on the information.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the term "at least one" herein means any one of a plurality or any combination of at least two of a plurality, for example, including at least one of A, B, C, and may mean including any one or more elements selected from the group consisting of A, B and C.

The disclosed embodiments may be implemented in computer systems comprising terminals and/or servers and may be operational with numerous other general purpose or special purpose computing system environments or configurations. Here, the terminal may be a thin client, a thick client, a hand-held or laptop device, a microprocessor-based system, a set-top box, a programmable consumer electronics, a network personal computer, a small computer system, etc., and the server may be a small computer system, a large computer system, a distributed cloud computing environment including any of the above, etc.

The electronic devices such as the terminal and the server can realize corresponding functions through the execution of the program modules. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so forth. The computer system/server may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media including memory storage devices.

The embodiment of the present disclosure provides an image processing method to process images of various sports fields, where a sports field represents a field for sports and games, for example, various fields such as a football field and a basketball field, and other non-ball fields such as an athletic field; the processing of the image of the motion field comprises at least human identification within the motion field.

Fig. 1 is a flowchart of an image processing method according to an embodiment of the disclosure, and as shown in fig. 1, the flowchart may include:

step 101: an image of the motion field is acquired.

Here, the image of the motion field may be at least one frame image; when the number of at least one frame of image is more than 1, the image of at least one frame of motion field can be a continuous multiframe image or a discontinuous multiframe image; the image of the motion field may be acquired by an image acquisition device, which may be a mobile terminal or a device such as a camera independent from the terminal, for example, or may be acquired from a local storage area or through a data transmission means such as a network; for example, in the case of acquiring an image of a motion field by using a plurality of image acquisition devices, images acquired by the plurality of image acquisition devices at the same time may be fused to obtain an image of the motion field after fusion processing.

The format of the motion field image may be Joint Photographic Experts GROUP (JPEG), Bitmap (BMP), Portable Network Graphics (PNG), or other formats; it should be noted that, the format and the source of the motion field image are merely exemplified, and the disclosed embodiments do not limit the format and the source of the motion field image.

Step 102: and carrying out human body detection on the image of the motion field to obtain at least one human body image.

In practical application, a human body can be detected from the image to obtain a human body detection frame corresponding to the image, and the image selected by the human body detection frame is used as a human body image. In one implementation, the above implementation may be implemented by training the human body detection model in advance, or by using other implementations. Wherein the human body detection model is used for detecting a human body from the image. After the trained human body detection model is obtained, the image of the motion field can be input into the human body detection model, the image of the motion field is processed by the human body detection model, a human body detection frame corresponding to the image is obtained, and the image in the human body detection frame is the human body image.

The Network structure of the human body detection model is not limited in the embodiments of the present disclosure, and may be a two-stage detection Network structure, for example, the Network structure of the human body detection model is a fast-Regions with a probabilistic Neural Network (fast RCNN), etc.; the network structure of the human body detection model may also be a single-stage detection network structure, for example, the network structure of the human body detection model is RetinaNet.

It is understood that the image within the motion field may include one human body, or may include a plurality of human bodies; thus, a human body image or a plurality of human body images can be obtained by performing human body detection on the image of the motion field.

Step 103: aiming at each human body image in at least one human body image, identifying multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image; and determining the human identity information in the human image according to the obtained identification result.

Here, the various attributes include, but are not limited to, color, size, text, shape, etc. information. In practical application, each human body image can be identified by utilizing a pre-trained target attribute identification model to obtain an identification result of target attributes in the human body images; wherein the target attribute may be one of the above-mentioned plurality of attributes.

As an implementation manner, after the identification results corresponding to the multiple attributes in the human body image are obtained, the human body identity information in the human body image can be determined according to the predetermined corresponding relationship between the multiple attributes and the human body identity. In an actual scene, various corresponding attribute information can be prestored according to each human identity information, so that the corresponding relation between various attributes and the human identity is determined.

Step 104: and processing the images of the motion fields according to the human identity information of each human image in the images of each motion field.

In practical applications, the steps 101 to 104 may be implemented by a Processor in an electronic Device, where the Processor may be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor.

In one implementation, the plurality of attributes includes clothing color and text content; correspondingly, the identifying the multiple attributes of the human body image to obtain the identification results corresponding to the multiple attributes in the human body image may include: carrying out clothing color identification on the human body image to obtain clothing color information of the human body image; and performing text recognition on the human body image to obtain text information in the human body image.

After the clothing color information and the text information are obtained, the human body identity information in each human body image can be determined according to the clothing color information and the text information.

Here, the text content may be content with unique identification for distinguishing different person identities, and the text content may be numeric text, literal text, etc., where the literal text may be text such as name, code number, etc.

In a real scene, each human body image in the motion field can represent an image of at least one color, that is, color information of each human body image can be used for representing at least one color; illustratively, the color information of each human body image may represent a skin color, a hair color, a clothing color of the human body, wherein the clothing color may include one or more colors.

In the embodiment of the disclosure, after each human body image is obtained, each human body image can be processed to obtain a clothing region of each human body image; then, color information can be extracted from the clothing region of each human body image, and the color information of each pixel point in the clothing region of each human body image is clustered, so that the color information of each part in the clothing region of each human body image is determined. In one implementation, the above implementation process may be implemented by a pre-trained clothing recognition model, or other implementations may be adopted.

It will be appreciated that the clothing of persons in a sports arena will typically contain numerical text information such as numbers to indicate the identity of a person, although the clothing of persons in a sports arena may also contain other non-numerical text information such as names of persons, names of sponsors, etc.

The following exemplarily illustrates an implementation of text recognition for each human body image.

In a first implementation manner, text Recognition may be performed on each human body image by using an Optical Character Recognition (OCR) method to obtain text information in each human body image, and then, the text information may be extracted from the text information in each human body image. Here, OCR refers to a process of translating a shape into a computer text using a character recognition method; namely, the process of analyzing and processing the image file of the text data and obtaining the character and the layout information.

In a second implementation manner, a text recognition model trained in advance may be used to perform text recognition on each human body image, so as to obtain text information in each human body image. Illustratively, the text recognition model may be RCNN or the like.

The digital text information in each human body image can be used for representing the human body identity information, and the clothing color of the human body image can be used for distinguishing the category of the person in the sports field, so that the human body identity information in each human body image can be uniquely and accurately determined according to the clothing color information and the digital text information; illustratively, two parties of a match in a sports field are personnel in team D and personnel in team E, the clothing colors of the personnel in team D and the personnel in team E are respectively red and blue, the numbers of the personnel in team D are respectively 1 to 7, and the numbers of the personnel in team E are respectively 1 to 7, so that the category of a human body can be determined according to the clothing color information of each human body image; under the condition that the category of the human body is determined to be D team personnel, the human body identity information in the human body image can be uniquely determined according to the digital text information in the human body image, for example, the human body identity information of the human body image is No. 5 personnel in the D team personnel; and under the condition that the category to which the human body belongs is determined to be the personnel in team E, the human body identity information in the human body image can be uniquely determined by referring to the mode of determining the human body identity information of the personnel in team D.

As can be seen from the above description, the human bodies in each human body image can be automatically grouped according to the category to which the person belongs, based on the clothing color information of each human body image.

It can be seen that, in the embodiment of the present disclosure, the identification of the human identity can be performed based on the color and the number in the human image, and even when the image capture device is far away from the human body, the color and the number in the human image can be easily and accurately identified.

In the embodiment of the disclosure, after the at least one human body image is obtained, under the condition that the quality score of the first human body image in the at least one human body image is lower than the quality score threshold, the multiple attributes of the first human body image may be identified, so as to obtain the identification results corresponding to the multiple attributes in the first human body image. Wherein the first human body image represents one image of the at least one human body image.

For example, the garment color recognition may be performed on the first human body image and the text recognition may be performed on the first human body image in a case where the quality score of the first human body image is lower than the quality score threshold.

Here, the quality score of the first human body image may be represented by the number of pixels of the human body, the number of pixels in a partial region of the human body, or a distance from the human body to the image capturing device, and the quality score of the first human body image is positively correlated with the number of pixels of the human body or the number of pixels in a partial region of the human body and negatively correlated with the distance from the human body to the image capturing device.

It should be noted that, when the quality score of the first human body image is not lower than the quality score threshold, the human body identity of the human body image may be identified by a pedestrian re-identification method, a face identification method, or the like, so as to determine the human body identity information in the human body image.

In the embodiment of the disclosure, the quality scoring threshold may be set according to experience, or trial values may be performed on the quality scoring threshold, and for each trial value of the quality scoring threshold, under the condition that the quality score of the human body image for testing is not lower than the quality scoring threshold, it is determined whether the human body identity of the human body image for testing can be accurately identified by a pedestrian re-identification method, a face identification method, or the like, and then the quality scoring threshold is selected from the various values of the quality scoring threshold according to the determination result; here, the human identity information of each human body in the human body image for testing is known information so as to determine whether the human identity of the human body image for testing can be accurately identified.

It can be seen that, under the condition that the quality score of the image of the human body is lower than the quality score threshold value, the distance between the image acquisition equipment and the human body can be considered to be far, and for the condition, the human identity of the image of the human body can be accurately determined based on the colors and the numbers in the accurately identified image of the human body, so that the processing precision of the image of the sports field can be favorably improved.

Under the condition that the quality score of the image of the human body is not lower than the quality score threshold value, the image acquisition equipment can be considered to be close to the human body, and for the condition, the human body identity of the image of the human body can be accurately identified by adopting a pedestrian re-identification method, a face identification method and the like.

In an embodiment of the present disclosure, one implementation of determining that the quality score of the first human body image is lower than the quality score threshold may be: and under the condition that the number of the pixel points of the first target object in the first human body image is smaller than the set number threshold, determining that the quality score of the first human body image is lower than the quality score threshold.

Here, the first target object represents at least a part of a human body, for example, the first target object is a human body, a human face, a human hand, human clothing, or the like.

The following exemplarily explains a determination manner of the set number threshold.

In a first implementation manner, the set number threshold may be determined according to the quality score threshold, that is, after the quality score threshold is determined, an upper limit value of the number of pixel points of the first target object in the image of the human body is determined by taking that the quality score of the image of the human body is lower than the quality score threshold as a target, and then the set number threshold is determined, where the set number threshold is less than or equal to the upper limit value.

In a second implementation manner, trial value taking may be performed on the set number threshold, and for each trial value taking of the set number threshold, under the condition that the quality score of the human body image for testing is not lower than the quality score threshold, it is determined whether the human body identity of the human body image for testing can be accurately identified by a pedestrian re-identification method, a face identification method, or the like, and then the set number threshold is selected from the various values of the set number threshold according to the determination result.

It can be seen that, under the condition that the number of the pixel points of the first target object in the first human body image is smaller than the set number threshold, the distance between the image acquisition device and the human body can be considered to be far, and for the condition, the disclosed embodiment can accurately determine the human body identity of the first human body image based on the color and the number in the accurately identified first human body image, and further, is favorable for improving the processing precision of the image of the playground.

In an embodiment of the present disclosure, another implementation manner of determining that the quality score of the first human body image is lower than the quality score threshold value may adopt the following method:

acquiring depth information corresponding to a first human body image; determining the distance between the human body in the first human body image and image acquisition equipment for acquiring the first human body image according to the depth information corresponding to the first human body image; and determining that the quality score of the first human body image is lower than a quality score threshold value under the condition that the distance between the human body in the first human body image and the image acquisition equipment for acquiring the first human body image is greater than a set distance.

In practical application, the image acquisition equipment adopts a binocular stereo vision technology, so that the depth information of the image acquired by the image acquisition equipment can be determined based on the binocular stereo vision technology; the depth information of the image is used to represent distance information of each region in the image from the image capture device, and thus, the disclosed embodiments may determine the distance of the human body in the first human body image from the image capture device used to capture the first human body image.

It can be seen that, under the condition that the distance between the human body in the first human body image and the image acquisition device for acquiring the first human body image is greater than the set distance, the embodiment of the disclosure can accurately determine the human body identity of the first human body image based on the accurately identified color and number in the first human body image, and is further beneficial to improving the processing precision of the image of the sports field.

In the embodiment of the present disclosure, after obtaining at least one human body image, each human body image in the at least one human body image may be processed to obtain a processing result of each human body image;

accordingly, according to the human identity information of each human image in the images of the motion field, the implementation manner of processing the images of the motion field may be: and processing the images of the motion field according to the processing result of each human body image and the human body identity information in each human body image.

It can be seen that the embodiment of the present disclosure is favorable to accurately obtain the data related to the human body in the image of the motion field on the basis of comprehensively considering the processing result of each human body image and the human body identity information in each human body image, so as to be favorable to improving the processing precision of the image of the motion field.

In the embodiment of the present disclosure, the processing result of each human body image may include at least one of the following: position information of a human body in each human body image, and behavior information of the human body in each human body image. In practical applications, each human body image in the at least one human body image is processed to obtain a processing result of each human body image, and the processing result may include at least one of the following items:

analyzing the position of the human body of each human body image to obtain the position information of the human body in each human body image;

The implementation manner of performing behavior analysis on each human body image may be as follows: and detecting human body key points of each human body image, and determining the behavior information of the human body in each human body image according to the angles among the detected human body key points.

It can be seen that the image of the motion field can be processed according to at least one of the position information and the behavior information of the human body and the human body identity information in each human body image, so that the data related to the human body in the image of the motion field can be accurately obtained according to at least one of the position information and the behavior information of the human body.

In the embodiment of the present disclosure, according to the processing result of each human body image and the human body identity information in each human body image, one implementation manner of processing the image of the motion field may be: and obtaining motion data corresponding to the human body identity information in the images of the motion field according to the processing result of each human body image and the human body identity information in each human body image.

Here, the motion data may include a motion distance, a motion speed, motion region information; when the image of the motion field includes a plurality of frames of images, motion data corresponding to the human identity information in the image of the motion field can be obtained through statistics according to the processing result of each human image and the human identity information in each human image.

After the motion data are obtained, further data processing can be performed on the motion data corresponding to the human identity information in the image of the motion field, illustratively, whether a corresponding person completes a preset target task or not can be judged according to the motion data, and the motion data can also be visually presented; of course, the above description is only an exemplary illustration of the usage scenario of the motion data, and the embodiment of the present disclosure does not limit this.

It can be understood that the motion data corresponding to the human identity information in the image of the motion field is obtained, so that the analysis of the motion capability of each human body in the image of the motion field is facilitated, and the generation of reasonable motion suggestion information for the motion capability of each human body is facilitated.

In one implementation, the playing field is a football field; the processing result of each human body image comprises position information of the human body and behavior information of the human body; therefore, the motion data corresponding to the human body identity information in the images of the football field can be obtained according to the position information of the human body in each human body image, the behavior information of the human body and the human body identity information in each human body image.

It can be seen that the embodiment of the present disclosure can accurately analyze the motion data corresponding to the human identity information according to the position information of the human body, the behavior information of the human body, and the human identity information in the football field.

In the embodiment of the present disclosure, after the motion field image is acquired, the second target object in the motion field image may be detected to obtain the second target object in the motion field image; analyzing the second target object, and determining an analysis result of the second target object in the image of the motion field; the second target object comprises at least one of a moving object and a motion field key point;

correspondingly, according to the processing result of each human body image and the human body identity information in each human body image, another implementation manner of processing the image of the motion field may be as follows:

Here, the moving object may be a football, a basketball, a volleyball, a javelin, a shot, etc., and the playground key points may be set according to actual requirements, for example, in the case that the playground is a football field, the playground key points may be a vertex point, a center line point, a forbidden area point, a drawing point, etc.; the vertex points are used for representing the positions of the corner points of the football court, the central line points are used for representing the end points of the central line in the football court, the forbidden zone points are used for representing the forbidden zone boundary points in the football court, and drawing points are used for representing the intersection points of the shooting area boundary of the image acquisition equipment and the football court boundary line; the moving object and the motion field key point are not limited to those described above.

In practical application, a detection model of the second target object may be trained in advance, and the detection model of the second target object is used for detecting the second target object from the image; after the trained detection model of the second target object is obtained, the image of the motion field may be input to the detection model of the second target object, and the image of the motion field is processed by using the detection model of the second target object, so as to obtain a detection frame of the second target object in the image of the motion field, where the image in the detection frame of the second target object is the image of the second target object.

For example, in the case that the second target object includes a football, the image of the football field may be processed by using a football detection model trained in advance, so as to obtain a football image in the image of the football field; in the case that the second target object includes the key point of the football pitch, the image of the football pitch may be processed by using a pre-trained football pitch detection model, so as to obtain the key point of the football pitch in the image of the football pitch.

The network structure of the detection model of the second target object is not limited in the embodiments of the present disclosure, and the network structure of the detection model of the second target object may be a two-stage detection network structure, for example, the network structure of the detection model of the second target object is fast RCNN or the like; the network structure of the body detection model of the second target object may also be a single-stage detection network structure, for example, the network structure of the detection model of the second target object is RetinaNet or the like.

In practical application, the human body image meeting the preset condition can be extracted from the motion field image according to the processing result of each human body image, the analysis result of the second target object in the image of the motion field and the human body identity information in each human body image.

For example, in the case that the behavior of the human body image is a shooting behavior, a human body image which is shooting can be extracted from the plurality of football field images, wherein the human body image which is shooting is a human body image meeting preset conditions; for example, when the human body of the human body image is closer to the football, the human body image can be determined to meet the preset condition, and at the moment, the human body image is extracted from the multi-frame football field image; of course, the embodiment of the present disclosure may also extract the corresponding human body image from the multiple frames of soccer field images according to other strategies.

It is also possible to analyze the trajectory of a moving object, for example, based on the detection result of the moving object in the image of the motion field.

It can be seen that the embodiment of the present disclosure is favorable to accurately obtain data related to the second target object and the human body in the image of the motion field on the basis of comprehensively considering the processing result of each human body image, the analysis result of the second target object in the image of the motion field, and the human body identity information in each human body image, so as to be favorable to improving the processing precision of the image of the motion field.

The image processing method according to the embodiment of the present disclosure is further described below with application scenarios and drawings.

In this application scenario, the playing field is a 5-person football pitch, a 6-person football pitch, or a 7-person football pitch, illustratively, the size of the football pitch is 40m × 20m or 64m × 32 m; at least 4 cameras with 4K resolution ratios can be set for the football field and used for shooting images of the football field, and after video data collected by the cameras at the same time are obtained, the images collected by the cameras at the same time can be subjected to fusion processing to obtain the corresponding image of one frame of the football field. The first target object is a human face, and the second target object comprises a football and football field key points.

In the disclosed embodiment, at the beginning of a game in a football field, game related data may be presented on an interface of an electronic device, and referring to fig. 2A, the presentation of the game related data on the interface of the electronic device may include target tasks of persons in the sports field; referring to fig. 2B, the game related data presented on the interface of the electronic device may include people and formation information of both parties of the game within the sports stadium, where F and G indicate categories of both parties of the game in fig. 2B.

After the image of each frame of the football court in the video data is obtained, a human body image, a football image and key points of the football court can be detected from the image of the football court according to the content recorded in the embodiment; for example, a human detection box in a football field image is shown in fig. 3, and a football field key point in the football field image is shown in fig. 4.

Under the condition that the quality score of the human body image is not lower than the quality score threshold value, identifying human body identity information in the human body image by using a face identification method; under the condition that the quality score of the image of the human body is lower than the quality score threshold value, clothing color recognition and text recognition are carried out on the image of the human body, and clothing color information and digital text information in the image of the human body are obtained; and determining the human identity information in the human body image according to the clothing color information in the human body image and the digital text information.

After the human identity information in the human body image is obtained, the image of the motion field in the video data can be processed according to the human identity information of the human body image.

Exemplarily, performing behavior analysis on the human body image to obtain behavior information of a human body in the human body image; analyzing the human body position of the human body image to obtain the position information of the human body in the human body image; determining the football position in the football field image according to the football detection result in the football field image; then, determining the distance between the human body and the football in the football court image according to the position information of the human body in the football court image and the position of the football; and then can extract the wonderful highlights from the video data according to the behavior information of the human body and the distance between the human body and the football in the football court image, or can analyze the video data according to the behavior information of the human body and the distance between the human body and the football in the football court image, thereby obtaining the statistical information of the match or the strategic attack map data. For example, game statistics are shown in FIG. 5A and highlight highlights and strategy aggression map data are shown in FIG. 5B.

For example, the sports data corresponding to each human identity information in each frame of image of the football court of the video data can be determined according to the above description, and referring to fig. 2, fig. 6A and fig. 6B, the sports data of the person can be presented on the interface of the electronic device, so as to facilitate the football player and the coach to further analyze the sports data.

After the images of the football court are processed, the processing results of the images of the football court can be shared, wherein the processing results of the images of the football court comprise sports data of personnel, match statistical data, highlight highlights, strategic attack map data and the like; referring to fig. 7, a highlight collection and a sharing button are presented on an interface of the electronic device, and a user can share the highlight collection to other people by clicking the sharing button.

As can be seen from the above, the image processing method according to the embodiment of the present disclosure can be applied to a scene in which images of indoor and outdoor football fields are processed, and can realize functions such as football detection, football field key point detection, human body identification, highlights extraction, and the like, and can analyze football match data.

It will be understood by those skilled in the art that in the method of the present invention, the order of writing the steps does not imply a strict order of execution and any limitations on the implementation, and the specific order of execution of the steps should be determined by their function and possible inherent logic.

On the basis of the image processing method proposed by the foregoing embodiment, an embodiment of the present disclosure proposes an image processing apparatus.

Fig. 8 is a schematic diagram of a composition structure of an image processing apparatus according to an embodiment of the disclosure, and as shown in fig. 8, the apparatus may include:

an obtaining module 801, configured to obtain an image of a motion field;

a detection module 802, configured to perform human body detection on the image of the motion field to obtain at least one human body image;

a first processing module 803, configured to perform multiple attribute identification on the human body image for each human body image in the at least one human body image, so as to obtain identification results corresponding to the multiple attributes in the human body image; determining human body identity information in the human body image according to the obtained identification result;

a second processing module 804, configured to process the image of the motion field according to the human identity information of each human image in the images of the motion field.

In some embodiments, the plurality of attributes includes clothing color and text content; the first processing module 803 is specifically configured to:

In some embodiments, the first processing module 803 is specifically configured to:

In some embodiments, the first processing module 803 is further configured to:

In some embodiments, the second processing module 804 is further configured to:

the second processing module 804 is configured to process the image of the motion field according to the human identity information of each human image in the images of the motion field, and includes:

the second processing module 804 processes each human body image of the at least one human body image to obtain a processing result of each human body image, where the processing result includes at least one of the following items:

analyzing the human body position of each human body image to obtain the position information of the human body in each human body image; in some embodiments, the second processing module 804 is specifically configured to:

In some embodiments, the second processing module 804 is specifically configured to:

In some embodiments, the playing field is a football field; the processing result of each human body image comprises position information and behavior information of a human body;

the second processing module 804 is specifically configured to:

In some embodiments, the second processing module 804 is further configured to:

and extracting the human body image meeting the preset condition from the image of the motion field according to the processing result of each human body image, the analysis result of the second target object in the image of the motion field and the human body identity information in each human body image.

In practical applications, the obtaining module 801, the detecting module 802, the first processing module 803, and the second processing module 804 may all be implemented by a processor in a computer device, and the processor may be at least one of an ASIC, a DSP, a DSPD, a PLD, an FPGA, a CPU, a controller, a microcontroller, and a microprocessor.

In addition, each functional module in this embodiment may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware or a form of a software functional module.

Based on the understanding that the technical solution of the present embodiment essentially or a part contributing to the prior art, or all or part of the technical solution may be embodied in the form of a software product stored in a storage medium, and include several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a Processor (Processor) to execute all or part of the steps of the method of the present embodiment. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

Specifically, the computer program instructions corresponding to an image processing method in the present embodiment may be stored on a storage medium such as an optical disc, a hard disk, a usb disk, or the like, and when the computer program instructions corresponding to an image processing method in the storage medium are read or executed by an electronic device, any one of the image processing methods of the foregoing embodiments is implemented.

Based on the same technical concept of the foregoing embodiment, referring to fig. 9, it is shown that an electronic device 90 provided by the embodiment of the present disclosure may include: a memory 901 and a processor 902; wherein the content of the first and second substances,

the memory 901 is used for storing computer programs and data;

the processor 902 is configured to execute the computer program stored in the memory to implement any one of the image processing methods of the foregoing embodiments.

In practical applications, the memory 901 may be a volatile memory (RAM); or a non-volatile memory (non-volatile memory) such as a ROM, a flash memory (flash memory), a Hard Disk (Hard Disk Drive, HDD) or a Solid-State Drive (SSD); or a combination of the above types of memories and provides instructions and data to the processor 902.

The processor 902 may be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understood that the electronic devices for implementing the above-described processor functions may be other devices, and the embodiments of the present disclosure are not particularly limited.

The embodiment of the present disclosure further provides a computer program, which includes computer readable codes, and when the computer readable codes are run in an electronic device, a processor in the electronic device executes a method for implementing any one of the image processing methods.

In some embodiments, functions of or modules included in the apparatus provided in the embodiments of the present disclosure may be used to execute the method described in the above method embodiments, and specific implementation thereof may refer to the description of the above method embodiments, and for brevity, will not be described again here.

The foregoing description of the various embodiments is intended to highlight various differences between the embodiments, and the same or similar parts may be referred to each other, which are not repeated herein for brevity

The methods disclosed in the method embodiments provided by the present application can be combined arbitrarily without conflict to obtain new method embodiments.

Features disclosed in various product embodiments provided by the application can be combined arbitrarily to obtain new product embodiments without conflict.

The features disclosed in the various method or apparatus embodiments provided herein may be combined in any combination to arrive at new method or apparatus embodiments without conflict.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present disclosure may be embodied in the form of a software product, which is stored in a storage medium (e.g., ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (e.g., a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present disclosure.

While the embodiments of the present disclosure have been described in connection with the drawings, the present disclosure is not limited to the specific embodiments described above, which are intended to be illustrative rather than limiting, and it will be apparent to those of ordinary skill in the art in light of the present disclosure that many more modifications can be made without departing from the spirit of the disclosure and the scope of the appended claims.

Claims

1. An image processing method, characterized in that the method comprises:

acquiring an image of a motion field;

aiming at each human body image in the at least one human body image, identifying multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image respectively;

determining human body identity information in the human body image according to the obtained identification result;

2. The method of claim 1, wherein the plurality of attributes includes clothing color and text content; the identifying the multiple attributes of the human body image to obtain the identification results respectively corresponding to the multiple attributes in the human body image comprises the following steps:

carrying out clothing color identification on the human body image to obtain clothing color information of the human body image;

and performing text recognition on the human body image to obtain text information in the human body image.

3. The method according to claim 1 or 2, wherein the identifying, for each human body image of the at least one human body image, multiple attributes of the human body image to obtain identification results corresponding to the multiple attributes in the human body image respectively comprises:

4. The method of claim 3, further comprising:

5. The method of claim 4, wherein the first target object represents at least a portion of a human body.

6. The method according to any one of claims 3 to 5, further comprising:

7. The method according to any one of claims 1 to 6, further comprising:

8. The method according to claim 7, wherein the processing result of each human body image includes at least one of position information and behavior information of a human body in each human body image;

9. The method according to claim 7 or 8, wherein the processing the image of the motion field according to the processing result of each human body image and the human body identity information in each human body image comprises:

10. The method of claim 9, wherein the playing field is a football field; the processing result of each human body image comprises position information and behavior information of a human body;

11. The method according to claim 7 or 8, characterized in that the method further comprises:

12. The method according to claim 11, wherein the processing the image of the motion field according to the processing result of each human body image, the analysis result of the second target object in the image of the motion field, and the human body identity information in each human body image comprises:

13. An image processing apparatus, characterized in that the apparatus comprises:

the acquisition module is used for acquiring an image of a motion field;

14. An electronic device comprising a processor and a memory for storing a computer program operable on the processor; wherein the content of the first and second substances,

the processor is configured to run the computer program to perform the method of any one of claims 1 to 12.

15. A computer storage medium on which a computer program is stored, characterized in that the computer program realizes the method of any one of claims 1 to 12 when executed by a processor.