CN112801862B

CN112801862B - Image transformation-based data processing method, data processing device and storage medium

Info

Publication number: CN112801862B
Application number: CN202110137590.2A
Authority: CN
Inventors: 黄跃峰; 尹倩倩; 董亮; 廖超
Original assignee: Zoomlion Heavy Industry Science and Technology Co Ltd; Zhongke Yungu Technology Co Ltd
Current assignee: Zoomlion Heavy Industry Science and Technology Co Ltd; Zhongke Yungu Technology Co Ltd
Priority date: 2021-02-01
Filing date: 2021-02-01
Publication date: 2024-01-19
Anticipated expiration: 2041-02-01
Also published as: CN112801862A

Abstract

The invention discloses a data processing method, a data processing device and a storage medium based on image transformation, wherein the method comprises the following steps: identifying a target object in the image to be processed, and obtaining corresponding object identification information; performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and performing data augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information. On the basis of the traditional data augmentation method, the data in the image to be processed is further augmented by adopting a perspective transformation mode, so that the data volume can be effectively increased, the comprehensiveness of the data is improved, and meanwhile, the diversity of the data is increased, thereby improving the accuracy of the identification processing of the image and meeting the actual demands of users.

Description

Image transformation-based data processing method, data processing device and storage medium

Technical Field

The present invention relates to the field of image processing technology, and in particular, to a data processing method based on image transformation, a data processing device based on image transformation, and a computer readable storage medium.

Background

In the field of image processing technology, there are a variety of processing algorithms, including deep learning algorithms. In practical applications, the deep learning model is highly dependent on data, so if the data volume is insufficient or the data quality is insufficient, the due processing effect still cannot be realized even if the processing accuracy of the deep learning model is high. Therefore, in order to solve the above technical problems, technicians perform data augmentation on input data to achieve better learning and processing effects.

In the prior art, data is mainly amplified by the following two methods: firstly, oversampling is carried out on data, for example, repeated sampling is carried out on an object with smaller data quantity in an image so as to obtain more data proportion than a large object, so that the processing effect of the object with smaller data quantity in the image is improved to a certain extent; secondly, data of an object with small data volume in the image is amplified according to a common method in a data synthesis mode, for example, the data of the object with small data volume can be amplified through methods such as cutting, rotation, overturning, noise adding and the like, and then repeatedly stuck to the image of the original training data set for multiple times, so that further data amplification is realized.

However, in the practical application process, the first data augmentation method is simply repeated sampling, so that the diversity of data is insufficient, and the image processing method cannot be effectively promoted; the second data augmentation method is insufficient in terms of data authenticity, so that the actual image processing requirements still cannot be met.

Disclosure of Invention

In order to solve the technical problems that in the prior art, the accuracy of a processing method for objects with smaller data volume in an image is insufficient and the actual processing requirement cannot be met, the embodiment of the invention provides the data processing method based on image transformation, which is used for carrying out data augmentation on data in the image to be processed in a perspective transformation or liquefaction transformation mode, so that the diversity of the data is increased, the identification accuracy of the image to be processed is improved and the actual requirement is met.

In order to achieve the above object, an embodiment of the present invention provides a data processing method based on image transformation, the data processing method including: identifying a target object in the image to be processed, and obtaining corresponding object identification information; performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and performing data augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information.

Preferably, the identifying the target object in the image to be processed to obtain corresponding object identification information includes: performing image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed; obtaining an object labeling frame for labeling each target object; and taking the labeling frame as the object identification information.

Preferably, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

Preferably, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes: acquiring an image limiting frame in the image to be processed; determining an operation limiting frame in the image limiting frame, wherein the operation limiting frame comprises all object labeling frames; executing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated annotation frame; and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

Preferably, the method further comprises: after the image limiting frame in the image to be processed is acquired, judging whether a cutting instruction for the image limiting frame is acquired or not; under the condition that the clipping instruction is acquired, corresponding clipping operation is carried out on the image limiting frame, and a clipped image limiting frame is obtained; and determining the operation limiting frame in the cut-out image limiting frame.

Preferably, the determining an operation restriction frame in the image restriction frame includes: generating an envelope region based on each object labeling frame, wherein the envelope region is used for enveloping each object labeling frame in the envelope region along the outer edge of the object labeling frame; generating an operable region based on the envelope region and the image bounding box; a plurality of random points are acquired within the operable region, and the operation restriction frame is determined based on the random points.

Preferably, the acquiring four random points in the operable area includes: acquiring a preset length; generating a point generation area corresponding to each vertex of the image restriction frame in the operable area based on the preset length, wherein the point generation area is positioned in the image restriction frame; one of the random points is acquired within each of the point generation areas.

Preferably, the performing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain a post-operation labeling frame includes: establishing an association relationship between the random points and the vertexes; based on the association relation, the length of the image limiting frame is used as the length after operation, the width of the image limiting frame is used as the width after operation, the first perspective transformation operation is carried out on the operation standard frame, the operation standard frame is obtained, and the vertexes of the operation marking frame correspond to the vertexes of the image limiting frame one by one.

Preferably, the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes: converting the object labeling frame based on the first perspective matrix to obtain a processed labeling frame; adjusting the processed marking frame to obtain an adjusted marking frame; and taking the adjusted annotation frame as the first transformed information.

Preferably, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: acquiring preset liquefaction parameters; and performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameters to obtain first transformed information corresponding to the target object.

Preferably, the preset liquefaction parameter includes a preset liquefaction radius, the performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameter, to obtain first transformed information corresponding to the target object, including: randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and the connecting line between the first point and the second point is in the operable area; generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area; determining a liquefaction direction based on the first point and the second point; taking the second point as a point after the transformation of the first point, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region; and processing the object identification information based on the transformed region to obtain the first transformed information.

Preferably, the object identification information further includes object category and area information of each of the object annotation frames, and the data processing method further includes: classifying the object labeling frames based on the object categories to obtain classified object labeling frames; acquiring area statistical information of the classified object labeling frames in each classification based on the classified object labeling frames and the area information; and re-amplifying the processed information according to a preset equalization rule based on the area statistical information to obtain re-processed information.

Preferably, the re-amplifying the processed information according to a preset equalization rule based on the area statistical information to obtain re-processed information, including: acquiring the re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the re-augmentation times to obtain corresponding second transformed information; and re-amplifying the processed information based on the second transformed information to obtain re-processed information.

Preferably, the data processing method further includes: acquiring a preset image recognition model; training the preset image recognition model based on the processed information to obtain a trained image recognition model.

Correspondingly, the embodiment of the invention also provides a data processing device based on image transformation, which comprises: the target identification module is used for identifying a target object in the image to be processed and acquiring corresponding object identification information; the transformation module is used for executing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and the data processing module is used for performing digital augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information.

Preferably, the object recognition module includes: the object identification unit is used for carrying out image identification on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed; the marking unit is used for obtaining an object marking frame for marking each target object; and the target determining unit is used for taking the labeling frame as the object identification information.

Preferably, the transformation module comprises a perspective transformation unit for: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

Preferably, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes: acquiring an image limiting frame in the image to be processed; determining an operation limiting frame in the image limiting frame, wherein the operation limiting frame comprises all object labeling frames; performing a first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated annotation frame; and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

Preferably, the perspective transformation unit is further configured to: after the image limiting frame in the image to be processed is acquired, judging whether a cutting instruction for the image limiting frame is acquired or not; under the condition that the clipping instruction is acquired, corresponding clipping operation is carried out on the image limiting frame, and a clipped image limiting frame is obtained; and determining the operation limiting frame in the cut-out image limiting frame.

Preferably, the transformation module comprises a liquefaction transformation unit for: acquiring preset liquefaction parameters; and performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameters to obtain first transformed information corresponding to the target object.

Preferably, the object identification information further includes object category and area information of each of the object annotation frames, and the data processing apparatus further includes a reprocessing module including: the classifying unit is used for classifying the object labeling frames based on the object categories to obtain classified object labeling frames; the statistical unit is used for obtaining the area statistical information of the classified object labeling frame in each classification based on the classified object labeling frame and the area information; and the reprocessing unit is used for carrying out re-augmentation processing on the processed information according to a preset equalization rule based on the area statistical information to obtain reprocessed information.

Preferably, the reprocessing unit is configured to: acquiring the re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the re-augmentation times to obtain corresponding second transformed information; and re-amplifying the processed information based on the second transformed information to obtain re-processed information.

Preferably, the data processing apparatus further comprises a model training module, the model training module comprising: the model acquisition unit is used for acquiring a preset image recognition model; the training unit is used for training the preset image recognition model based on the processed data to obtain a trained image recognition model.

On the other hand, the embodiment of the present invention further provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the data processing method provided by the embodiment of the present invention.

Through the technical scheme provided by the invention, the invention has at least the following technical effects:

by further data augmentation of the data in the image to be processed by adopting a perspective transformation and/or liquefaction transformation mode on the basis of the traditional data augmentation method, the data volume can be effectively increased, the comprehensiveness of the data is improved, and meanwhile, the diversity of the data is increased, so that the accuracy of the identification processing of the image to be processed is effectively improved, and the actual demands of users are met.

Additional features and advantages of embodiments of the invention will be set forth in the detailed description which follows.

Drawings

The accompanying drawings are included to provide a further understanding of embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain, without limitation, the embodiments of the invention. In the drawings:

FIG. 1 is a flowchart of a specific implementation of a data processing method based on image transformation according to an embodiment of the present invention;

FIG. 2 is a flowchart of a specific implementation of identifying a target object in an image transformation-based data processing method according to an embodiment of the present invention;

FIG. 3 is a flowchart of a specific implementation of performing a first perspective transformation on object identification information in an image transformation-based data processing method according to an embodiment of the present invention;

FIG. 4 is a schematic diagram of determining an operation restriction frame in an image transformation-based data processing method according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of determining an operation restriction frame within an image restriction frame in a data processing method based on image transformation according to another embodiment of the present invention;

FIG. 6 is a schematic diagram of acquiring random points in an operable area in an image transformation-based data processing method according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of performing a first perspective transformation on object identification information in an image transformation-based data processing method according to an embodiment of the present invention;

FIG. 8 is a flowchart of a specific implementation of converting object identification information in the image transformation-based data processing method according to the embodiment of the present invention;

fig. 9 is a schematic diagram of liquefying object identification information in the data processing method based on image transformation according to the embodiment of the present invention;

fig. 10 is a schematic structural diagram of a data processing apparatus based on image transformation according to an embodiment of the present invention.

Detailed Description

In order to solve the technical problems that in the prior art, the accuracy of a processing method for objects with smaller data volume in an image is insufficient and the actual processing requirements cannot be met, the embodiment of the invention provides the data processing method based on image transformation, which is used for increasing the data in the image to be processed by adopting perspective transformation and liquefaction transformation, so that the diversity of the data is increased, the identification accuracy of the image to be processed is improved and the actual requirements are met.

The following describes the detailed implementation of the embodiments of the present invention with reference to the drawings. It should be understood that the detailed description and specific examples, while indicating and illustrating the invention, are not intended to limit the invention.

The terms "system" and "network" in embodiments of the invention may be used interchangeably. "plurality" means two or more, and "plurality" may also be understood as "at least two" in this embodiment of the present invention. "and/or", describes an association relationship of an association object, and indicates that there may be three relationships, for example, a and/or B, and may indicate: a exists alone, A and B exist together, and B exists alone. The character "/", unless otherwise specified, generally indicates that the associated object is an "or" relationship. In addition, it should be understood that in the description of embodiments of the present invention, the words "first," "second," and the like are used merely for distinguishing between the descriptions and not be construed as indicating or implying a relative importance or order.

Referring to fig. 1, an embodiment of the present invention provides a data processing method based on image transformation, where the data processing method includes:

s10) identifying a target object in the image to be processed to obtain corresponding object identification information;

s20) performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object;

S30) performing data augmentation processing on the target object based on the object identification information and the first transformed information, obtaining processed information.

In a possible implementation manner, the scale of a company is faster, so personnel are increased faster, in order to better monitor the working condition of staff in the office environment, image monitoring is performed on the office environment, and in order to better identify a target object in a monitored image, for example, a mobile phone in the monitored image, the image data is processed by adopting the data processing method based on image transformation provided by the embodiment of the invention.

In the application process, firstly, an image to be processed is obtained, and a target object in the image to be processed is identified, for example, in the embodiment of the invention, the target object such as a mobile phone, a head portrait and a computer screen in a monitoring image is identified, and at least one object identification information is obtained, for example, the object identification information is the head portrait identification information and the computer screen identification information of the mobile phone identification information. Since the data acquired by directly identifying is less, especially for the object with smaller object outline, the first transformation operation is performed on the object identification information, for example, in the embodiment of the invention, the position of the camera is fixed, so that the object in the monitoring image is far, small, near and large, and therefore, the first perspective transformation operation can be performed on the monitoring image, so that the far and near proportion transformation in the monitoring image is consistent, and the transformed image is obtained.

In this case, the data augmentation operation is performed on the target object based on the object identification information and the first transformed information, and for example, the first transformed information and the object identification information may be superimposed as processed information, thereby achieving the data augmentation of the monitoring image.

In the embodiment of the invention, the data in the image to be processed is amplified by adopting an image transformation method on the basis of original images, so that the data volume of the data is improved on the basis of ensuring the authenticity of the data, and meanwhile, the diversity of the data is also improved, thereby being beneficial to the later study of the data, improving the accuracy of identifying the target object in the image to be processed and meeting the actual demands.

Referring to fig. 2, in the embodiment of the present invention, the identifying the target object in the image to be processed to obtain corresponding object identification information includes:

s101) carrying out image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed;

s102) obtaining an object labeling frame for labeling each target object;

s103) using the label frame as the object identification information.

In a possible implementation manner, in a process of identifying a target object in an image to be processed, firstly, image identification is performed on the image to be processed according to a preset algorithm, for example, in the embodiment of the present invention, in order to improve compatibility and an applicable range of a data processing method provided by the present invention, a common target detection processing algorithm is used to perform image identification on the image to be processed, for example, a user may select the target object to be identified according to an actual requirement, and identify at least one target object in the image to be processed. At this time, an object labeling frame for labeling each target object is obtained, for example, in a target detection processing algorithm, a labeling frame for each human body is generated after identifying the human body in the monitoring image, the object class of each target object can be obtained, for example, whether each target object is a human body, a pet, a building or the like can be identified, and then the object labeling frame and the object class are used as object identification information.

In an embodiment of the present invention, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

Referring to fig. 3, in an embodiment of the present invention, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes:

s201), acquiring an image limiting frame in the image to be processed;

s202) determining an operation limiting frame in the image limiting frame, wherein the operation limiting frame comprises all object labeling frames;

s203) executing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated annotation frame;

s204) generating a corresponding first perspective matrix based on the operation restriction frame and the post-operation annotation frame.

In an embodiment of the present invention, the method further includes: after the image limiting frame in the image to be processed is acquired, judging whether a cutting instruction for the image limiting frame is acquired or not; under the condition that the clipping instruction is acquired, corresponding clipping operation is carried out on the image limiting frame, and a clipped image limiting frame is obtained; and determining the operation limiting frame in the cut-out image limiting frame.

In an embodiment of the present invention, the determining an operation restriction frame in the image restriction frame includes: generating an envelope region based on each object labeling frame, wherein the envelope region is used for enveloping each object labeling frame in the envelope region along the outer edge of the object labeling frame; generating an operable region based on the envelope region and the image bounding box; a plurality of random points are acquired within the operable region, and the operation restriction frame is determined based on the random points.

Further, in an embodiment of the present invention, the obtaining four random points in the operable area includes: acquiring a preset length; generating a point generation area corresponding to each vertex of the image restriction frame in the operable area based on the preset length, wherein the point generation area is positioned in the image restriction frame; one of the random points is acquired within each of the point generation areas.

Referring to fig. 4, in one possible implementation manner, after the object identification information in the image to be processed is acquired, an image limiting frame is first acquired in the image to be processed, for example, in an embodiment of the present invention, the image limiting frame may be manually determined by an operator according to a monitoring area actually required to be monitored in a current monitoring field of view, and four points (for example, A, B, C, D points) are input in the monitoring area to determine an image limiting frame, where an operation limiting frame is further determined in the image limiting frame.

In another possible implementation manner, in order to reduce the data processing amount when processing an image, reduce the complexity of image processing, improve the accuracy of image processing, further determine whether to acquire a clipping instruction for an image limiting frame after acquiring the image limiting frame in the image to be processed, for example, in an embodiment of the present invention, please refer to fig. 5, a user may designate a clipping area in the image limiting frame, for example, draw a clipping frame, where the clipping area includes an area that is a clipping area, and click a corresponding clipping button to issue a clipping instruction, at this time, perform a clipping operation on the image limiting frame, acquire a clipped image limiting frame, and further determine an operation limiting frame in the clipped image limiting frame.

In the embodiment of the invention, each object labeling frame in the image limiting frame or the cut image limiting frame can be enveloped according to a preset enveloping algorithm to generate an enveloping area, and then an operable area is generated according to the enveloping area and the image limiting frame, for example, the area between the enveloping area and the image limiting frame is used as the operable area. Then, a plurality of random points are randomly acquired in the operability area, and an operation restriction frame is determined based on the plurality of random points, for example, in the embodiment of the present invention, four points (for example, H, I, J, K points) are randomly determined in the operability area of the image restriction frame, thereby determining the operation restriction frame. In another embodiment of the present invention, 2 points (e.g., H, I points) are randomly determined within the operable area of the cropped image bounding box, thereby determining the operation bounding box (H, I, J, K).

In order to avoid that the operation limiting frame generated by randomly determined points causes excessive distortion of the target object after perspective transformation, please refer to fig. 6, in the embodiment of the present invention, a preset length is further obtained, and a point generating area corresponding to each vertex of the image limiting frame is generated in the operable area based on the preset length, for example, in the embodiment of the present invention, a rectangular frame is generated in the image limiting frame with each vertex as an origin, the preset length is taken as a side length, the rectangular frame is taken as a point generating area, random points are randomly generated in each point generating area, and the corresponding operation limiting frame is further determined.

In an embodiment of the present invention, the performing, based on the image restriction frame, the first perspective transformation operation on the operation restriction frame to obtain an operated annotation frame includes: establishing an association relationship between the random points and the vertexes; based on the association relation, the length of the image limiting frame is used as the length after operation, the width of the image limiting frame is used as the width after operation, the first perspective transformation operation is carried out on the operation standard frame, the operation standard frame is obtained, and the vertexes of the operation marking frame correspond to the vertexes of the image limiting frame one by one.

Referring to fig. 7, in one possible implementation, after the operation restriction frame is acquired, a first perspective transformation operation is performed on the operation restriction frame according to the image restriction frame to obtain a post-operation labeling frame. First, an association relationship between a random point and a vertex of an image limiting frame is established, for example, in the embodiment of the invention, a point A is associated with a point H, a point B is associated with a point I, a point C is associated with a point J, a point D is associated with a point K, and then a first perspective transformation operation is performed on the operation limiting frame according to the length and width of the preset standard frame, namely, the operation limiting frame is transformed into a labeling frame with the same size as the image limiting frame, so that a post-operation labeling frame (for example, consisting of points H ', I', J 'and K') is obtained. At this time, the corresponding first perspective matrix may be obtained according to the operation limiting frame and the post-operation labeling frame, for example, in the embodiment of the present invention, the first perspective matrix may be generated according to the mapping relationship between the corresponding points (for example, the corresponding H point and the corresponding H' point) in the operation limiting frame and the post-operation labeling frame.

In the embodiment of the invention, the perspective phenomenon in the image to be processed is eliminated by carrying out perspective transformation on the image to be processed, so that on one hand, the problem of poor monitoring effect on a far distance caused by the perspective effect of the monitoring area in the process of monitoring the monitoring area by adopting the fixed-position camera can be effectively solved, and the monitoring area with the same monitoring effect on the far and near distances of the camera is ensured; on the other hand, in the subsequent data processing process, the deep learning processing can be performed by combining the original data and the data after perspective transformation, so that more comprehensive and diversified learning data are obtained, the learning efficiency and the learning accuracy can be effectively improved, and the processing accuracy of image processing is improved.

Referring to fig. 8, in an embodiment of the present invention, the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes:

s301) converting the object labeling frame based on the first perspective matrix to obtain a processed labeling frame;

s302) adjusting the processed marking frame to obtain an adjusted marking frame;

s303) using the adjusted annotation box as the first transformed information.

In a possible embodiment, after the first perspective matrix is acquired, it is determined that a transform coefficient for processing the image to be processed into a general tile image is obtained. In this case, the object labeling frame is subjected to corresponding transformation processing, for example, each point on the object labeling frame is multiplied by the first perspective matrix, so as to obtain a point after multiplication, and a post-processing labeling frame is determined according to the point after multiplication.

In an embodiment of the present invention, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: acquiring preset liquefaction parameters; and performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameters to obtain first transformed information corresponding to the target object.

Further, in an embodiment of the present invention, the preset liquefaction parameter includes a preset liquefaction radius, and the performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameter, to obtain first transformed information corresponding to the target object includes: randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and the connecting line between the first point and the second point is in the operable area; generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area; determining a liquefaction direction based on the first point and the second point; taking the second point as a point after the transformation of the first point, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region; and processing the object identification information based on the transformed region to obtain the first transformed information.

Referring to fig. 9, in one possible implementation manner, a preset liquefaction area where the liquefaction operation is performed is first obtained, for example, in an embodiment of the present invention, a preset liquefaction radius (for example, R) may be first set, and then a first point (for example, point C) and a second point (for example, point M) are randomly obtained within the determined operable area, where a distance between the first point and the second point is greater than the preset liquefaction radius, and a connection line between the first point and the second point is located within the operable area. At this time, a preset liquefaction area is generated according to the point C and the preset liquefaction radius R, for example, the preset liquefaction area is generated with the point C as the center of a circle and the radius R as the radius, and the preset liquefaction area is located in the operable area. At this time, the liquefaction direction is determined according to the first point and the second point, for example, the liquefaction direction is determined with the point C as a starting point and the point M as an ending point, then the second point (i.e., the point M) is used as a point after the transformation of the point C, and the first liquefaction transformation operation is performed on the preset liquefaction region according to the liquefaction direction to obtain a transformed region, for example, in the embodiment of the present invention, the point u in the preset liquefaction region becomes the point x after transformation, at this time, the background image near the object identification information is changed, at this time, the object identification information is processed based on the transformed region, for example, the transformed region is combined with the object identification information to generate the first transformed information.

In the embodiment of the invention, the data augmentation operation is carried out on the target object by adopting the liquefaction conversion mode, so that more forms of image data can be generated on the target object on the basis of the image data of the real target object, the data volume of the target object can be effectively increased, and the increased data volume has high authenticity.

In an embodiment of the present invention, the object identification information further includes object category and area information of each of the object annotation frames, and the data processing method further includes: classifying the object labeling frames based on the object categories to obtain classified object labeling frames; acquiring area statistical information of the classified object labeling frames in each classification based on the classified object labeling frames and the area information; and re-amplifying the processed information according to a preset equalization rule based on the area statistical information to obtain re-processed information.

Further, in an embodiment of the present invention, the re-amplifying the processed information according to a preset equalization rule based on the area statistics information to obtain re-processed information includes: acquiring the re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the re-augmentation times to obtain corresponding second transformed information; and re-amplifying the processed information based on the second transformed information to obtain re-processed information.

In the practical application process, since the object located far away in the image to be processed before processing becomes very small due to the perspective effect, the data of the object obtained in the process of identifying or processing the object is less, so that the accuracy of identifying or processing the object can be reduced. In order to solve the above technical problem, in one possible implementation manner, after the preliminary data processing is performed on the image to be processed, the data re-amplification is further performed on the target object in the image to be processed.

For example, in the embodiment of the present invention, the object labeling frames are classified according to the identified object types, so as to obtain classified object labeling frames under each type, and then the area statistics information of each classified object labeling frame is obtained according to the area information of each object labeling frame, for example, the area distribution information of the classified object labeling frames under each type is obtained, and at this time, the processed information is processed according to a preset equalization rule. For example, in the embodiment of the present invention, the preset equalization rule is to equalize the area ratio of the target object under the same category, for example, the total area of the target object in the mobile phone category is 30, where the area of the mobile phone a is 15, the area of the mobile phone B is 11, and the area of the mobile phone C is 4, so that the number of augmentation times of the mobile phone a is 1, the number of augmentation times of the mobile phone B is 3, and the number of augmentation times of the mobile phone C is 8 according to the above area distribution information. And then carrying out second perspective transformation operation on the object identification information corresponding to each mobile phone according to the augmentation times, obtaining a corresponding second perspective matrix, further obtaining corresponding second converted identification information according to the second perspective matrix, and superposing the second converted identification information to the processed information, thereby realizing the re-augmentation operation on the mobile phone data and obtaining the corresponding re-processed information.

According to the embodiment of the invention, the object with small data size is repeatedly processed for many times on the basis of perspective transformation operation, and the data processed for many times are overlapped to obtain larger data size, so that the reliability and the accuracy of the data can be effectively improved, and the processing accuracy of the image to be processed is improved.

In an embodiment of the present invention, the data processing method further includes: acquiring a preset image recognition model; training the preset image recognition model based on the processed information to obtain a trained image recognition model.

In a possible implementation manner, after the image data after the augmentation process is acquired, a preset image recognition model is further acquired, for example, in an embodiment of the present invention, the preset image recognition model may be a deep learning model, and the deep learning model is trained based on the processed information, so as to obtain a trained image recognition model. The image to be processed is identified through the trained image identification model, so that more accurate and reliable identification or processing results can be obtained, and the actual requirements of users are met.

A perspective transformation-based data processing apparatus provided by an embodiment of the present invention is described below with reference to the accompanying drawings.

Referring to fig. 10, based on the same inventive concept, an embodiment of the present invention provides a perspective transformation-based data processing apparatus, including: the target identification module is used for identifying a target object in the image to be processed and acquiring corresponding object identification information; the transformation module is used for executing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and the data processing module is used for performing digital augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information.

In an embodiment of the present invention, the object recognition module includes: the object identification unit is used for carrying out image identification on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed; the marking unit is used for obtaining an object marking frame for marking each target object; and the target determining unit is used for taking the labeling frame as the object identification information.

In an embodiment of the invention, the transformation module comprises a perspective transformation unit for: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

In an embodiment of the present invention, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes: acquiring an image limiting frame in the image to be processed; determining an operation limiting frame in the image limiting frame, wherein the operation limiting frame comprises all object labeling frames; performing a first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated annotation frame; and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

In an embodiment of the invention, the perspective transformation unit is further configured to: after the image limiting frame in the image to be processed is acquired, judging whether a cutting instruction for the image limiting frame is acquired or not; under the condition that the clipping instruction is acquired, corresponding clipping operation is carried out on the image limiting frame, and a clipped image limiting frame is obtained; and determining the operation limiting frame in the cut-out image limiting frame.

In the embodiment of the present invention, the acquiring four random points in the operable area includes: acquiring a preset length; generating a point generation area corresponding to each vertex of the image restriction frame in the operable area based on the preset length, wherein the point generation area is positioned in the image restriction frame; one of the random points is acquired within each of the point generation areas.

In an embodiment of the present invention, the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes: converting the object labeling frame based on the first perspective matrix to obtain a processed labeling frame; adjusting the processed marking frame to obtain an adjusted marking frame; and taking the adjusted annotation frame as the first transformed information.

In an embodiment of the present invention, the transformation module includes a liquefaction transformation unit for: acquiring preset liquefaction parameters; and performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameters to obtain first transformed information corresponding to the target object.

In an embodiment of the present invention, the preset liquefaction parameter includes a preset liquefaction radius, and the performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameter to obtain first transformed information corresponding to the target object includes: randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and the connecting line between the first point and the second point is in the operable area; generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area; determining a liquefaction direction based on the first point and the second point; taking the second point as a point after the transformation of the first point, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region; and processing the object identification information based on the transformed region to obtain the first transformed information.

In an embodiment of the present invention, the object identification information further includes object category and area information of each of the object annotation frames, and the data processing apparatus further includes a reprocessing module, where the reprocessing module includes: the classifying unit is used for classifying the object labeling frames based on the object categories to obtain classified object labeling frames; the statistical unit is used for obtaining the area statistical information of the classified object labeling frame in each classification based on the classified object labeling frame and the area information; and the reprocessing unit is used for carrying out re-augmentation processing on the processed information according to a preset equalization rule based on the area statistical information to obtain reprocessed information.

In an embodiment of the present invention, the reprocessing unit is configured to: acquiring the re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the re-augmentation times to obtain corresponding second transformed information; and re-amplifying the processed information based on the second transformed information to obtain re-processed information.

In an embodiment of the present invention, the data processing apparatus further includes a model training module, where the model training module includes: the model acquisition unit is used for acquiring a preset image recognition model; the training unit is used for training the preset image recognition model based on the processed data to obtain a trained image recognition model.

Further, the embodiment of the present invention also provides a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the data processing method according to the embodiment of the present invention.

The foregoing details of the optional implementation of the embodiment of the present invention have been described in detail with reference to the accompanying drawings, but the embodiment of the present invention is not limited to the specific details of the foregoing implementation, and various simple modifications may be made to the technical solution of the embodiment of the present invention within the scope of the technical concept of the embodiment of the present invention, and these simple modifications all fall within the protection scope of the embodiment of the present invention.

In addition, the specific features described in the above embodiments may be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, various possible combinations of embodiments of the present invention are not described in detail.

Those skilled in the art will appreciate that all or part of the steps in implementing the methods of the embodiments described above may be implemented by a program stored in a storage medium, including instructions for causing a single-chip microcomputer, chip or processor (processor) to perform all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

In addition, any combination of various embodiments of the present invention may be performed, so long as the concept of the embodiments of the present invention is not violated, and the disclosure of the embodiments of the present invention should also be considered.

Claims

1. A data processing method based on image transformation, the data processing method comprising:

identifying a target object in the image to be processed, and obtaining corresponding object identification information;

performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object;

Performing data augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information;

the identifying the target object in the image to be processed to obtain corresponding object identification information comprises the following steps:

performing image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed;

obtaining an object labeling frame for labeling each target object;

taking the marking frame as the object identification information;

the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes:

performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object;

converting the object identification information based on the first perspective matrix to obtain first converted information;

the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes:

acquiring an image limiting frame in the image to be processed;

Determining an operation limiting frame in the image limiting frame, wherein the operation limiting frame comprises all object labeling frames;

executing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated annotation frame;

generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame;

the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes: converting the object labeling frame based on the first perspective matrix to obtain a processed labeling frame; adjusting the processed marking frame to obtain an adjusted marking frame; taking the adjusted annotation frame as the first transformed information;

the object identification information further comprises object category and area information of each object labeling frame, and the data processing method further comprises the following steps:

classifying the object labeling frames based on the object categories to obtain classified object labeling frames;

acquiring area statistical information of the classified object labeling frames in each classification based on the classified object labeling frames and the area information;

And re-amplifying the processed information according to a preset equalization rule based on the area statistical information to obtain re-processed information.

2. The data processing method of claim 1, wherein the method further comprises:

after the image limiting frame in the image to be processed is acquired, judging whether a cutting instruction for the image limiting frame is acquired or not;

under the condition that the clipping instruction is acquired, corresponding clipping operation is carried out on the image limiting frame, and a clipped image limiting frame is obtained;

and determining the operation limiting frame in the cut-out image limiting frame.

3. The data processing method according to claim 1, wherein the determining an operation restriction frame within the image restriction frame includes:

generating an envelope region based on each object labeling frame, wherein the envelope region is used for enveloping each object labeling frame in the envelope region along the outer edge of the object labeling frame;

generating an operable region based on the envelope region and the image bounding box;

a plurality of random points are acquired within the operable region, and the operation restriction frame is determined based on the random points.

4. A data processing method according to claim 3, wherein obtaining four random points within the operable area comprises:

acquiring a preset length;

generating a point generation area corresponding to each vertex of the image restriction frame in the operable area based on the preset length, wherein the point generation area is positioned in the image restriction frame;

one of the random points is acquired within each of the point generation areas.

5. The method according to claim 4, wherein the performing the first perspective transformation operation on the operation restriction frame based on the image restriction frame to obtain a post-operation annotation frame includes:

establishing an association relationship between the random points and the vertexes;

based on the association relation, the length of the image limiting frame is used as the length after operation, the width of the image limiting frame is used as the width after operation, the first perspective transformation operation is carried out on the operation standard frame, the operation standard frame is obtained, and the vertexes of the operation marking frame correspond to the vertexes of the image limiting frame one by one.

6. A data processing method according to claim 3, wherein said performing a first transformation operation on said object identification information to obtain first transformed information corresponding to said target object comprises:

Acquiring preset liquefaction parameters;

and performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameters to obtain first transformed information corresponding to the target object.

7. The data processing method according to claim 6, wherein the preset liquefaction parameter includes a preset liquefaction radius, the performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameter to obtain first transformed information corresponding to the target object, comprising:

randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and the connecting line between the first point and the second point is in the operable area;

generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area;

determining a liquefaction direction based on the first point and the second point;

taking the second point as a point after the transformation of the first point, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region;

And processing the object identification information based on the transformed region to obtain the first transformed information.

8. The data processing method according to claim 1, wherein the re-augmenting the processed information based on the area statistics according to a preset equalization rule to obtain re-processed information comprises:

acquiring the re-amplification times of each target object according to the preset equalization rule based on the area statistical information;

performing a second transformation operation on the corresponding object identification information based on the re-augmentation times to obtain corresponding second transformed information;

and re-amplifying the processed information based on the second transformed information to obtain re-processed information.

9. The data processing method according to claim 1, characterized in that the data processing method further comprises:

acquiring a preset image recognition model;

training the preset image recognition model based on the processed information to obtain a trained image recognition model.

10. A data processing apparatus based on image transformation, the data processing apparatus comprising:

The target identification module is used for identifying a target object in the image to be processed and acquiring corresponding object identification information;

the transformation module is used for executing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object;

the data processing module is used for performing digital augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information;

the object recognition module includes:

the object identification unit is used for carrying out image identification on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed;

the marking unit is used for obtaining an object marking frame for marking each target object;

the target determining unit is used for taking the labeling frame as the object identification information;

the transformation module comprises a perspective transformation unit for:

acquiring an image limiting frame in the image to be processed;

performing a first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated annotation frame;

the object identification information further includes object category and area information of each of the object annotation frames, and the data processing apparatus further includes a reprocessing module including: the classifying unit is used for classifying the object labeling frames based on the object categories to obtain classified object labeling frames; the statistical unit is used for obtaining the area statistical information of the classified object labeling frame in each classification based on the classified object labeling frame and the area information; and the reprocessing unit is used for carrying out re-augmentation processing on the processed information according to a preset equalization rule based on the area statistical information to obtain reprocessed information.

11. The data processing apparatus of claim 10, wherein the perspective transformation unit is further configured to:

12. The data processing apparatus of claim 10, wherein the determining an operation restriction frame within the image restriction frame comprises:

13. The data processing apparatus of claim 12, wherein obtaining four random points within the operable region comprises:

Acquiring a preset length;

one of the random points is acquired within each of the point generation areas.

14. The data processing apparatus of claim 13, wherein the performing the first perspective transformation operation on the operation restriction frame based on the image restriction frame to obtain a post-operation annotation frame comprises:

15. The data processing apparatus of claim 12, wherein the transformation module comprises a liquefaction transformation unit to:

acquiring preset liquefaction parameters;

16. The data processing apparatus according to claim 15, wherein the preset liquefaction parameter includes a preset liquefaction radius, the performing a first liquefaction transformation operation on the object identification information based on the preset liquefaction parameter to obtain first transformed information corresponding to the target object, comprising:

17. The data processing apparatus according to claim 16, wherein the reprocessing unit is configured to:

18. The data processing apparatus of claim 10, further comprising a model training module, the model training module comprising:

the model acquisition unit is used for acquiring a preset image recognition model;

the training unit is used for training the preset image recognition model based on the processed data to obtain a trained image recognition model.

19. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the data processing method of any of claims 1-9.