CN112801862A

CN112801862A - Data processing method, data processing device and storage medium based on image transformation

Info

Publication number: CN112801862A
Application number: CN202110137590.2A
Authority: CN
Inventors: 黄跃峰; 尹倩倩; 董亮; 廖超
Original assignee: Zoomlion Heavy Industry Science and Technology Co Ltd; Zhongke Yungu Technology Co Ltd
Current assignee: Zoomlion Heavy Industry Science and Technology Co Ltd; Zhongke Yungu Technology Co Ltd
Priority date: 2021-02-01
Filing date: 2021-02-01
Publication date: 2021-05-14
Anticipated expiration: 2041-02-01
Also published as: CN112801862B

Abstract

The invention discloses a data processing method, a data processing device and a storage medium based on image transformation, wherein the method comprises the following steps: identifying a target object in an image to be processed, and acquiring corresponding object identification information; performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; data augmentation processing is performed on the target object based on the object identification information and the first converted information, and processed information is obtained. On the basis of the traditional data augmentation method, the data in the image to be processed is further augmented in a perspective transformation mode, so that the data volume can be effectively increased, the comprehensiveness of the data is improved, and the diversity of the data is increased on the basis of considering the authenticity of the data, so that the accuracy of the identification processing of the image is improved, and the actual requirements of users are met.

Description

Data processing method, data processing device and storage medium based on image transformation

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to a data processing method based on image transformation, a data processing apparatus based on image transformation, and a computer-readable storage medium.

Background

In the field of image processing technology, there are a variety of processing algorithms, including deep learning algorithms. In practical applications, the deep learning model is highly dependent on data, and therefore if the data amount is insufficient or the data quality is insufficient, even if the processing accuracy of the deep learning model is high, the processing effect that should be achieved still cannot be achieved. Therefore, in order to solve the above technical problems, technicians have better learning and processing effects by performing data amplification on input data.

In the prior art, data is augmented mainly by the following two methods: firstly, data is oversampled, for example, an object with a small data volume in an image is repeatedly sampled to obtain a data proportion larger than that of a large object, so that the processing effect of the object with the small data volume in the image is improved to a certain extent; secondly, data of an object with a small data amount in the image is augmented by a data synthesis method according to a common method, for example, the data of the object with the small data amount can be augmented by methods such as cutting, rotating, turning and adding noise, and then repeatedly pasted to the image of the original training data set for many times, so that further data augmentation is realized.

However, in the practical application process, the first data augmentation method is only simple repeated sampling, so that the diversity of data is insufficient, and the image processing method cannot be effectively improved; the second data augmentation method does not have sufficient evidence for the authenticity of the data and therefore still fails to meet the actual image processing requirements.

Disclosure of Invention

In order to solve the technical problems that in the prior art, a processing method for an object with a small data amount in an image is not accurate enough and cannot meet actual processing requirements, an embodiment of the invention provides a data processing method based on image transformation.

In order to achieve the above object, an embodiment of the present invention provides a data processing method based on image transformation, where the data processing method includes: identifying a target object in an image to be processed, and acquiring corresponding object identification information; performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and executing data augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information.

Preferably, the identifying a target object in the image to be processed and acquiring corresponding object identification information includes: performing image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed; acquiring an object marking frame for marking each target object; and taking the marking frame as the object identification information.

Preferably, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

Preferably, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes: acquiring an image limiting frame in the image to be processed; determining an operation limiting frame in the image limiting frame, wherein all the object labeling frames are contained in the operation limiting frame; executing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated labeling frame; and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

Preferably, the method further comprises: after the image limit frame in the image to be processed is obtained, judging whether a cutting instruction for the image limit frame is obtained or not; under the condition that the clipping instruction is obtained, performing corresponding clipping operation on the image limiting frame to obtain a clipped image limiting frame; determining the operation limiting frame in the cut image limiting frame.

Preferably, the determining an operation limiting frame within the image limiting frame includes: generating an envelope region based on each object labeling frame, wherein the envelope region is used for enveloping each object labeling frame in the envelope region along the outer edge of the object labeling frame; generating an operable region based on the envelope region and the image bounding box; a plurality of random points are acquired within the operational area, and the operational limit box is determined based on the random points.

Preferably, the acquiring four random points in the operable region includes: acquiring a preset length; generating a point generation region corresponding to each vertex of the image bounding box within the operable region based on the preset length, the point generation region being located within the image bounding box; one of the random points is acquired in each of the point generation areas.

Preferably, the performing the first perspective transformation operation on the operation limiting box based on the image limiting box to obtain a post-operation labeling box includes: establishing an incidence relation between the random point and the vertex; based on the association relationship, the length of the image limiting frame is used as the length after operation, the width of the image limiting frame is used as the width after operation, the first perspective transformation operation is executed on the operation standard frame, the standard frame after operation is obtained, and the vertexes of the marking frame after operation are in one-to-one correspondence with the vertexes of the image limiting frame.

Preferably, the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes: performing conversion processing on the object labeling frame based on the first perspective matrix to obtain a processed labeling frame; adjusting the processed marking frame to obtain an adjusted marking frame; and taking the adjusted marking frame as the first transformed information.

Preferably, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: acquiring preset liquefaction parameters; and executing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameters to obtain first converted information corresponding to the target object.

Preferably, the preset liquefaction parameter includes a preset liquefaction radius, and the performing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameter to obtain first converted information corresponding to the target object includes: randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and a connecting line between the first point and the second point is in the operable area; generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area; determining a liquefaction direction based on the first point and the second point; taking the second point as the point after the first point transformation, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region; and processing the object identification information based on the transformed region to obtain the first transformed information.

Preferably, the object identification information further includes object type and area information of each object labeling box, and the data processing method further includes: classifying the object marking frames based on the object categories to obtain classified object marking frames; obtaining area statistical information of the classified object labeling boxes in each classification based on the classified object labeling boxes and the area information; and performing re-amplification processing on the processed information according to a preset equalization rule based on the area statistical information to obtain the re-processed information.

Preferably, the re-amplifying the processed information according to a preset equalization rule based on the area statistical information to obtain re-processed information includes: acquiring the number of re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the number of re-augmentation times to obtain corresponding second transformed information; and performing re-amplification processing on the processed information based on the second transformed information to obtain re-processed information.

Preferably, the data processing method further includes: acquiring a preset image recognition model; and training the preset image recognition model based on the processed information to obtain a trained image recognition model.

Correspondingly, an embodiment of the present invention further provides a data processing apparatus based on image transformation, where the data processing apparatus includes: the target identification module is used for identifying a target object in the image to be processed and acquiring corresponding object identification information; the transformation module is used for executing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and the data processing module is used for executing data augmentation processing on the target object based on the object identification information and the first converted information to obtain processed information.

Preferably, the object recognition module includes: the object recognition unit is used for carrying out image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed; the labeling unit is used for acquiring an object labeling frame for labeling each target object; and the target determining unit is used for taking the marking frame as the object identification information.

Preferably, the transformation module comprises a perspective transformation unit for: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

Preferably, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes: acquiring an image limiting frame in the image to be processed; determining an operation limiting frame in the image limiting frame, wherein all the object labeling frames are contained in the operation limiting frame; executing a first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated labeling frame; and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

Preferably, the perspective transformation unit is further configured to: after the image limit frame in the image to be processed is obtained, judging whether a cutting instruction for the image limit frame is obtained or not; under the condition that the clipping instruction is obtained, performing corresponding clipping operation on the image limiting frame to obtain a clipped image limiting frame; determining the operation limiting frame in the cut image limiting frame.

Preferably, the conversion module comprises a liquefaction conversion unit for: acquiring preset liquefaction parameters; and executing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameters to obtain first converted information corresponding to the target object.

Preferably, the object identification information further includes object type and area information of each object labeling frame, the data processing apparatus further includes a reprocessing module, and the reprocessing module includes: the classification unit is used for classifying the object marking frames based on the object types to obtain classified object marking frames; a statistical unit, configured to obtain area statistical information of the classified object labeling boxes in each classification based on the classified object labeling boxes and the area information; and the reprocessing unit is used for carrying out the reprocessing on the processed information according to a preset equalization rule based on the area statistical information to obtain the reprocessed information.

Preferably, the reprocessing unit is configured to: acquiring the number of re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the number of re-augmentation times to obtain corresponding second transformed information; and performing re-amplification processing on the processed information based on the second transformed information to obtain re-processed information.

Preferably, the data processing apparatus further comprises a model training module, the model training module comprising: the model acquisition unit is used for acquiring a preset image recognition model; and the training unit is used for training the preset image recognition model based on the processed data to obtain a trained image recognition model.

On the other hand, the embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the data processing method provided by the embodiment of the present invention.

Through the technical scheme provided by the invention, the invention at least has the following technical effects:

on the basis of the traditional data augmentation method, the data in the image to be processed is further augmented in a perspective transformation and/or liquefaction transformation mode, so that the data volume can be effectively increased on the basis of considering the authenticity of the data, the comprehensiveness of the data is improved, and meanwhile, the diversity of the data is increased, so that the accuracy of identification processing of the image to be processed is effectively improved, and the actual requirements of users are met.

Additional features and advantages of embodiments of the invention will be set forth in the detailed description which follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention and together with the description serve to explain the embodiments of the invention without limiting the embodiments of the invention. In the drawings:

FIG. 1 is a flowchart of a specific implementation of a data processing method based on image transformation according to an embodiment of the present invention;

fig. 2 is a flowchart illustrating a specific implementation of identifying a target object in a data processing method based on image transformation according to an embodiment of the present invention;

fig. 3 is a flowchart illustrating a specific implementation of performing a first perspective transformation on object identification information in a data processing method based on image transformation according to an embodiment of the present invention;

FIG. 4 is a diagram illustrating the determination of an operation limiting box within an image limiting box in the data processing method based on image transformation according to the embodiment of the present invention;

FIG. 5 is a diagram illustrating an operation limiting box determined within an image limiting box in a data processing method based on image transformation according to another embodiment of the present invention;

FIG. 6 is a schematic diagram of acquiring random points in an operable area in a data processing method based on image transformation according to an embodiment of the present invention;

fig. 7 is a schematic diagram of performing a first perspective transformation on object identification information in the data processing method based on image transformation according to the embodiment of the present invention;

fig. 8 is a flowchart of a specific implementation of the conversion process performed on the object identification information in the data processing method based on image transformation according to the embodiment of the present invention;

fig. 9 is a schematic diagram of a liquefaction process performed on object identification information in the data processing method based on image transformation according to the embodiment of the present invention;

fig. 10 is a schematic structural diagram of a data processing apparatus based on image transformation according to an embodiment of the present invention.

Detailed Description

The following detailed description of embodiments of the invention refers to the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating embodiments of the invention, are given by way of illustration and explanation only, not limitation.

The terms "system" and "network" in embodiments of the present invention may be used interchangeably. The "plurality" means two or more, and in view of this, the "plurality" may also be understood as "at least two" in the embodiments of the present invention. "and/or" describes the association relationship of the associated objects, meaning that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" generally indicates that the preceding and following related objects are in an "or" relationship, unless otherwise specified. In addition, it should be understood that the terms first, second, etc. in the description of the embodiments of the invention are used for distinguishing between the descriptions and are not intended to indicate or imply relative importance or order to be construed.

Referring to fig. 1, an embodiment of the present invention provides a data processing method based on image transformation, where the data processing method includes:

s10) identifying the target object in the image to be processed, and acquiring corresponding object identification information;

s20) performing a first transformation operation on the object identification information, obtaining first transformed information corresponding to the target object;

s30) performs data augmentation processing on the target object based on the object identification information and the first converted information, obtaining processed information.

In a possible implementation manner, the scale of a certain company is developed faster, so that the number of people is increased faster, in order to better supervise the working conditions of the staff in the office environment, the office environment is monitored by images, and in order to better identify the target object in the monitored image, for example, a mobile phone in the monitored image, the data processing method based on image transformation provided by the embodiment of the present invention is adopted to process the image data.

In the application process, firstly, an image to be processed is obtained, and a target object in the image to be processed is identified, for example, in the embodiment of the present invention, the target object in the monitored image, such as a mobile phone, a head portrait, a computer screen, etc., is identified, and at least one piece of object identification information is obtained, for example, the object identification information is identification information of a head portrait of mobile phone identification information and identification information of a computer screen. Since the obtained data is less when the identification is directly performed, and particularly, the obtained data is less for an object with a small outline, at this time, a first transformation operation is performed on the object identification information, for example, in the embodiment of the present invention, the position of the camera is fixed, so that the object in the monitored image is far and small, and therefore, the far and near scale transformation in the monitored image can be consistent by performing the first perspective transformation operation on the monitored image, so as to obtain the transformed image.

In this case, a data augmentation operation is performed on the target object based on the object identification information and the first converted information, and for example, the first converted information and the object identification information may be superimposed as processed information, thereby implementing data augmentation of the monitored image.

In the embodiment of the invention, on the basis of the original image, the data in the image to be processed is augmented by adopting an image transformation method, so that on the basis of ensuring the authenticity of the data, the data quantity of the data is improved, and meanwhile, the diversity of the data is also improved, thereby being beneficial to the later learning of the data, improving the accuracy of identifying the target object in the image to be processed and meeting the actual requirement.

Referring to fig. 2, in the embodiment of the present invention, identifying the target object in the image to be processed to obtain the corresponding object identification information includes:

s101) carrying out image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed;

s102) obtaining an object labeling frame for labeling each target object;

s103) using the labeling frame as the object identification information.

In a possible implementation manner, in the process of identifying a target object in an image to be processed, image identification is performed on the image to be processed according to a preset algorithm, for example, in an embodiment of the present invention, in order to improve compatibility and an applicable range of the data processing method provided by the present invention, image identification is performed on the image to be processed by using a common target detection processing algorithm, for example, a user may select a target object to be identified according to an actual requirement, and identify at least one target object in the image to be processed. At this time, an object labeling frame for labeling each target object is obtained, for example, in a target detection processing algorithm, a human body in the monitored image is identified and then a labeling frame for each human body is generated, and an object type of each target object can be obtained, for example, whether each target object is a human body, a pet, a building or other types can be identified, and then the object labeling frame and the object type are used as object identification information.

In an embodiment of the present invention, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

Referring to fig. 3, in an embodiment of the present invention, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes:

s201) acquiring an image limiting frame in the image to be processed;

s202) determining an operation limiting frame in the image limiting frame, wherein all the object labeling frames are contained in the operation limiting frame;

s203) executing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated labeling frame;

s204) generating a corresponding first perspective matrix based on the operation limiting box and the operated labeling box.

In an embodiment of the present invention, the method further comprises: after the image limit frame in the image to be processed is obtained, judging whether a cutting instruction for the image limit frame is obtained or not; under the condition that the clipping instruction is obtained, performing corresponding clipping operation on the image limiting frame to obtain a clipped image limiting frame; determining the operation limiting frame in the cut image limiting frame.

In an embodiment of the present invention, the determining an operation restriction frame within the image restriction frame includes: generating an envelope region based on each object labeling frame, wherein the envelope region is used for enveloping each object labeling frame in the envelope region along the outer edge of the object labeling frame; generating an operable region based on the envelope region and the image bounding box; a plurality of random points are acquired within the operational area, and the operational limit box is determined based on the random points.

Further, in this embodiment of the present invention, the acquiring four random points in the operable region includes: acquiring a preset length; generating a point generation region corresponding to each vertex of the image bounding box within the operable region based on the preset length, the point generation region being located within the image bounding box; one of the random points is acquired in each of the point generation areas.

Referring to fig. 4, in a possible implementation, after acquiring the object identification information in the image to be processed, an image restriction box is first acquired in the image to be processed, for example, in an embodiment of the present invention, the image restriction box may be manually determined by an operator according to a monitoring area actually needed to be monitored in a current monitoring field of view, and an image restriction box is determined by inputting four points (for example, A, B, C, D points) in the monitoring area, at which time the operation restriction box is further determined in the image restriction box.

In another possible implementation, in order to reduce the data processing amount when processing an image, reduce the complexity of image processing, and improve the accuracy of image processing, after an image restriction frame in the image to be processed is obtained, it is further determined whether a clipping instruction for the image restriction frame is obtained, for example, in an embodiment of the present invention, referring to fig. 5, a user may specify a clipping region in the image restriction frame, for example, draw a clipping frame, a region included in the clipping frame is the clipping region, and click a corresponding clipping button to issue a clipping instruction, at this time, perform a clipping operation on the image restriction frame, obtain a clipped image restriction frame, and further determine an operation restriction frame in the clipped image restriction frame.

In the embodiment of the present invention, each object labeling box in the image bounding box or the cropped image bounding box may be enveloped according to a preset enveloping algorithm to generate an enveloping region, and then an operable region may be generated according to the enveloping region and the image bounding box, for example, a region between the enveloping region and the image bounding box is used as the operable region. Then, a plurality of random points are randomly acquired in the operable region, and the operation limiting frame is determined based on the plurality of random points, for example, in the embodiment of the present invention, four points (for example, H, I, J, K points) are randomly determined in the operable region of the image limiting frame, thereby determining the operation limiting frame. In another embodiment of the present invention, 2 points (for example, H, I points) are randomly determined within the operable region of the image bounding box after the cropping, thereby determining the operation bounding box (H, I, J, K).

In order to avoid that the operation limiting frame generated by the randomly determined points causes excessive distortion of the target object after perspective transformation, please refer to fig. 6, in the embodiment of the present invention, a preset length is further obtained, and a point generating area corresponding to each vertex of the image limiting frame is generated in the operable area based on the preset length, for example, in the embodiment of the present invention, each vertex is used as an origin, a rectangular frame is generated in the image limiting frame with the preset length as a side length, the rectangular frame is used as a point generating area, a random point is randomly generated in each point generating area, and a corresponding operation limiting frame is further determined.

In an embodiment of the present invention, the performing the first perspective transformation operation on the operation limiting box based on the image limiting box to obtain an operated annotation box includes: establishing an incidence relation between the random point and the vertex; based on the association relationship, the length of the image limiting frame is used as the length after operation, the width of the image limiting frame is used as the width after operation, the first perspective transformation operation is executed on the operation standard frame, the standard frame after operation is obtained, and the vertexes of the marking frame after operation are in one-to-one correspondence with the vertexes of the image limiting frame.

Referring to fig. 7, in one possible embodiment, after the operation restriction box is obtained, a first perspective transformation operation is performed on the operation restriction box according to the image restriction box to obtain a post-operation annotation box. Firstly, establishing an association relationship between a random point and a vertex of an image bounding box, for example, in the embodiment of the present invention, a point is associated with a point H, a point B is associated with a point I, a point C is associated with a point J, and a point D is associated with a point K, and then, according to the length and width of the preset standard box, performing a first perspective transformation operation on the operation bounding box, that is, transforming the operation bounding box into a labeled box having the same size as the image bounding box, thereby obtaining a labeled box after the operation (for example, composed of points H ', I', J ', and K'). At this time, the corresponding first perspective matrix may be obtained according to the operation limiting frame and the post-operation labeling frame, for example, in the embodiment of the present invention, the first perspective matrix may be generated according to a mapping relationship between corresponding points (e.g., corresponding to the H point and the H' point) in the operation limiting frame and the post-operation labeling frame.

In the embodiment of the invention, the perspective transformation is carried out on the image to be processed, so that the perspective phenomenon in the image to be processed is eliminated, on one hand, the problem of poor remote monitoring effect caused by the perspective effect of the monitoring area in the monitoring process of the monitoring area by adopting the fixed position camera can be effectively eliminated, and the same monitoring effect on the monitoring area at the far distance and the near distance of the camera is ensured; on the other hand, in the subsequent data processing process, the original data and the data after perspective transformation can be combined to perform deep learning processing, so that more comprehensive and diversified learning data can be obtained, the learning efficiency and the learning accuracy can be effectively improved, and the processing accuracy of image processing is improved.

Referring to fig. 8, in the embodiment of the present invention, the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes:

s301) converting the object labeling frame based on the first perspective matrix to obtain a processed labeling frame;

s302) adjusting the processed marking frame to obtain an adjusted marking frame;

s303) taking the adjusted marking frame as the first converted information.

In a possible embodiment, after the first perspective matrix is obtained, it is determined that the transform coefficients for processing the image to be processed into a general tiled image are obtained. In this case, a corresponding transformation process is performed on the object labeling frame, for example, each point on the object labeling frame is multiplied by the first perspective matrix to obtain a point after the multiplication process, and a processed labeling frame is determined according to the multiplied point.

In an embodiment of the present invention, the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object includes: acquiring preset liquefaction parameters; and executing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameters to obtain first converted information corresponding to the target object.

Further, in an embodiment of the present invention, the preset liquefaction parameter includes a preset liquefaction radius, and the performing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameter to obtain first converted information corresponding to the target object includes: randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and a connecting line between the first point and the second point is in the operable area; generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area; determining a liquefaction direction based on the first point and the second point; taking the second point as the point after the first point transformation, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region; and processing the object identification information based on the transformed region to obtain the first transformed information.

Referring to fig. 9, in a possible embodiment, a preset liquefaction region for liquefaction operation is obtained first, for example, in an embodiment of the present invention, a preset liquefaction radius (for example, R) may be set first, and then a first point (for example, point C) and a second point (for example, point M) may be randomly obtained within the determined operable region, where a distance between the first point and the second point is greater than the preset liquefaction radius, and a connecting line between the first point and the second point is located within the operable region. At this time, a preset liquefaction area is generated according to the point C and the preset liquefaction radius R, for example, the point C is used as a center of a circle, and the radius R is used as a radius to generate the preset liquefaction area, and the preset liquefaction area is located in the operable area. At this time, the liquefaction direction is determined according to the first point and the second point, for example, the liquefaction direction is determined by taking the point C as a starting point and the point M as an end point, then the second point (i.e., the point M) is taken as a point after the point C is transformed, and the first liquefaction transformation operation is performed on the preset liquefaction region according to the liquefaction direction to obtain a transformed region.

In the embodiment of the invention, the data augmentation operation is carried out on the target object by adopting a liquefaction conversion mode, more forms of image data can be generated on the target object on the basis of the image data of the real target object, so that the data volume of the target object can be effectively increased, and meanwhile, the increased data volume has high reality.

In this embodiment of the present invention, the object identification information further includes an object type and area information of each object labeling frame, and the data processing method further includes: classifying the object marking frames based on the object categories to obtain classified object marking frames; obtaining area statistical information of the classified object labeling boxes in each classification based on the classified object labeling boxes and the area information; and performing re-amplification processing on the processed information according to a preset equalization rule based on the area statistical information to obtain the re-processed information.

Further, in the embodiment of the present invention, the performing, based on the area statistical information, a re-amplification process on the processed information according to a preset equalization rule to obtain re-processed information includes: acquiring the number of re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the number of re-augmentation times to obtain corresponding second transformed information; and performing re-amplification processing on the processed information based on the second transformed information to obtain re-processed information.

In the actual application process, in the image to be processed before processing, an object located far away becomes very small due to the perspective effect, so that in the process of identifying or processing the object, the acquired data of the object is less, and the accuracy of identifying or processing the object is reduced. In order to solve the above technical problem, in one possible embodiment, after the preliminary data processing is performed on the image to be processed, data re-expansion is further performed on the target object in the image to be processed.

For example, in the embodiment of the present invention, the object labeling frames are classified according to the identified object categories, so as to obtain the classified object labeling frames in each category, and then the area statistical information of each classified object labeling frame is obtained according to the area information of each object labeling frame, for example, the area distribution information of the classified object labeling frames in each category is obtained, and at this time, the processed information is processed according to the preset equalization rule. For example, in the embodiment of the present invention, the preset equalization rule is to equalize the area ratio of the target objects in the same category, for example, the total area of the target objects in the mobile phone category is 30, where the area of the mobile phone a is 15, the area of the mobile phone B is 11, and the area of the mobile phone C is 4, so that the number of times of augmentation to the mobile phone a is 1, the number of times of augmentation to the mobile phone B is 3, and the number of times of augmentation to the mobile phone C is 8 are obtained according to the area distribution information. And then carrying out second perspective transformation operation on the object identification information corresponding to each mobile phone according to the augmentation times, acquiring a corresponding second perspective matrix, further acquiring corresponding second converted identification information according to the second perspective matrix, and overlapping the second converted identification information to the processed information, so that the mobile phone data is augmented again, and corresponding information after reprocessing is obtained.

In the embodiment of the invention, on the basis of perspective transformation operation, objects with small data volume are repeatedly processed for many times, and data processed for many times are superposed to obtain larger data volume, so that the reliability and the accuracy of the data can be effectively improved, and the processing accuracy of the image to be processed is improved.

In this embodiment of the present invention, the data processing method further includes: acquiring a preset image recognition model; and training the preset image recognition model based on the processed information to obtain a trained image recognition model.

In a possible implementation manner, after the image data after the augmentation process is acquired, a preset image recognition model is further acquired, for example, in an embodiment of the present invention, the preset image recognition model may be a deep learning model, and the deep learning model is trained based on the processed information, so as to obtain a trained image recognition model. The trained image recognition model is used for recognizing the image to be processed, so that a more accurate and reliable recognition or processing result can be obtained, and the actual requirements of users are met.

A data processing apparatus based on perspective transformation according to an embodiment of the present invention will be described with reference to the accompanying drawings.

Referring to fig. 10, based on the same inventive concept, an embodiment of the present invention provides a data processing apparatus based on perspective transformation, including: the target identification module is used for identifying a target object in the image to be processed and acquiring corresponding object identification information; the transformation module is used for executing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object; and the data processing module is used for executing data augmentation processing on the target object based on the object identification information and the first converted information to obtain processed information.

In an embodiment of the present invention, the target identification module includes: the object recognition unit is used for carrying out image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed; the labeling unit is used for acquiring an object labeling frame for labeling each target object; and the target determining unit is used for taking the marking frame as the object identification information.

In an embodiment of the present invention, the transformation module comprises a perspective transformation unit, the perspective transformation unit is configured to: performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object; and converting the object identification information based on the first perspective matrix to obtain the first converted information.

In an embodiment of the present invention, the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object includes: acquiring an image limiting frame in the image to be processed; determining an operation limiting frame in the image limiting frame, wherein all the object labeling frames are contained in the operation limiting frame; executing a first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated labeling frame; and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

In an embodiment of the present invention, the perspective transformation unit is further configured to: after the image limit frame in the image to be processed is obtained, judging whether a cutting instruction for the image limit frame is obtained or not; under the condition that the clipping instruction is obtained, performing corresponding clipping operation on the image limiting frame to obtain a clipped image limiting frame; determining the operation limiting frame in the cut image limiting frame.

In this embodiment of the present invention, the acquiring four random points in the operable region includes: acquiring a preset length; generating a point generation region corresponding to each vertex of the image bounding box within the operable region based on the preset length, the point generation region being located within the image bounding box; one of the random points is acquired in each of the point generation areas.

In an embodiment of the present invention, the converting the object identification information based on the first perspective matrix to obtain the first transformed information includes: performing conversion processing on the object labeling frame based on the first perspective matrix to obtain a processed labeling frame; adjusting the processed marking frame to obtain an adjusted marking frame; and taking the adjusted marking frame as the first transformed information.

In an embodiment of the invention, the transformation module comprises a liquefaction transformation unit for: acquiring preset liquefaction parameters; and executing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameters to obtain first converted information corresponding to the target object.

In an embodiment of the present invention, the preset liquefaction parameter includes a preset liquefaction radius, and the performing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameter to obtain first converted information corresponding to the target object includes: randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and a connecting line between the first point and the second point is in the operable area; generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area; determining a liquefaction direction based on the first point and the second point; taking the second point as the point after the first point transformation, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region; and processing the object identification information based on the transformed region to obtain the first transformed information.

In an embodiment of the present invention, the object identification information further includes an object type and area information of each object labeling frame, the data processing apparatus further includes a reprocessing module, and the reprocessing module includes: the classification unit is used for classifying the object marking frames based on the object types to obtain classified object marking frames; a statistical unit, configured to obtain area statistical information of the classified object labeling boxes in each classification based on the classified object labeling boxes and the area information; and the reprocessing unit is used for carrying out the reprocessing on the processed information according to a preset equalization rule based on the area statistical information to obtain the reprocessed information.

In an embodiment of the present invention, the reprocessing unit is configured to: acquiring the number of re-amplification times of each target object according to the preset equalization rule based on the area statistical information; performing a second transformation operation on the corresponding object identification information based on the number of re-augmentation times to obtain corresponding second transformed information; and performing re-amplification processing on the processed information based on the second transformed information to obtain re-processed information.

In an embodiment of the present invention, the data processing apparatus further includes a model training module, and the model training module includes: the model acquisition unit is used for acquiring a preset image recognition model; and the training unit is used for training the preset image recognition model based on the processed data to obtain a trained image recognition model.

Further, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and the computer program, when executed by a processor, implements the data processing method according to the embodiment of the present invention.

Although the embodiments of the present invention have been described in detail with reference to the accompanying drawings, the embodiments of the present invention are not limited to the details of the above embodiments, and various simple modifications can be made to the technical solutions of the embodiments of the present invention within the technical idea of the embodiments of the present invention, and the simple modifications all belong to the protection scope of the embodiments of the present invention.

It should be noted that the various features described in the above embodiments may be combined in any suitable manner without departing from the scope of the invention. In order to avoid unnecessary repetition, the embodiments of the present invention do not describe every possible combination.

Those skilled in the art will understand that all or part of the steps in the method according to the above embodiments may be implemented by a program, which is stored in a storage medium and includes several instructions to enable a single chip, a chip, or a processor (processor) to execute all or part of the steps in the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

In addition, any combination of various different implementation manners of the embodiments of the present invention is also possible, and the embodiments of the present invention should be considered as disclosed in the embodiments of the present invention as long as the combination does not depart from the spirit of the embodiments of the present invention.

Claims

1. A data processing method based on image transformation, the data processing method comprising:

identifying a target object in an image to be processed, and acquiring corresponding object identification information;

performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object;

and executing data augmentation processing on the target object based on the object identification information and the first transformed information to obtain processed information.

2. The data processing method according to claim 1, wherein the identifying the target object in the image to be processed and acquiring corresponding object identification information comprises:

performing image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed;

acquiring an object marking frame for marking each target object;

and taking the marking frame as the object identification information.

3. The data processing method according to claim 2, wherein the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object comprises:

performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object;

and converting the object identification information based on the first perspective matrix to obtain the first converted information.

4. The data processing method according to claim 3, wherein the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object comprises:

acquiring an image limiting frame in the image to be processed;

determining an operation limiting frame in the image limiting frame, wherein all the object labeling frames are contained in the operation limiting frame;

executing the first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated labeling frame;

and generating a corresponding first perspective matrix based on the operation limiting frame and the post-operation labeling frame.

5. The data processing method of claim 4, wherein the method further comprises:

after the image limit frame in the image to be processed is obtained, judging whether a cutting instruction for the image limit frame is obtained or not;

under the condition that the clipping instruction is obtained, performing corresponding clipping operation on the image limiting frame to obtain a clipped image limiting frame;

determining the operation limiting frame in the cut image limiting frame.

6. The data processing method of claim 4, wherein determining an operation bounding box within the image bounding box comprises:

generating an envelope region based on each object labeling frame, wherein the envelope region is used for enveloping each object labeling frame in the envelope region along the outer edge of the object labeling frame;

generating an operable region based on the envelope region and the image bounding box;

a plurality of random points are acquired within the operational area, and the operational limit box is determined based on the random points.

7. The data processing method of claim 6, wherein the obtaining four random points within the operable area comprises:

acquiring a preset length;

generating a point generation region corresponding to each vertex of the image bounding box within the operable region based on the preset length, the point generation region being located within the image bounding box;

one of the random points is acquired in each of the point generation areas.

8. The data processing method of claim 7, wherein the performing the first perspective transformation operation on the operation limiting box based on the image limiting box to obtain a post-operation labeling box comprises:

establishing an incidence relation between the random point and the vertex;

based on the association relationship, the length of the image limiting frame is used as the length after operation, the width of the image limiting frame is used as the width after operation, the first perspective transformation operation is executed on the operation standard frame, the standard frame after operation is obtained, and the vertexes of the marking frame after operation are in one-to-one correspondence with the vertexes of the image limiting frame.

9. The data processing method according to claim 3, wherein the converting the object identification information based on the first perspective matrix to obtain the first transformed information comprises:

performing conversion processing on the object labeling frame based on the first perspective matrix to obtain a processed labeling frame;

adjusting the processed marking frame to obtain an adjusted marking frame;

and taking the adjusted marking frame as the first transformed information.

10. The data processing method according to claim 6, wherein the performing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object comprises:

acquiring preset liquefaction parameters;

and executing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameters to obtain first converted information corresponding to the target object.

11. The data processing method of claim 10, wherein the preset liquefaction parameter comprises a preset liquefaction radius, and the performing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameter to obtain first converted information corresponding to the target object comprises:

randomly acquiring a first point and a second point in the operable area, wherein the distance between the first point and the second point is larger than the preset liquefaction radius, and a connecting line between the first point and the second point is in the operable area;

generating a preset liquefaction area based on the preset liquefaction center and the preset liquefaction radius by taking the first point as a preset liquefaction center, wherein the preset liquefaction area is positioned in the operable area;

determining a liquefaction direction based on the first point and the second point;

taking the second point as the point after the first point transformation, and executing the first liquefaction transformation operation on the preset liquefaction region according to the liquefaction direction to obtain a transformed region;

and processing the object identification information based on the transformed region to obtain the first transformed information.

12. The data processing method according to claim 1, wherein the object identification information further includes object category and area information of each of the object labeling boxes, the data processing method further comprising:

classifying the object marking frames based on the object categories to obtain classified object marking frames;

obtaining area statistical information of the classified object labeling boxes in each classification based on the classified object labeling boxes and the area information;

and performing re-amplification processing on the processed information according to a preset equalization rule based on the area statistical information to obtain the re-processed information.

13. The data processing method of claim 12, wherein the performing re-amplification processing on the processed information according to a preset equalization rule based on the area statistical information to obtain re-processed information comprises:

acquiring the number of re-amplification times of each target object according to the preset equalization rule based on the area statistical information;

performing a second transformation operation on the corresponding object identification information based on the number of re-augmentation times to obtain corresponding second transformed information;

and performing re-amplification processing on the processed information based on the second transformed information to obtain re-processed information.

14. The data processing method of claim 1, further comprising:

acquiring a preset image recognition model;

and training the preset image recognition model based on the processed information to obtain a trained image recognition model.

15. A data processing apparatus based on image transformation, characterized in that the data processing apparatus comprises:

the target identification module is used for identifying a target object in the image to be processed and acquiring corresponding object identification information;

the transformation module is used for executing a first transformation operation on the object identification information to obtain first transformed information corresponding to the target object;

and the data processing module is used for executing data augmentation processing on the target object based on the object identification information and the first converted information to obtain processed information.

16. The data processing apparatus of claim 15, wherein the object identification module comprises:

the object recognition unit is used for carrying out image recognition on the image to be processed according to a preset algorithm to obtain at least one target object in the image to be processed;

the labeling unit is used for acquiring an object labeling frame for labeling each target object;

and the target determining unit is used for taking the marking frame as the object identification information.

17. The data processing apparatus of claim 16, wherein the transformation module comprises a perspective transformation unit configured to:

18. The data processing apparatus of claim 17, wherein the performing a first perspective transformation operation on the object identification information to obtain a first perspective matrix corresponding to the target object comprises:

acquiring an image limiting frame in the image to be processed;

executing a first perspective transformation operation on the operation limiting frame based on the image limiting frame to obtain an operated labeling frame;

19. The data processing apparatus of claim 18, wherein the perspective transformation unit is further configured to:

determining the operation limiting frame in the cut image limiting frame.

20. The data processing apparatus of claim 18, wherein determining an operation bounding box within the image bounding box comprises:

21. The data processing apparatus of claim 20, wherein said obtaining four random points within the operable region comprises:

acquiring a preset length;

one of the random points is acquired in each of the point generation areas.

22. The data processing apparatus of claim 21, wherein the performing the first perspective transformation operation on the operation restriction box based on the image restriction box to obtain a post-operation labeling box comprises:

establishing an incidence relation between the random point and the vertex;

23. The data processing apparatus according to claim 17, wherein the performing a conversion process on the object identification information based on the first perspective matrix to obtain the first transformed information comprises:

adjusting the processed marking frame to obtain an adjusted marking frame;

and taking the adjusted marking frame as the first transformed information.

24. The data processing apparatus of claim 20, wherein the transformation module comprises a liquefaction transformation unit to:

acquiring preset liquefaction parameters;

25. The data processing apparatus of claim 24, wherein the preset liquefaction parameter comprises a preset liquefaction radius, and the performing a first liquefaction conversion operation on the object identification information based on the preset liquefaction parameter to obtain first converted information corresponding to the target object comprises:

26. The data processing apparatus of claim 15, wherein the object identification information further includes object category and area information of each of the object labeling boxes, the data processing apparatus further comprising a reprocessing module, the reprocessing module comprising:

the classification unit is used for classifying the object marking frames based on the object types to obtain classified object marking frames;

a statistical unit, configured to obtain area statistical information of the classified object labeling boxes in each classification based on the classified object labeling boxes and the area information;

and the reprocessing unit is used for carrying out the reprocessing on the processed information according to a preset equalization rule based on the area statistical information to obtain the reprocessed information.

27. The data processing apparatus of claim 26, wherein the reprocessing unit is configured to:

28. The data processing apparatus of claim 16, further comprising a model training module, the model training module comprising:

the model acquisition unit is used for acquiring a preset image recognition model;

and the training unit is used for training the preset image recognition model based on the processed data to obtain a trained image recognition model.

29. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the data processing method of any one of claims 1 to 14.