CN111915642B - Image sample generation method, device, equipment and readable storage medium - Google Patents
Image sample generation method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN111915642B CN111915642B CN202010961958.2A CN202010961958A CN111915642B CN 111915642 B CN111915642 B CN 111915642B CN 202010961958 A CN202010961958 A CN 202010961958A CN 111915642 B CN111915642 B CN 111915642B
- Authority
- CN
- China
- Prior art keywords
- target object
- new
- image sample
- shadow
- range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000005286 illumination Methods 0.000 claims abstract description 52
- 230000011218 segmentation Effects 0.000 claims abstract description 13
- 230000015654 memory Effects 0.000 claims description 19
- 238000012549 training Methods 0.000 claims description 12
- 238000013528 artificial neural network Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 8
- 238000012545 processing Methods 0.000 description 5
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000013527 convolutional neural network Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000013519 translation Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a method, a device, equipment and a readable storage medium for generating an image sample, and relates to the technical fields of neural networks and instance segmentation. The specific implementation scheme is as follows: acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object; generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range; and generating a new image sample according to the target object range and the new shadow range. The embodiment of the application can generate the image sample with reality and fitting actual conditions.
Description
Technical Field
The application relates to a computer vision technology, in particular to the technical field of neural networks and instance segmentation.
Background
When the image is subjected to deep learning, a large number of image samples subjected to manual labeling are required to train the neural network model so as to obtain the model with sufficient precision.
Because image samples are generally limited and manually noted at a relatively high cost, image samples are generally expanded by data enhancement. In the prior art, the whole image sample is generally subjected to simple operations such as translation, overturning, noise adding or rotation, and the like, so that a new image sample is generated, and the labels can be correspondingly generated in the same way.
The existing image sample generation method is implemented by simple geometric transformation on the whole image sample, and when shadows exist in the image, a new image sample obtained through the geometric transformation does not accord with the actual situation, so that the image sample lacks of reality.
Disclosure of Invention
The embodiment of the application provides a method, a device, equipment and a readable storage medium for generating an image sample.
In a first aspect, an embodiment of the present application provides a method for generating an image sample, including:
Acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object;
Generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
and generating a new image sample according to the target object range and the new shadow range.
In a second aspect, an embodiment of the present application further provides an apparatus for generating an image sample, including:
The acquisition module is used for acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object;
the shadow generation module is used for generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
And the sample generation module is used for generating a new image sample according to the target object range and the new shadow range.
In a third aspect, an embodiment of the present application provides an electronic device, including:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of generating an image sample as provided in any one of the embodiments.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform a method of generating an image sample provided by any of the embodiments.
The embodiment of the application can generate the image sample with reality and fitting actual conditions.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are included to provide a better understanding of the present application and are not to be construed as limiting the application. Wherein:
FIG. 1a is a flowchart of a first method of generating an image sample in an embodiment of the application;
FIG. 1b is a schematic diagram of a target object range and shadow range provided by an embodiment of the present application;
FIG. 2a is a flow chart of a second method of generating an image sample in an embodiment of the application;
FIG. 2b is a schematic illustration of an original image sample in an embodiment of the application;
FIG. 2c is a schematic view of a new far-end corner point and a new shadow range provided by an embodiment of the present application;
FIG. 3 is a flowchart of a third method of generating an image sample in an embodiment of the application;
Fig. 4 is a block diagram of an image sample generating apparatus in an embodiment of the present application;
fig. 5 is a block diagram of an electronic device in an embodiment of the application.
Detailed Description
Exemplary embodiments of the present application will now be described with reference to the accompanying drawings, in which various details of the embodiments of the present application are included to facilitate understanding, and are to be considered merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1a is a flowchart of a first method for generating an image sample according to an embodiment of the present application, where the embodiment of the present application is applicable to a case of generating a new image sample based on an original image sample. The method is executed by a generating device of the image sample, and the device is realized by software and/or hardware and is specifically configured in electronic equipment with certain data operation capability.
The method for generating the image sample shown in fig. 1a comprises the following steps:
S110, acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object.
The original image sample is an acquired image sample, for example, an image sample taken by a camera, and is used for training an image processing related model. The number of the original image samples is at least one, and each original image sample is displayed with a target object. Wherein the target object is a processing object of the model, including but not limited to an object, a person, an animal, a plant, and the like.
Optionally, the target object range and the shadow range of the target object are marked on each original image sample in a manual marking mode. FIG. 1b is a schematic diagram of a target object range and shadow range provided by an embodiment of the present application. The target object range may be a contour of the target object, the shadow of the target object is due to illumination, and the shadow range may be a contour of the shadow.
S120, generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range.
Specifically, the outline of the shadow range is abstracted into a geometric figure, so that the geometric feature of the geometric figure is obtained. For example, the shadow contours in fig. 1b are abstracted to polygons, the geometrical features comprising the length of the sides and/or the position of the corner points. According to the shadow generation principle, when the illumination angles are different, the shape and the length of the shadow range are also different, namely, the geometric characteristics are different. However, since the target object is stationary, the geometry of the shadow range will exhibit regular changes when only the illumination angles are different. Based on the above analysis, when the geometry of the shadow range of the target object under one illumination angle is known, a new shadow range under other different illumination angles can be approximately deduced.
It should be noted that, the difference here refers to that the illumination angle of the new shadow range is different from the illumination angle in the original image sample, and also refers to that the illumination angles between the multiple new shadow ranges are different.
Illustratively, the illumination angle in fig. 1b is adjusted to the right, and each corner of the original shadow range is adjusted to the left, resulting in a new shadow range. The new shadow range is shown with dashed lines.
S130, generating a new image sample according to the target object range and the new shadow range.
Alternatively, for one original image sample, new shadow ranges of the target object under a plurality of different illumination angles may be generated, and a new image sample may be generated for each new shadow range.
In this embodiment, it is assumed that the target object and the photographing angle are unchanged, and a new image sample is generated when only the illumination angle is adjusted. In an application scene, the target object and the shooting angle are kept unchanged, and the target object is shot at different shooting times, so that the target object and a new shadow range under different illumination angles can be obtained. When it is difficult to photograph the target object at different photographing times, an original image sample obtained at one photographing time may be obtained, and a new shadow range of the target object under different illumination angles may be generated according to geometric features of the shadow range therein, thereby simulating the new shadow range obtained at different photographing times. Based on this, the target object range in the new image sample should coincide with the target object range in the original image sample, but the shadow range of the target object needs to be replaced with the new shadow range.
In the embodiment, under the condition of acquiring an original image sample and labeling a target object range and a shadow range, a new shadow range of a target object under different illumination angles can be generated through geometric features of the shadow range, so that the image sample under different illumination angles is obtained through single illumination angle simulation; in the embodiment, considering that the new shadow is related to the geometric feature and the illumination angle, generating a new shadow range under different illumination angles according to the geometric feature, and fitting the shadow in the actual situation; further, a new image sample with reality and fitting actual conditions is generated according to the target object range and the new shadow range.
In the above embodiment and the following embodiments, generating a new shadow range of the target object under different illumination angles according to geometric features of the shadow range includes: determining the angular point positions of new shadows of the target object under different illumination angles according to the angular point positions of the shadow ranges; a new shadow range is generated, surrounded by the corner positions of the new shadows.
In this embodiment, the geometric features of the shadow range include corner positions of the shadow range, that is, corner positions of the geometric figure into which the contour of the shadow range is abstracted. When the target object is under different illumination angles, although the position and the length of the shadow are different, the angular points of the shadow generally exist, and the position only shows regular change.
Based on the analysis, when the angular point position of the shadow range of the target object under one illumination angle is known, the angular point position of the new shadow of the target object under other different illumination angles can be calculated. Then, according to the connection relation among the corner points of the shadow range, the connection relation among the corner points of the new shadow is obtained, and the connection line is connected according to the connection relation, so that the new shadow range can be enclosed.
According to the embodiment, the corner is used as a cut-in, the position of the corner of the target object under different illumination angles is determined, so that a new shadow range is generated, the new shadow range is approximately determined through the position of the corner, and the calculated amount is reduced while certain precision is ensured.
Fig. 2a is a flowchart of a second method for generating an image sample according to an embodiment of the present application, where the determining process of the position of the corner point of the new shadow is optimized based on the technical solutions of the above embodiments.
The method for generating the image sample shown in fig. 2a comprises the following steps:
S210, acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object.
S220, determining the motion trail of the far-end angular point under different illumination angles according to the position of the far-end angular point of the shadow range.
In the original image sample, the distance between the far-end corner point and the target object is larger than the distance between the near-end corner point and the target object. That is, in the original image sample, the far-end corner is a corner far from the target object, and the near-end corner is a corner near to the target object. Optionally, to simplify the calculation, the far-end corner is the one furthest from the target object.
Fig. 2b is a schematic diagram of an original image sample in an embodiment of the application. The shadow range of the target object in fig. 2B includes corner a, corner B and corner C. Corner a is far from the target object, is a far-end corner, and corners B and C are near to the target object, are near-end corners.
When the target object is under different illumination angles, the position of the far-end corner point approximately moves on a motion track. Optionally, the motion trail is reduced to an elliptical arc. Determining the length of a major axis and the length of a minor axis of the elliptical arc according to the relative position relation between the light source and the target object and the geometric characteristics of the target object; and determining an elliptical arc passing through the distal corner point as a motion track according to the length of the long axis and the length of the short axis of the elliptical arc and the position of the distal corner point.
Specifically, the shadow range has the following relation with the light source: 1) The length of the shadow is related to the irradiation angle of the light source, and the direction of the shadow is opposite to the direction of the light source; 2) The size of the shadow range is related to the distance between the target object and the light source; 3) The shape of the shadow is related to the geometrical features of the target object illuminated by the light source, including the position of the corner points. It can be seen that the position of the far corner of the shadow range and the relative positional relationship of the light source and the target object are related. In a real situation, the light source is the sun, and the position of the target object can be represented by the longitude and latitude of the target object. Then, when the position of the target object is unchanged along with the east and west falling of the light source at different moments in the day, the relative position relation between the light source and the target object can be calculated, and then the position of the far-end angular point of the target object projected in the image under the irradiation of the light source at different positions is obtained by combining the geometric characteristics of the target object, so that the length of the long axis and the length of the short axis of the elliptical arc are obtained.
In an alternative embodiment, as shown in fig. 2b, the geometric feature of the target object includes the position of the highest point of the target object, and the positions of the highest point of the target object projected in the image, i.e. points D, E and F, when the light source is directed to the front and the side of the target object are calculated. The front surface of the target object is the surface of the target object facing the outer side of the image in the original image sample, and the side surface of the target object is the surface vertical to the front surface. When the light source faces the front of the target object, the highest point is projected at the position of the D point right behind the target object. When the light source faces the side face of the target object, the highest point is projected at the positions of points E and F on the side face of the target object. Then, 2 times the length of the target object from the point D is calculated as the short axis length of the elliptical arc, and 2 times the length of the target object from the point E or the point F is calculated as the long axis length of the elliptical arc.
In another alternative embodiment, an observation mode may be adopted to analyze images obtained through shooting at a plurality of shooting times, so as to obtain the length of the major axis and the length of the minor axis of the elliptical arc of the far-end corner point under different illumination angles.
Then, an elliptical arc passing through the distal corner point is determined as a motion trajectory according to the major axis length and the minor axis length of the elliptical arc and the position of the distal corner point.
Specifically, knowing the major axis length and the minor axis length of the elliptical arcs can obtain the shape of the elliptical arcs, and combining the fact that the distal corner points are located on the elliptical arcs can determine a plurality of elliptical arcs passing through the distal corner points, and one elliptical arc can be selected as the motion trail of the distal corner points. Preferably, an elliptical arc with a long axis at a horizontal angle is selected as the motion trajectory of the distal corner point, as shown in fig. 2 b.
The embodiment simplifies the motion trail into an elliptical arc, so that the reality is not lost, and the calculated amount can be reduced; the relation between the shadow range and the light source is skillfully utilized, the elliptical arcs of the far-end angular points running under different illumination angles are obtained in a geometric operation mode, and the practical conditions are attached while the calculation is simple.
S230, selecting a new far-end corner point from the motion trail.
And selecting a new far-end corner point at any position of the motion trail. Optionally, in order to ensure uniformity of the sample, a plurality of new far-end corner points are selected at equal intervals on the motion track.
S240, determining the position of the new corner point corresponding to the new corner point according to the relative position relation between the far corner point and the near corner point in the shadow range.
The relative positional relationship includes a relative distance and a direction. When the far-end angular point moves on the motion trail, the relative direction of the far-end angular point and the near-end angular point is kept unchanged, and the relative distance shows equal proportion change. Accordingly, the position of the new corner point corresponding to the new corner point can be determined. The position of the new far-end corner point and the position of the new end corner point form the corner point position of the new shadow.
Fig. 2C is a schematic diagram of a new far-end corner point and a new shadow range provided by the embodiment of the present application, where a line segment L formed by a far-end corner point a and a near-end corner point B is parallel to a line segment L1 formed by a new far-end corner point A1 and a new far-end corner point B1, and a line segment M formed by a far-end corner point a and a near-end corner point C is parallel to a line segment M1 formed by a new far-end corner point A1 and a new far-end corner point C1, i.e. the relative direction is unchanged. The length ratio of line segment L to L1 is equal to the length ratio of line segment M to M1.
S250, generating a new shadow range surrounded by the corner positions of the new shadows.
According to the connection relation among the corner points of the shadow range, the connection relation among the corner points of the new shadow is obtained, and the new shadow range can be enclosed according to the connection relation connection line.
S260, generating a new image sample according to the target object range and the new shadow range.
In this embodiment, the motion track of the far-end corner point under different illumination angles is determined by the position of the far-end corner point of the shadow range, on the basis of the motion track, a new far-end corner point is selected from the motion track, and then the position of the new far-end corner point corresponding to the new far-end corner point is determined according to the relative position relation between the far-end corner point and the near-end corner point in the shadow range, so that the position of the far-end corner point of the shadow range and the relative position relation between the far-end corner point and the near-end corner point are skillfully utilized to determine the new shadow range, the required data is less, the calculation is simple, and the obtained new shadow range is not lost.
In the above-described embodiments and the following embodiments, the original image sample is a remote sensing image obtained by satellite photographing of a fixed orbit. Fixed orbit satellites will take shots at different times. Under the constraint of the satellite running orbit and under the condition of different shooting time, the shooting angles and the illumination conditions are different. However, in an actual industrial environment, because the photographing period is long, only a remote sensing image with a certain photographing time can be obtained. Moreover, because of the huge volume of remote sensing image data, even if the data with more shooting time and angles are collected for a long time, a great burden is caused on image storage, preprocessing and training.
Based on the analysis, in an application scene for processing a remote sensing image, in order to shorten the image shooting period and reduce the burden of image storage, preprocessing and training, a limited number of original image samples are shot at the same shooting angle, and preferably one original image sample is shot, and a new image sample is generated by adopting the image sample generation method provided by the embodiment of the application.
Fig. 3 is a flowchart of a third method for generating an image sample according to an embodiment of the present application, where the method for generating an image sample is optimized based on the technical solutions of the embodiments.
Optionally, after the operation "generate new image samples from target object range and new shadow range", the instance segmentation model is trained with the original image samples and the new image samples "is appended.
The method for generating the image sample shown in fig. 3 comprises the following steps:
s310, acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object.
S320, generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range.
S330, generating a new image sample according to the target object range and the new shadow range.
S340, training an instance segmentation model by adopting the original image sample and the new image sample.
An example segmentation model is a deep neural network module, such as Mask R-CNN (Region-Convolutional Neural Networks, regional convolutional neural network), YOLO, and the like. Training an instance segmentation model by using the original image sample and the new image sample generated by simulation to obtain at least one of a target object range and a shadow range of the target object output by the instance segmentation model.
After the example segmentation model is trained, the remote sensing image can be used as input, and at least one of a target object range and a shadow range of the target object in the image can be output.
In the embodiment, the image sample is applied to the training process of the example segmentation model, and the generated image sample is adopted for training, so that the requirement on the data size of the image sample is low, and the effect of the example segmentation model can be effectively improved.
Fig. 4 is a block diagram of an apparatus for generating an image sample according to an embodiment of the present application, where the embodiment of the present application is applicable to a case of generating a new image sample based on an original image sample, and the apparatus is implemented by software and/or hardware and is specifically configured in an electronic device having a certain data computing capability.
An image sample generation apparatus 400 as shown in fig. 4, comprising: an acquisition module 401, a shadow generation module 402, and a sample generation module 403; wherein,
An obtaining module 401, configured to obtain an original image sample, where the original image sample is labeled with a target object range and a shadow range of the target object;
A shadow generating module 402, configured to generate a new shadow range of the target object under different illumination angles according to geometric features of the shadow range;
a sample generation module 403, configured to generate a new image sample according to the target object range and the new shadow range.
In the embodiment of the application, under the condition of acquiring an original image sample and labeling a target object range and a shadow range, a new shadow range of a target object under different illumination angles can be generated through the geometric characteristics of the shadow range, so that the image sample under different illumination angles is obtained through single illumination angle simulation; in the embodiment, considering that the new shadow is related to the geometric feature and the illumination angle, generating a new shadow range under different illumination angles according to the geometric feature, and fitting the shadow in the actual situation; further, a new image sample with reality and fitting actual conditions is generated according to the target object range and the new shadow range.
Optionally, the shadow generation module 402 includes: the angular point position determining submodule is used for determining the angular point position of a new shadow of the target object under different illumination angles according to the angular point position of the shadow range; and the shadow generation submodule is used for generating a new shadow range surrounded by the corner positions of the new shadow.
Optionally, the corner position determining submodule includes: the track determining unit is used for determining the motion track of the far-end angular point under different illumination angles according to the position of the far-end angular point of the shadow range; a selecting unit for selecting a new far-end corner point from the motion trail; the position determining unit is used for determining the position of the new corner point corresponding to the new corner point according to the relative position relation between the far corner point and the near corner point in the shadow range; wherein, in the original image sample, the distance between the far-end corner point and the target object is larger than the distance between the near-end corner point and the target object.
Optionally, the track determining unit includes: a length determining subunit, configured to determine a major axis length and a minor axis length of the elliptical arc according to the relative positional relationship between the light source and the target object and the geometric feature of the target object; and the track determining subunit is used for determining the elliptic arc passing through the far-end corner point as a motion track according to the long axis length and the short axis length of the elliptic arc and the position of the far-end corner point.
Alternatively, the original image sample is a remote sensing image taken by a fixed orbit satellite.
Optionally, the apparatus 400 further includes: and the training module is used for training an instance segmentation model by adopting the original image sample and the new image sample after generating the new image sample according to the target object range and the new shadow range.
The image sample generating device can execute the image sample generating method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the image sample generating method.
According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.
As shown in fig. 5, a block diagram of an electronic device implementing the method for generating an image sample according to an embodiment of the present application is shown. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the applications described and/or claimed herein.
As shown in fig. 5, the electronic device includes: one or more processors 501, memory 502, and interfaces for connecting components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple electronic devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 501 is illustrated in fig. 5.
Memory 502 is a non-transitory computer readable storage medium provided by the present application. The memory stores instructions executable by the at least one processor to cause the at least one processor to perform the method for generating image samples provided by the present application. The non-transitory computer readable storage medium of the present application stores computer instructions for causing a computer to execute the image sample generation method provided by the present application.
The memory 502 is used as a non-transitory computer readable storage medium for storing a non-transitory software program, a non-transitory computer executable program, and modules, such as program instructions/modules corresponding to the image sample generation method in the embodiment of the present application (for example, the acquisition module 401, the shadow generation module 402, and the sample generation module 403 shown in fig. 4). The processor 501 executes various functional applications of the server and data processing, i.e., a method of implementing the generation of image samples in the above-described method embodiments, by running non-transitory software programs, instructions, and modules stored in the memory 502.
Memory 502 may include a storage program area that may store an operating system, at least one application program required for functionality, and a storage data area; the storage data area may store data created by use of an electronic device implementing the generation method of the image sample, and the like. In addition, memory 502 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 502 may optionally include memory remotely located relative to processor 501, which may be connected via a network to an electronic device performing the method of generating image samples. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device that performs the image sample generation method may further include: an input device 503 and an output device 504. The processor 501, memory 502, input devices 503 and output devices 504 may be connected by a bus or otherwise, for example in fig. 5.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function controls of an electronic device performing the method of generating image samples, such as input devices for a touch screen, a keypad, a mouse, a track pad, a touch pad, a joystick, one or more mouse buttons, a track ball, a joystick, etc. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASIC (application specific integrated circuit), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computing programs (also referred to as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present application may be performed in parallel, sequentially, or in a different order, provided that the desired results of the disclosed embodiments are achieved, and are not limited herein.
The above embodiments do not limit the scope of the present application. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application should be included in the scope of the present application.
Claims (10)
1. A method of generating an image sample, comprising:
Acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object;
Generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
generating a new image sample according to the target object range and the new shadow range;
Wherein the generating a new shadow range of the target object under different illumination angles according to the geometric features of the shadow range comprises:
Determining the angular point positions of new shadows of the target object under different illumination angles according to the angular point positions of the shadow ranges;
Generating a new shadow range surrounded by the angular point positions of the new shadows;
wherein the geometric features of the shadow range comprise the length of each edge and/or the position of each corner point;
the determining the angular point position of the new shadow of the target object under different illumination angles according to the angular point position of the shadow range comprises the following steps:
determining the motion trail of the far-end angular point under different illumination angles according to the position of the far-end angular point of the shadow range;
selecting a new far-end angular point from the motion trail;
determining the position of a recent corner corresponding to the new corner according to the relative position relation between the far corner and the near corner in the shadow range;
in the original image sample, the distance between the far-end corner point and the target object is larger than the distance between the near-end corner point and the target object.
2. The method of claim 1, wherein the determining the motion trail of the far-end corner under different illumination angles according to the position of the far-end corner of the shadow range comprises:
determining the length of a major axis and the length of a minor axis of an elliptical arc according to the relative position relation between a light source and the target object and the geometric characteristics of the target object;
and determining an elliptical arc passing through the distal corner point as the motion trail according to the length of the long axis and the length of the short axis of the elliptical arc and the position of the distal corner point.
3. The method of any of claims 1-2, wherein the raw image sample is a remote sensing image taken by a fixed orbit satellite.
4. The method of any of claims 1-2, wherein after the generating a new image sample from the target object range and the new shadow range, further comprising:
and training an instance segmentation model by adopting the original image sample and the new image sample.
5. An image sample generation apparatus comprising:
The acquisition module is used for acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object;
the shadow generation module is used for generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
the sample generation module is used for generating a new image sample according to the target object range and the new shadow range;
Wherein, the shadow generation module includes:
The angular point position determining submodule is used for determining the angular point positions of the new shadows of the target object under different illumination angles according to the angular point positions of the shadow ranges;
a shadow generation sub-module, configured to generate a new shadow range surrounded by the corner positions of the new shadow;
wherein the geometric features of the shadow range comprise the length of each edge and/or the position of each corner point;
Wherein, the corner position determination submodule includes:
the track determining unit is used for determining the motion track of the far-end angular point under different illumination angles according to the position of the far-end angular point of the shadow range;
a selecting unit, configured to select a new far-end corner point from the motion trail;
the position determining unit is used for determining the position of the recent corner point corresponding to the new corner point according to the relative position relation between the far corner point and the near corner point in the shadow range;
in the original image sample, the distance between the far-end corner point and the target object is larger than the distance between the near-end corner point and the target object.
6. The apparatus of claim 5, wherein the trajectory determination unit comprises:
A length determining subunit, configured to determine a major axis length and a minor axis length of the elliptical arc according to a relative positional relationship between the light source and the target object and a geometric feature of the target object;
And the track determining subunit is used for determining an elliptic arc passing through the far-end angular point as the motion track according to the long axis length and the short axis length of the elliptic arc and the position of the far-end angular point.
7. The apparatus of any of claims 5-6, wherein the raw image sample is a remote sensing image taken by a fixed orbit satellite.
8. The apparatus of any of claims 5-6, further comprising:
And the training module is used for training an instance segmentation model by adopting the original image sample and the new image sample after generating the new image sample according to the target object range and the new shadow range.
9. An electronic device, comprising:
at least one processor; and
A memory communicatively coupled to the at least one processor; wherein,
The memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of generating an image sample according to any one of claims 1-4.
10. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform a method of generating an image sample as claimed in any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010961958.2A CN111915642B (en) | 2020-09-14 | 2020-09-14 | Image sample generation method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010961958.2A CN111915642B (en) | 2020-09-14 | 2020-09-14 | Image sample generation method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111915642A CN111915642A (en) | 2020-11-10 |
| CN111915642B true CN111915642B (en) | 2024-05-14 |
Family
ID=73267882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010961958.2A Active CN111915642B (en) | 2020-09-14 | 2020-09-14 | Image sample generation method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111915642B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114626468B (en) * | 2022-03-17 | 2024-02-09 | 小米汽车科技有限公司 | Method, device, electronic equipment and storage medium for generating shadow in image |
| CN117557466B (en) * | 2024-01-11 | 2024-04-09 | 中国科学院空天信息创新研究院 | Optical remote sensing image target image enhancement method and device based on imaging conditions |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455998A (en) * | 2012-06-04 | 2013-12-18 | 中兴通讯股份有限公司 | Method and device for detecting shadows in video image |
| CN108388833A (en) * | 2018-01-15 | 2018-08-10 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device and equipment |
| CN109118571A (en) * | 2018-07-23 | 2019-01-01 | 太平洋未来科技（深圳）有限公司 | Method, apparatus and electronic equipment based on light information rendering virtual objects |
| CN110033423A (en) * | 2019-04-16 | 2019-07-19 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN111292408A (en) * | 2020-01-21 | 2020-06-16 | 武汉大学 | Shadow generation method based on attention mechanism |
| WO2020151750A1 (en) * | 2019-01-24 | 2020-07-30 | 北京三快在线科技有限公司 | Image processing method and device |
| CN111583161A (en) * | 2020-06-17 | 2020-08-25 | 上海眼控科技股份有限公司 | Blurred image enhancement method, computer device and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070040832A1 (en) * | 2003-07-31 | 2007-02-22 | Tan Tiow S | Trapezoidal shadow maps |
-
2020
- 2020-09-14 CN CN202010961958.2A patent/CN111915642B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103455998A (en) * | 2012-06-04 | 2013-12-18 | 中兴通讯股份有限公司 | Method and device for detecting shadows in video image |
| CN108388833A (en) * | 2018-01-15 | 2018-08-10 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device and equipment |
| CN109118571A (en) * | 2018-07-23 | 2019-01-01 | 太平洋未来科技（深圳）有限公司 | Method, apparatus and electronic equipment based on light information rendering virtual objects |
| WO2020151750A1 (en) * | 2019-01-24 | 2020-07-30 | 北京三快在线科技有限公司 | Image processing method and device |
| CN110033423A (en) * | 2019-04-16 | 2019-07-19 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN111292408A (en) * | 2020-01-21 | 2020-06-16 | 武汉大学 | Shadow generation method based on attention mechanism |
| CN111583161A (en) * | 2020-06-17 | 2020-08-25 | 上海眼控科技股份有限公司 | Blurred image enhancement method, computer device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 基于深度学习的城市建筑物阴影提取方法;任慧群;蔡国印;李志强;;地理信息世界;20200425(第02期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111915642A (en) | 2020-11-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102590841B1 (en) | virtual object driving Method, apparatus, electronic device, and readable storage medium | |
| CN111783948A (en) | Model training method and device, electronic equipment and storage medium | |
| CN112270669B (en) | Human body 3D key point detection method, model training method and related devices | |
| CN111612852B (en) | Method and apparatus for verifying camera parameters | |
| CN111832614A (en) | Training method and device of target detection model, electronic equipment and storage medium | |
| CN111523468A (en) | Human body key point identification method and device | |
| CN112529073A (en) | Model training method, attitude estimation method and apparatus, and electronic device | |
| CN112652016A (en) | Point cloud prediction model generation method, pose estimation method and device | |
| CN111860167A (en) | Face fusion model acquisition and face fusion method, device and storage medium | |
| CN111915642B (en) | Image sample generation method, device, equipment and readable storage medium | |
| CN110619312B (en) | Method, device and equipment for enhancing positioning element data and storage medium | |
| CN111524166A (en) | Video frame processing method and device | |
| KR102566300B1 (en) | Method for indoor localization and electronic device | |
| CN112241716B (en) | Training sample generation method and device | |
| CN111784757A (en) | Training method of depth estimation model, depth estimation method, device and equipment | |
| CN111275827A (en) | Edge-based augmented reality three-dimensional tracking registration method and device and electronic equipment | |
| CN111311743B (en) | Three-dimensional reconstruction precision testing method and device and electronic equipment | |
| CN113447128B (en) | Multi-human-body-temperature detection method and device, electronic equipment and storage medium | |
| CN111462179B (en) | Three-dimensional object tracking method and device and electronic equipment | |
| CN112102417A (en) | Method and device for determining world coordinates and external reference calibration method for vehicle-road cooperative roadside camera | |
| CN111599002A (en) | Method and apparatus for generating image | |
| CN112508027B (en) | Head model for instance segmentation, instance segmentation model, image segmentation method and device | |
| CN111369571B (en) | Three-dimensional object pose accuracy judging method and device and electronic equipment | |
| CN113052753B (en) | Panoramic topological structure generation method, device and equipment and readable storage medium | |
| CN112017304A (en) | Method, apparatus, electronic device, and medium for presenting augmented reality data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |