CN111915642A - Image sample generation method, device, equipment and readable storage medium - Google Patents
Image sample generation method, device, equipment and readable storage medium Download PDFInfo
- Publication number
- CN111915642A CN111915642A CN202010961958.2A CN202010961958A CN111915642A CN 111915642 A CN111915642 A CN 111915642A CN 202010961958 A CN202010961958 A CN 202010961958A CN 111915642 A CN111915642 A CN 111915642A
- Authority
- CN
- China
- Prior art keywords
- target object
- new
- image sample
- shadow
- range
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 48
- 238000003860 storage Methods 0.000 title claims abstract description 17
- 238000005286 illumination Methods 0.000 claims abstract description 52
- 230000011218 segmentation Effects 0.000 claims abstract description 12
- 230000015654 memory Effects 0.000 claims description 20
- 238000012549 training Methods 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 7
- 239000000126 substance Substances 0.000 claims description 3
- 238000013528 artificial neural network Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 10
- 238000004364 calculation method Methods 0.000 description 4
- 238000004891 communication Methods 0.000 description 4
- 238000004590 computer program Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 238000004458 analytical method Methods 0.000 description 3
- 238000002372 labelling Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004088 simulation Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000000630 rising effect Effects 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/207—Analysis of motion for motion estimation over a hierarchy of resolutions
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10032—Satellite or aerial image; Remote sensing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the application discloses a method, a device and equipment for generating an image sample and a readable storage medium, and relates to the technical field of neural networks and example segmentation. The specific implementation scheme is as follows: acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object; generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range; and generating a new image sample according to the target object range and the new shadow range. The embodiment of the application can generate the image sample with authenticity and fitting practical situation.
Description
Technical Field
The present application relates to computer vision technology, and more particularly, to the field of neural networks and instance segmentation technology.
Background
When the image is deeply learned, a large number of image samples which are artificially labeled are needed to train a neural network model, and the model with sufficient precision can be obtained.
Since image samples are generally limited and the cost of manual labeling is high, image samples are generally augmented by data enhancement. In the prior art, generally, a whole image sample is simply translated, flipped, noised or rotated to generate a new image sample, and the label can be correspondingly generated in the same way.
The existing image sample generation method realizes simple geometric transformation on the whole image sample, and when a shadow exists in an image, a new image sample obtained through geometric transformation does not conform to the actual situation, so that the image sample lacks of reality.
Disclosure of Invention
The embodiment of the application provides a method, a device and equipment for generating an image sample and a readable storage medium.
In a first aspect, an embodiment of the present application provides a method for generating an image sample, including:
acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object;
generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
and generating a new image sample according to the target object range and the new shadow range.
In a second aspect, an embodiment of the present application further provides an apparatus for generating an image sample, including:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring an original image sample, and the original image sample is marked with a target object range and a shadow range of a target object;
the shadow generating module is used for generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
and the sample generating module is used for generating a new image sample according to the target object range and the new shadow range.
In a third aspect, an embodiment of the present application provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method for generating an image sample as provided in any of the embodiments.
In a fourth aspect, embodiments of the present application provide a non-transitory computer-readable storage medium storing computer instructions for causing a computer to execute a method for generating an image sample provided in any of the embodiments.
The embodiment of the application can generate the image sample with authenticity and fitting practical situation.
It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.
Drawings
The drawings are included to provide a better understanding of the present solution and are not intended to limit the present application. Wherein:
fig. 1a is a flowchart of a first method for generating an image sample in an embodiment of the present application;
FIG. 1b is a schematic diagram of a target object range and a shadow range provided by an embodiment of the present application;
FIG. 2a is a flow chart of a second method for generating an image sample according to an embodiment of the present application;
FIG. 2b is a schematic diagram of an original image sample in an embodiment of the present application;
FIG. 2c is a diagram of a new distal corner point and a new shadow range provided by an embodiment of the present application;
fig. 3 is a flowchart of a third image sample generation method in the embodiment of the present application;
fig. 4 is a block diagram of an image sample generation apparatus in the embodiment of the present application;
fig. 5 is a block diagram of an electronic device in the embodiment of the present application.
Detailed Description
The following description of the exemplary embodiments of the present application, taken in conjunction with the accompanying drawings, includes various details of the embodiments of the application for the understanding of the same, which are to be considered exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
According to the embodiment of the present application, fig. 1a is a flowchart of a first method for generating an image sample in the embodiment of the present application, and the embodiment of the present application is suitable for a case where a new image sample is generated on the basis of an original image sample. The method is executed by an image sample generating device, which is realized by software and/or hardware and is specifically configured in electronic equipment with certain data computing capability.
The method for generating an image sample as shown in fig. 1a comprises:
s110, obtaining an original image sample, wherein the original image sample is marked with a target object range and a shadow range of a target object.
The original image sample is an acquired image sample, for example, an image sample captured by a camera, and is used for training an image processing-related model. The number of the original image samples is at least one, and each original image sample displays the target object. The target object is a processing object of the model, including but not limited to an object, a person, an animal, and a plant.
Optionally, a target object range and a shadow range of the target object are labeled on each original image sample in a manual labeling manner. FIG. 1b is a schematic diagram of a target object range and a shadow range provided by an embodiment of the present application. The target object range may be an outline of the target object, the shadow of the target object is due to illumination, and the shadow range may be an outline of the shadow.
And S120, generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range.
Specifically, the outline of the shadow range is abstracted into a geometric figure to obtain the geometric characteristics of the geometric figure. For example, the shadow contours in fig. 1b are abstracted as polygons, and the geometric features include the length of each edge and/or the position of each corner point. According to the principle of light shadow generation, when the illumination angles are different, the shapes and the lengths of the shadow ranges are different, namely the geometric characteristics are different. However, since the target object is fixed, the geometric characteristics of the shadow range may exhibit regular variations when only the illumination angle is different. Based on the above analysis, when the geometric characteristics of the shadow range of the target object under one illumination angle are known, new shadow ranges under other different illumination angles can be roughly deduced.
It should be noted that the difference here means that the illumination angle of the new shadow range is different from the illumination angle in the original image sample, and also means that the illumination angles between a plurality of new shadow ranges are different.
Illustratively, in fig. 1b, the illumination angle is adjusted to the right, and each corner point of the original shadow range is adjusted to the left, so as to obtain a new shadow range. The new shaded area is shown in dashed lines.
And S130, generating a new image sample according to the target object range and the new shadow range.
Optionally, for one original image sample, new shadow ranges of the target object under a plurality of different illumination angles may be generated, and a new image sample may be generated for each new shadow range.
In this embodiment, assuming that the target object and the shooting angle are not changed, a new image sample is generated only when the illumination angle is adjusted. In an application scene, the target object and the shooting angle are kept unchanged, the target object is shot at different shooting time, and the target object and a new shadow range under different illumination angles can be obtained. When the target object is difficult to shoot at different shooting times, an original image sample obtained at one shooting time can be obtained, and new shadow ranges of the target object under different illumination angles are generated according to the geometric characteristics of the shadow ranges, so that the new shadow ranges obtained at different shooting times are simulated. Based on this, the target object range in the new image sample should coincide with the target object range in the original image sample, but the shadow range of the target object needs to be replaced with the new shadow range.
In the embodiment, under the condition of obtaining an original image sample and labeling a target object range and a shadow range, a new shadow range of a target object under different illumination angles can be generated through the geometric characteristics of the shadow range, so that image samples under different illumination angles are obtained through single illumination angle simulation; in the embodiment, considering that the new shadow is related to the geometric characteristics and the illumination angle, the new shadow range under different illumination angles is generated according to the geometric characteristics, and the shadow in the real situation is attached; and further generating a new image sample with reality and fitting practical situation according to the target object range and the new shadow range.
In the above and following embodiments, generating a new shadow range of the target object under different illumination angles according to the geometric features of the shadow range includes: determining the corner position of a new shadow of the target object under different illumination angles according to the corner position of the shadow range; and generating a new shadow range enclosed by the positions of the corner points of the new shadow.
In this embodiment, the geometric features of the shadow range include corner positions of the shadow range, that is, corner positions of a geometric figure to which the outline of the shadow range is abstracted. When the target object is under different illumination angles, although the shadow positions and lengths are different, the angular points of the shadow generally exist, and only the positions show regular changes.
Based on the above analysis, when the corner position of the shadow range of the target object under one illumination angle is known, the corner position of the new shadow of the target object under other different illumination angles can be calculated. Then, the connection relation between the angular points of the new shadow is obtained according to the connection relation between the angular points of the shadow range, and the new shadow range can be enclosed by connecting lines according to the connection relation.
In the embodiment, the cut-in is performed from the corner point, a new shadow range is generated by determining the corner point position of the target object under different illumination angles, and the new shadow range is roughly determined by the corner point position, so that the calculation amount is reduced while certain precision is ensured.
According to an embodiment of the present application, fig. 2a is a flowchart of a second method for generating an image sample in the embodiment of the present application, and the embodiment of the present application optimizes a process of determining a corner position of a new shadow based on the technical solutions of the above embodiments.
The method for generating an image sample as shown in fig. 2a comprises:
s210, obtaining an original image sample, wherein the original image sample is marked with a target object range and a shadow range of a target object.
S220, determining the motion tracks of the far-end corner points under different illumination angles according to the positions of the far-end corner points in the shadow range.
In the original image sample, the distance between the far-end corner point and the target object is greater than the distance between the near-end corner point and the target object. That is, in the original image sample, the far-end corner point is a corner point farther from the target object, and the near-end corner point is a corner point closer to the target object. Optionally, to simplify the calculation, the far-end corner point is a corner point farthest from the target object.
Fig. 2b is a schematic diagram of an original image sample in an embodiment of the present application. The shadow range of the target object in fig. 2B comprises corner a, corner B and corner C. Corner a is far from the target object and is a far-end corner, and corner B and corner C are near to the target object and are near-end corners.
When the target object is under different illumination angles, the position of the far-end angular point approximately moves on a motion trail. Optionally, the motion trajectory is simplified to an elliptical arc. Determining the length of the long axis and the length of the short axis of the elliptic arc according to the relative position relation of the light source and the target object and the geometric characteristics of the target object; and determining the elliptical arc passing through the far-end angular point as a motion track according to the length of the long axis and the length of the short axis of the elliptical arc and the position of the far-end angular point.
Specifically, the shadow range has the following relationship with the light source: 1) the length of the shadow is related to the angle of light source irradiation, and the direction of the shadow is opposite to that of the light source; 2) the size of the shadow range is related to the distance between the target object and the light source; 3) the shape of the shadow is related to the geometric features of the target object illuminated by the light source, including the position of the corner points. It can be seen that the position of the far-end corner point of the shadow range is related to the relative position relationship between the light source and the target object. In the real case, the light source is the sun, and the position of the target object can be represented by the latitude and longitude of the target object. Then, when the position of the target object is unchanged along with the rising and falling of the light source at different times of the day, the relative position relationship between the light source and the target object can be calculated, and then the position of the far-end corner point of the target object projected in the image under the irradiation of the light source at different positions is obtained by combining the geometric features of the target object, so as to obtain the length of the long axis and the length of the short axis of the elliptical arc.
In an alternative embodiment, as shown in fig. 2b, the geometric feature of the target object includes the position of the highest point of the target object, and the positions of the highest point of the target object projected in the image when the light source is directed to the front and the side of the target object, namely, the D point, the E point and the F point, are calculated. The front face of the target object is a face, facing the outer side of the image, of the target object in the original image sample, and the side face of the target object is a face perpendicular to the front face. When the light source faces the front of the target object, the highest point is projected at the position of a D point right behind the target object. When the light source faces the side face of the target object, the highest point is projected at the positions of the point E and the point F on the side of the target object. Then, 2 times the length of the target object distance D point is calculated as the minor axis length of the elliptical arc, and 2 times the length of the target object distance E point or F point is calculated as the major axis length of the elliptical arc.
In another alternative embodiment, the images captured at multiple capturing times may be analyzed in an observation manner to obtain the length of the major axis and the length of the minor axis of the elliptical arc of the distal corner point under different illumination angles.
Then, an elliptical arc passing through the distal corner point is determined as a motion trajectory based on the major axis length and the minor axis length of the elliptical arc and the position of the distal corner point.
Specifically, the shape of the elliptical arc can be known by knowing the length of the major axis and the length of the minor axis of the elliptical arc, and then, by combining the fact that the distal corner point is located on the elliptical arc, a plurality of elliptical arcs passing through the distal corner point can be determined, and any one elliptical arc can be selected as the motion trajectory of the distal corner point. Preferably, an elliptical arc with the long axis at a horizontal angle is selected as the motion trajectory of the distal corner point, as shown in fig. 2 b.
The embodiment simplifies the motion track into the elliptic arc, namely, the method is not distorted and real, and can reduce the calculated amount; the relation between the shadow range and the light source is skillfully utilized, the elliptical arc where the far-end angular point runs under different illumination angles is obtained in a geometric operation mode, and the calculation is simple and the practical situation is fitted.
And S230, selecting a new far-end corner point from the motion trail.
And selecting a new far-end angular point at any position of the motion trail. Optionally, in order to ensure the uniformity of the sample, a plurality of new distal corner points are selected at equal intervals on the motion trajectory.
S240, determining the position of a new near-end corner corresponding to the new far-end corner according to the relative position relation between the far-end corner and the near-end corner in the shadow range.
The relative positional relationship includes a relative distance and a direction. When the far-end angular point moves on the motion trail, the relative direction of the far-end angular point and the near-end angular point is kept unchanged, and the relative distance is changed in an equal proportion. From this, the position of the new near-end corner point corresponding to the new far-end corner point can be determined. The position of the new distal corner and the position of the new proximal corner constitute the corner positions of the new shadow.
Fig. 2C is a schematic diagram of a new distal corner point and a new shadow range provided in this embodiment, a line segment L formed by the distal corner point a and the proximal corner point B is parallel to a line segment L1 formed by the new distal corner point a1 and the new distal corner point B1, and a line segment M formed by the distal corner point a and the proximal corner point C is parallel to a line segment M1 formed by the new distal corner point a1 and the new distal corner point C1, that is, the relative directions are unchanged. The ratio of the lengths of line segments L and L1 is equal to the ratio of the lengths of line segments M and M1.
And S250, generating a new shadow range surrounded by the corner point positions of the new shadow.
And acquiring the connection relation between the angular points of the new shadow according to the connection relation between the angular points of the shadow range, and connecting lines according to the connection relation to enclose the new shadow range.
And S260, generating a new image sample according to the target object range and the new shadow range.
In this embodiment, the motion trajectories of the distal corner points at different illumination angles are determined according to the position of the distal corner point of the shadow range, on this basis, a new distal corner point is selected from the motion trajectories, and then the position of a new proximal corner point corresponding to the new distal corner point is determined according to the relative position relationship between the distal corner point and the proximal corner point in the shadow range, so that the new shadow range is determined by ingeniously using the position of the distal corner point of the shadow range and the relative position relationship between the distal corner point and the proximal corner point, the required data is few, the calculation is simple, and the obtained new shadow range is not distorted.
In the above-described embodiment and the following embodiments, the original image sample is a remote sensing image obtained by satellite shooting in a fixed orbit. Fixed orbit satellites may be photographed at different times. The conditions of different shooting times and constrained by the satellite orbit make the shooting angle and the illumination condition different. However, in an actual industrial environment, a high-resolution image is long in shooting period, and therefore only a remote sensing image of a certain shooting time can be obtained. Moreover, since the volume of remote sensing image data is huge, even if data with a large amount of shooting time and angle is collected for a long time, a large burden is imposed on image storage, preprocessing and training.
Based on the analysis, in an application scene of processing the remote sensing image, in order to shorten an image shooting period and reduce image storage, preprocessing and training loads, a limited number of original image samples are shot at the same shooting angle, preferably one original image sample is shot, and a new image sample is generated by adopting the image sample generation method provided by the embodiment of the application.
According to the embodiment of the present application, fig. 3 is a flowchart of a third method for generating an image sample in the embodiment of the present application, and the embodiment of the present application optimizes a method for generating an image sample based on the technical solutions of the above embodiments.
Optionally, after the operation "generate a new image sample according to the target object range and the new shadow range", the operation "train the example segmentation model using the original image sample and the new image sample" is added.
The method for generating an image sample as shown in fig. 3 includes:
s310, obtaining an original image sample, wherein the original image sample is marked with a target object range and a shadow range of a target object.
And S320, generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range.
And S330, generating a new image sample according to the target object range and the new shadow range.
And S340, training an example segmentation model by adopting the original image sample and the new image sample.
Example segmentation models are deep Neural network modules, such as Mask R-CNN (Region-Convolutional Neural Networks), YOLO, and the like. And training the original image sample and the new image sample generated by simulation to obtain at least one of a target object range and a shadow range of the target object output by the example segmentation model.
After the example segmentation model is trained, the remote sensing image can be used as input, and at least one of a target object range and a shadow range of a target object in the image is output.
In the embodiment, the image sample is applied to the training process of the example segmentation model, and the generated image sample is adopted for training, so that the requirement on the image sample size is low, and the effect of the example segmentation model can be effectively improved.
Fig. 4 is a block diagram of an apparatus for generating an image sample according to an embodiment of the present application, which is implemented by software and/or hardware and is specifically configured in an electronic device having a certain data computation capability.
An apparatus 400 for generating an image sample as shown in fig. 4 comprises: an acquisition module 401, a shadow generation module 402 and a sample generation module 403; wherein the content of the first and second substances,
an obtaining module 401, configured to obtain an original image sample, where the original image sample is labeled with a target object range and a shadow range of a target object;
a shadow generating module 402, configured to generate a new shadow range of the target object at different illumination angles according to the geometric features of the shadow range;
a sample generating module 403, configured to generate a new image sample according to the target object range and the new shadow range.
In the embodiment of the application, under the condition of obtaining an original image sample and marking a target object range and a shadow range, a new shadow range of a target object under different illumination angles can be generated through the geometric characteristics of the shadow range, so that the image samples under different illumination angles are obtained through single illumination angle simulation; in the embodiment, considering that the new shadow is related to the geometric characteristics and the illumination angle, the new shadow range under different illumination angles is generated according to the geometric characteristics, and the shadow in the real situation is attached; and further generating a new image sample with reality and fitting practical situation according to the target object range and the new shadow range.
Optionally, the shadow generating module 402 includes: the angular point position determining submodule is used for determining the angular point position of a new shadow of the target object under different illumination angles according to the angular point position of the shadow range; and the shadow generation submodule is used for generating a new shadow range enclosed by the corner point positions of the new shadow.
Optionally, the corner position determining sub-module includes: the track determining unit is used for determining the motion tracks of the far-end angular points under different illumination angles according to the positions of the far-end angular points in the shadow range; the selection unit is used for selecting a new far-end angular point from the motion trail; the position determining unit is used for determining the position of a new near-end angular point corresponding to the new far-end angular point according to the relative position relation between the far-end angular point and the near-end angular point in the shadow range; wherein, in the original image sample, the distance between the far-end corner point and the target object is larger than the distance between the near-end corner point and the target object.
Optionally, the trajectory determination unit includes: the length determining subunit is used for determining the length of the long axis and the length of the short axis of the elliptic arc according to the relative position relationship between the light source and the target object and the geometric characteristics of the target object; and the track determining subunit is used for determining the elliptical arc passing through the far-end angular point as the motion track according to the length of the long axis and the length of the short axis of the elliptical arc and the position of the far-end angular point.
Optionally, the original image sample is a remote sensing image obtained by shooting through a satellite in a fixed orbit.
Optionally, the apparatus 400 further includes: and the training module is used for training the example segmentation model by adopting the original image sample and the new image sample after generating the new image sample according to the target object range and the new shadow range.
The image sample generation device can execute the image sample generation method provided by any embodiment of the application, and has the corresponding functional modules and beneficial effects of executing the image sample generation method.
According to an embodiment of the present application, an electronic device and a readable storage medium are also provided.
Fig. 5 is a block diagram of an electronic device that implements the image sample generation method according to the embodiment of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application that are described and/or claimed herein.
As shown in fig. 5, the electronic apparatus includes: one or more processors 501, memory 502, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions for execution within the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input/output apparatus (such as a display device coupled to the interface). In other embodiments, multiple processors and/or multiple buses may be used, along with multiple memories and multiple memories, as desired. Also, multiple electronic devices may be connected, with each device providing portions of the necessary operations (e.g., as a server array, a group of blade servers, or a multi-processor system). In fig. 5, one processor 501 is taken as an example.
The memory 502, which is a non-transitory computer readable storage medium, may be used to store non-transitory software programs, non-transitory computer executable programs, and modules, such as program instructions/modules corresponding to the image sample generation method in the embodiment of the present application (for example, the system shown in fig. 4 includes an acquisition module 401, a shadow generation module 402, and a sample generation module 403). The processor 501 executes various functional applications of the server and data processing, i.e., a method of generating an image sample in the above-described method embodiments, by executing non-transitory software programs, instructions, and modules stored in the memory 502.
The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by use of an electronic device that implements the generation method of the image sample, and the like. Further, the memory 502 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 502 optionally includes memory located remotely from the processor 501, and these remote memories may be connected over a network to an electronic device that performs the method of generating the image samples. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The electronic device performing the method of generating an image sample may further include: an input device 503 and an output device 504. The processor 501, the memory 502, the input device 503 and the output device 504 may be connected by a bus or other means, and fig. 5 illustrates the connection by a bus as an example.
The input device 503 may receive input numeric or character information and generate key signal inputs related to user settings and function control of an electronic apparatus that performs the method of generating the image sample, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, a pointing stick, one or more mouse buttons, a track ball, a joystick, or other input devices. The output devices 504 may include a display device, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibrating motors), among others. The display device may include, but is not limited to, a Liquid Crystal Display (LCD), a Light Emitting Diode (LED) display, and a plasma display. In some implementations, the display device can be a touch screen.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software applications, or code) include machine instructions for a programmable processor, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.
The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.
It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present application may be executed in parallel, sequentially, or in different orders, and the present invention is not limited thereto as long as the desired results of the technical solutions disclosed in the present application can be achieved.
The above-described embodiments should not be construed as limiting the scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims (14)
1. A method of generating an image sample, comprising:
acquiring an original image sample, wherein the original image sample is marked with a target object range and a shadow range of the target object;
generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
and generating a new image sample according to the target object range and the new shadow range.
2. The method of claim 1, wherein the generating a new shadow range of the target object at different illumination angles according to the geometric features of the shadow range comprises:
determining the corner position of the new shadow of the target object under different illumination angles according to the corner position of the shadow range;
and generating a new shadow range enclosed by the positions of the corner points of the new shadow.
3. The method of claim 2, wherein the determining the corner positions of the new shadow of the target object under different illumination angles according to the corner positions of the shadow range comprises:
determining the motion trajectories of the far-end angular points under different illumination angles according to the positions of the far-end angular points of the shadow range;
selecting a new far-end angular point from the motion trail;
determining the position of a new near-end corner corresponding to the new far-end corner according to the relative position relation between the far-end corner and the near-end corner in the shadow range;
wherein, in the original image sample, the distance between the far-end corner point and the target object is greater than the distance between the near-end corner point and the target object.
4. The method according to claim 3, wherein the determining, according to the position of the distal corner point of the shadow range, the motion trajectory of the distal corner point under different illumination angles includes:
determining the length of the long axis and the length of the short axis of the elliptical arc according to the relative position relation of the light source and the target object and the geometric characteristics of the target object;
and determining the elliptical arc passing through the far-end angular point as the motion track according to the length of the long axis and the length of the short axis of the elliptical arc and the position of the far-end angular point.
5. The method according to any one of claims 1-4, wherein the original image sample is a remote sensing image taken by a fixed orbit satellite.
6. The method of any of claims 1-4, wherein, after said generating new image samples from said target object range and said new shadow range, further comprising:
and training an example segmentation model by using the original image sample and the new image sample.
7. An apparatus for generating an image sample, comprising:
the system comprises an acquisition module, a processing module and a display module, wherein the acquisition module is used for acquiring an original image sample, and the original image sample is marked with a target object range and a shadow range of a target object;
the shadow generating module is used for generating a new shadow range of the target object under different illumination angles according to the geometric characteristics of the shadow range;
and the sample generating module is used for generating a new image sample according to the target object range and the new shadow range.
8. The apparatus of claim 7, wherein the shadow generation module comprises:
the corner position determining submodule is used for determining the corner position of a new shadow of the target object under different illumination angles according to the corner position of the shadow range;
and the shadow generation submodule is used for generating a new shadow range enclosed by the corner point positions of the new shadow.
9. The apparatus of claim 8, wherein the corner location determination sub-module comprises:
the track determining unit is used for determining the motion tracks of the far-end angular points under different illumination angles according to the positions of the far-end angular points of the shadow range;
the selection unit is used for selecting a new far-end angular point from the motion trail;
a position determining unit, configured to determine a position of a new near-end corner corresponding to the new far-end corner according to a relative position relationship between the far-end corner and the near-end corner in the shadow range;
wherein, in the original image sample, the distance between the far-end corner point and the target object is greater than the distance between the near-end corner point and the target object.
10. The apparatus of claim 9, wherein the trajectory determination unit comprises:
the length determining subunit is used for determining the length of the long axis and the length of the short axis of the elliptic arc according to the relative position relationship between the light source and the target object and the geometric characteristics of the target object;
and the track determining subunit is used for determining the elliptical arc passing through the far-end angular point according to the length of the long axis and the length of the short axis of the elliptical arc and the position of the far-end angular point, and the elliptical arc is used as the motion track.
11. The apparatus of any of claims 7-10, wherein the original image sample is a remotely sensed image taken by a fixed orbit satellite.
12. The apparatus of any of claims 7-10, further comprising:
and the training module is used for training an example segmentation model by adopting the original image sample and the new image sample after generating the new image sample according to the target object range and the new shadow range.
13. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of generating an image sample as claimed in any one of claims 1 to 6.
14. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform a method of generating an image sample as claimed in any one of claims 1 to 6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010961958.2A CN111915642B (en) | 2020-09-14 | 2020-09-14 | Image sample generation method, device, equipment and readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010961958.2A CN111915642B (en) | 2020-09-14 | 2020-09-14 | Image sample generation method, device, equipment and readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111915642A true CN111915642A (en) | 2020-11-10 |
| CN111915642B CN111915642B (en) | 2024-05-14 |
Family
ID=73267882
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010961958.2A Active CN111915642B (en) | 2020-09-14 | 2020-09-14 | Image sample generation method, device, equipment and readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111915642B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114626468A (en) * | 2022-03-17 | 2022-06-14 | 小米汽车科技有限公司 | Method and device for generating shadow in image, electronic equipment and storage medium |
| CN117557466A (en) * | 2024-01-11 | 2024-02-13 | 中国科学院空天信息创新研究院 | Optical remote sensing image target image enhancement method and device based on imaging conditions |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070040832A1 (en) * | 2003-07-31 | 2007-02-22 | Tan Tiow S | Trapezoidal shadow maps |
| CN103455998A (en) * | 2012-06-04 | 2013-12-18 | 中兴通讯股份有限公司 | Method and device for detecting shadows in video image |
| CN108388833A (en) * | 2018-01-15 | 2018-08-10 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device and equipment |
| CN109118571A (en) * | 2018-07-23 | 2019-01-01 | 太平洋未来科技（深圳）有限公司 | Method, apparatus and electronic equipment based on light information rendering virtual objects |
| CN110033423A (en) * | 2019-04-16 | 2019-07-19 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN111292408A (en) * | 2020-01-21 | 2020-06-16 | 武汉大学 | Shadow generation method based on attention mechanism |
| WO2020151750A1 (en) * | 2019-01-24 | 2020-07-30 | 北京三快在线科技有限公司 | Image processing method and device |
| CN111583161A (en) * | 2020-06-17 | 2020-08-25 | 上海眼控科技股份有限公司 | Blurred image enhancement method, computer device and storage medium |
-
2020
- 2020-09-14 CN CN202010961958.2A patent/CN111915642B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070040832A1 (en) * | 2003-07-31 | 2007-02-22 | Tan Tiow S | Trapezoidal shadow maps |
| CN103455998A (en) * | 2012-06-04 | 2013-12-18 | 中兴通讯股份有限公司 | Method and device for detecting shadows in video image |
| CN108388833A (en) * | 2018-01-15 | 2018-08-10 | 阿里巴巴集团控股有限公司 | A kind of image-recognizing method, device and equipment |
| CN109118571A (en) * | 2018-07-23 | 2019-01-01 | 太平洋未来科技（深圳）有限公司 | Method, apparatus and electronic equipment based on light information rendering virtual objects |
| WO2020151750A1 (en) * | 2019-01-24 | 2020-07-30 | 北京三快在线科技有限公司 | Image processing method and device |
| CN110033423A (en) * | 2019-04-16 | 2019-07-19 | 北京字节跳动网络技术有限公司 | Method and apparatus for handling image |
| CN111292408A (en) * | 2020-01-21 | 2020-06-16 | 武汉大学 | Shadow generation method based on attention mechanism |
| CN111583161A (en) * | 2020-06-17 | 2020-08-25 | 上海眼控科技股份有限公司 | Blurred image enhancement method, computer device and storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 任慧群;蔡国印;李志强;: "基于深度学习的城市建筑物阴影提取方法", 地理信息世界, no. 02, 25 April 2020 (2020-04-25) * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114626468A (en) * | 2022-03-17 | 2022-06-14 | 小米汽车科技有限公司 | Method and device for generating shadow in image, electronic equipment and storage medium |
| CN114626468B (en) * | 2022-03-17 | 2024-02-09 | 小米汽车科技有限公司 | Method, device, electronic equipment and storage medium for generating shadow in image |
| CN117557466A (en) * | 2024-01-11 | 2024-02-13 | 中国科学院空天信息创新研究院 | Optical remote sensing image target image enhancement method and device based on imaging conditions |
| CN117557466B (en) * | 2024-01-11 | 2024-04-09 | 中国科学院空天信息创新研究院 | Optical remote sensing image target image enhancement method and device based on imaging conditions |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111915642B (en) | 2024-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102590841B1 (en) | virtual object driving Method, apparatus, electronic device, and readable storage medium | |
| CN112652016B (en) | Point cloud prediction model generation method, pose estimation method and pose estimation device | |
| CN111612852B (en) | Method and apparatus for verifying camera parameters | |
| CN112270669B (en) | Human body 3D key point detection method, model training method and related devices | |
| CN111832614A (en) | Training method and device of target detection model, electronic equipment and storage medium | |
| CN112529073A (en) | Model training method, attitude estimation method and apparatus, and electronic device | |
| CN111783948A (en) | Model training method and device, electronic equipment and storage medium | |
| CN111401251B (en) | Lane line extraction method, lane line extraction device, electronic equipment and computer readable storage medium | |
| CN111753961A (en) | Model training method and device, and prediction method and device | |
| CN111860167A (en) | Face fusion model acquisition and face fusion method, device and storage medium | |
| CN112308051B (en) | Text box detection method and device, electronic equipment and computer storage medium | |
| CN111767853A (en) | Lane line detection method and device | |
| CN111784834A (en) | Point cloud map generation method and device and electronic equipment | |
| CN111915642B (en) | Image sample generation method, device, equipment and readable storage medium | |
| CN112241716B (en) | Training sample generation method and device | |
| CN110619312A (en) | Method, device and equipment for enhancing positioning element data and storage medium | |
| CN112668428A (en) | Vehicle lane change detection method, roadside device, cloud control platform and program product | |
| CN113447128B (en) | Multi-human-body-temperature detection method and device, electronic equipment and storage medium | |
| CN111539347A (en) | Method and apparatus for detecting target | |
| CN111462179B (en) | Three-dimensional object tracking method and device and electronic equipment | |
| CN111275827B (en) | Edge-based augmented reality three-dimensional tracking registration method and device and electronic equipment | |
| CN111311743B (en) | Three-dimensional reconstruction precision testing method and device and electronic equipment | |
| KR20210146770A (en) | Method for indoor localization and electronic device | |
| CN112508027B (en) | Head model for instance segmentation, instance segmentation model, image segmentation method and device | |
| CN111369571B (en) | Three-dimensional object pose accuracy judging method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant |