CN111461227B - Sample generation method, device, electronic equipment and computer readable medium - Google Patents
Sample generation method, device, electronic equipment and computer readable medium Download PDFInfo
- Publication number
- CN111461227B CN111461227B CN202010252334.3A CN202010252334A CN111461227B CN 111461227 B CN111461227 B CN 111461227B CN 202010252334 A CN202010252334 A CN 202010252334A CN 111461227 B CN111461227 B CN 111461227B
- Authority
- CN
- China
- Prior art keywords
- objects
- combined
- sample
- annotation information
- target object
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Processing Or Creating Images (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Embodiments of the present disclosure disclose a sample generation method, apparatus, electronic device, and computer-readable medium. One embodiment of the method comprises the following steps: obtaining a training sample, wherein the training sample comprises a sample image and corresponding labeling information, and the labeling information is used for representing positions of a plurality of objects displayed in the sample image; combining at least two objects in the plurality of objects to obtain a combined object; determining the annotation information corresponding to the combined object to obtain combined annotation information; the sample image and the combined annotation information are determined to be a new training sample. This embodiment achieves an improved salience of the combined object in the whole sample image.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a sample generation method, apparatus, electronic device, and computer readable medium.
Background
Object detection is a common scenario in image processing. In practical applications, a plurality of objects are often in reality in the image to be processed. The saliency of these objects often varies, and in a specific business scenario, a need is raised to detect saliency objects.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose a sample generation method, apparatus, electronic device, and computer-readable medium to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a sample generation method, including: obtaining a training sample, wherein the training sample comprises a sample image and corresponding labeling information, and the labeling information is used for representing positions of a plurality of objects displayed in the sample image; combining at least two objects in the plurality of objects to obtain a combined object; determining the annotation information corresponding to the combined object to obtain combined annotation information; the sample image and the combined annotation information are determined to be a new training sample.
In a second aspect, some embodiments of the present disclosure provide a sample generation apparatus comprising: the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is configured to acquire a training sample, the training sample comprises a sample image and corresponding labeling information, and the labeling information is used for representing positions of a plurality of objects displayed in the sample image; a combining unit configured to combine at least two objects of the plurality of objects to obtain a combined object; the annotation information determining unit is configured to determine annotation information corresponding to the combined object and obtain combined annotation information; and a sample generation unit configured to determine the sample image and the combined annotation information as a new training sample.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; and a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the method of any of the above.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program when executed by a processor implements any of the methods described above.
One of the above embodiments of the present disclosure has the following advantageous effects: the objects displayed in the sample image are combined to improve the significance of the combined objects in the whole sample image. Therefore, a foundation is provided for improving the accuracy of the significance detection model.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a schematic illustration of one application scenario of a sample generation method according to some embodiments of the present disclosure;
FIG. 2 is a flow chart of some embodiments of a sample generation method according to the present disclosure;
FIG. 3 is a flow chart of further embodiments of a sample generation method according to the present disclosure;
FIG. 4 is a schematic illustration of one application scenario of a sample generation method according to some embodiments of the present disclosure;
FIG. 5 is a schematic structural view of some embodiments of a sample generation device according to the present disclosure;
fig. 6 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 is a schematic diagram of one application scenario of a sample generation method according to some embodiments of the present disclosure.
The sample generation method provided by some embodiments of the present disclosure is generally performed by a terminal device. The terminal device may be hardware or software. When the terminal device is hardware, it may be various electronic devices supporting video processing, including but not limited to smart phones, tablet computers, electronic book readers, car terminals, etc. When the terminal device is software, it can be installed in the above-listed electronic device. It may be implemented as a plurality of software or software modules, for example, for providing distributed services, or as a single software or software module. The present invention is not particularly limited herein.
As shown in fig. 1, the execution subject of the sample generation method may be a terminal device, such as a smart phone. The terminal device may first obtain training samples. The training sample comprises a sample image 101 and corresponding labeling information. As an example, the annotation information may be annotation boxes 102, 103, 104, 105, 106, respectively, that are used to characterize the position of the 5 objects "printer", "bell", "clock", "calendar", "red flag" displayed in the sample image 101. At least two objects of the 5 objects may then be combined to obtain a combined object. As an example, three objects of "bell", "calendar", "red flag" may be combined to obtain a combined object. On this basis, the annotation information corresponding to the combined object may be determined, and the combined annotation information, for example, the annotation box 107, is obtained. Finally, the sample image 101 and the combined annotation information 107 are determined as a new training sample.
With continued reference to fig. 2, a flow 200 of some embodiments of a sample generation method according to the present disclosure is shown. The sample generation method comprises the following steps:
in step 201, a training sample is obtained.
In some embodiments, the execution subject of the sample generation method (e.g., the terminal device in fig. 1) may first obtain training samples. The training sample comprises a sample image and corresponding labeling information, wherein the labeling information is used for representing positions of a plurality of (two or more) objects displayed in the sample image. In practice, the objects displayed in the sample image may include real objects such as people, objects, animals and plants, and various virtual objects such as animated figures. The present disclosure is not limited in this regard. On this basis, as an example, the labeling information may be a circumscribed rectangular frame of each of the plurality of objects displayed in the sample image. Of course, the labeling information may be coordinates of each object, and the like.
In some embodiments, the execution body may combine at least two objects from the plurality of objects to obtain a combined object. In practice, at least two of the plurality of objects may be determined in a variety of ways. For example, two objects closest to each other may be selected.
As an example, an image located at the center position of the sample image may be first selected as the first object. On the basis, the object closest to the first object is selected as the second object. At least two objects may then be combined to obtain a combined object. In particular, at least two objects may be combined as a whole, i.e. a combined object.
And 203, determining the labeling information corresponding to the combined object to obtain the combined labeling information.
In some embodiments, the execution body may determine the annotation information corresponding to the combined object, to obtain the combined annotation information corresponding to the combined object. In practice, according to the difference of the labeling information, the execution subject can determine the labeling information corresponding to the combined object in a plurality of modes to obtain the combined labeling information. For example, in the case where the labeling information is a circumscribed rectangular frame, the circumscribed rectangular frame of the combined object may be determined from the circumscribed rectangular frames of the respective objects of the at least two objects that constitute the combined object. Specifically, as an example, a minimum bounding rectangle of bounding rectangle frames of respective objects of at least two objects may be determined as a bounding rectangle frame of a combined object.
At step 204, the sample image and the combined annotation information are determined to be a new training sample.
In some embodiments, the execution subject may determine the sample image and the combined annotation information as a new training sample.
In some alternative implementations of some embodiments, the target detection model may be trained with new training samples. Specifically, the target detection model can be trained by using new training samples through various machine learning methods, so as to improve the detection accuracy of the target detection model on the salient objects. Among other things, the object detection model may include, but is not limited to, SSD, YOLO, and the like.
Some embodiments of the present disclosure provide methods for enhancing the saliency of a combined object in an entire sample image by combining objects displayed in the sample image. Therefore, a foundation is provided for improving the accuracy of the significance detection model.
With further reference to fig. 3, a flow 300 of further embodiments of the sample generation method of the present disclosure is shown. The flow 300 of the sample generation method includes the steps of:
in step 301, a training sample is obtained.
In some embodiments, the specific implementation of step 301 and the technical effects thereof may refer to step 201 in those embodiments corresponding to fig. 2, which are not described herein.
Step 302, combining at least two objects in a plurality of objects to obtain a combined object, including the following sub-steps:
first, an object closest to a target object among a plurality of objects is determined. Specifically, the distance of each of the plurality of objects from the target object may be determined separately. On the basis, selecting the object nearest to the target object.
And secondly, combining the determined object with the target object to obtain a candidate combined object.
And thirdly, determining whether other objects exist in the preset range of the candidate combined object. In practice, the preset range may be set as needed. For example, the preset range may be a certain radius range centered on the combined object. On the basis of this, it can be determined whether there are other objects than those constituting the combined object among the plurality of objects within the preset range of the combined object.
And fourth, if the candidate combined object does not exist, determining the candidate combined object as the combined object.
If there is a candidate combination object, the step 3022 continues with the combination step by setting the candidate combination object as the target object and setting a plurality of objects other than the target object and the object closest to the target object as the plurality of objects.
In some embodiments, if it is determined whether other objects exist within the preset range of the candidate combined object, the candidate combined object may be taken as a target object, a plurality of objects other than the target object and the object closest to the target object may be taken as a plurality of objects, and the combining step may be continued. Thus, the combined object is obtained by performing steps 3021 and 3022 in a loop until no other object is present within the preset range of the candidate combined object.
And 303, determining the labeling information corresponding to the combined object to obtain the combined labeling information.
In some embodiments, steps 303-304 are specifically implemented and technical effects thereof may refer to steps 203-204 in those embodiments corresponding to fig. 2, which are not described herein.
Compared with the description of some embodiments corresponding to fig. 2, the combined object is finally obtained by selecting the target object and combining with other objects within the preset range. The combined object obtained in this way and the combined object obtained in other ways can more represent the significant content in the image. Thereby further improving the accuracy of the significance detection model.
With continued reference to fig. 4, an exemplary application scenario diagram of combining at least two objects of a plurality of objects to obtain a combined object in some embodiments of the sample generation method of the present disclosure is shown.
As shown in fig. 4, the sample image training sample includes a sample image 101 and corresponding annotation information. As an example, the annotation information may be annotation boxes 102, 103, 104, 105, 106 for characterizing the positions of the 5 objects "car", "bell", "pot", and "bulb", respectively, displayed in the sample image 101. At least two objects of the 5 objects may then be combined to obtain a combined object.
As an example, a target object (for example, object "bell") may be first selected from 5 objects. Then, the object closest to the object "bell", that is, the object "bulb", among the 5 objects is determined. Next, the specified object "bulb" is combined with the target object "bell" to obtain a candidate combination object 107. Thereafter, it is determined whether other objects exist within the preset range of the candidate combined object 107. That is, whether or not other objects exist within the preset range of the combined candidate object after the combination of the "bulb" and the "bell". Taking the object "potted" as an example in which the object is within the preset range, the candidate combination object 107 (i.e., the combination of "bulb" and "bell") may be taken as the target object, and a plurality of objects other than the target object ("bell") and the object closest to the target object ("bulb"), i.e., "car", "bell", "potted", may be taken as the plurality of objects, and the combination step may be continued. That is, the object closest to the target object, i.e., the candidate combination object (i.e., the combination of "bulb" and "bell"), i.e., the object "pot", among the plurality of objects ("cars", "bells", "pot"), is determined. Next, the object "potted" and the candidate combination object 107 (i.e., the combination of "bulb" and "bell") may be combined to obtain the candidate combination object 108, i.e., the combination of "bulb", "bell" and "potted". Next, it may be determined whether other objects exist within a preset range of the combined candidate object 108 of "light bulb", "bell", and "potted". Taking the example that "clock" and "car" are not within the preset range, the candidate combined object 108 may be determined as the combined object.
With further reference to fig. 5, as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of a sample generation apparatus, which correspond to those method embodiments shown in fig. 2, which are particularly applicable in various electronic devices.
As shown in fig. 5, a sample generation apparatus 500 of some embodiments of the present disclosure includes: an acquisition unit 501, a combination unit 502, a labeling information determination unit 503, and a sample generation unit 504. Wherein the obtaining unit 501 is configured to obtain a training sample, wherein the training sample comprises a sample image and corresponding annotation information for characterizing a position of a plurality of objects displayed in the sample image. A combining unit 502 is configured to combine at least two objects of the plurality of objects to obtain a combined object. The labeling information determining unit 503 is configured to determine labeling information corresponding to the combined object, and obtain combined labeling information. The sample generation unit 504 is configured to determine the sample image and the combined annotation information as a new training sample.
In alternative implementations of some embodiments, the combining unit 502 may be further configured to: selecting a target object from a plurality of objects, and executing the following combination steps: determining an object closest to the target object in the plurality of objects; combining the determined object with the target object to obtain a candidate combined object; determining whether other objects exist in a preset range of the candidate combined object; if not, the candidate combined object is determined to be the combined object.
In alternative implementations of some embodiments, the combining unit 502 may be further configured to: if so, the candidate combined object is taken as a target object, a plurality of objects except the target object and the object closest to the target object are taken as a plurality of objects, and the combining step is continuously executed.
In an alternative implementation of some embodiments, the method may further include: training unit (not shown in the figures). Wherein the training unit is configured to train the target detection model with the new training samples.
In an alternative implementation of some embodiments, the annotation information is a circumscribed rectangular box of each of the plurality of objects displayed in the sample image.
In some embodiments, the objects displayed in the sample image are combined to increase the saliency of the combined objects throughout the sample image. Therefore, a foundation is provided for improving the accuracy of the significance detection model.
Referring now to fig. 6, a schematic diagram of an electronic device 600 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), car terminals (e.g., car navigation terminals), and the like, as well as stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 6 is merely an example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 6, the electronic device 600 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 601, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 602 or a program loaded from a storage means 608 into a Random Access Memory (RAM) 603. In the RAM603, various programs and data required for the operation of the electronic apparatus 600 are also stored. The processing device 601, the ROM 602, and the RAM603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.
In general, the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 607 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 608 including, for example, a memory card; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While fig. 6 shows an electronic device 600 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 6 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 609, or from storage device 608, or from ROM 602. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing device 601.
It should be noted that the computer readable medium according to some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: obtaining a training sample, wherein the training sample comprises a sample image and corresponding labeling information, and the labeling information is used for representing positions of a plurality of objects displayed in the sample image; combining at least two objects in the plurality of objects to obtain a combined object; determining the annotation information corresponding to the combined object to obtain combined annotation information; the sample image and the combined annotation information are determined to be a new training sample.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes an acquisition unit, a combination unit, a labeling information determination unit, and a sample generation unit. The names of these units do not in any way limit the unit itself, for example, the acquisition unit may also be described as "unit for acquiring training samples".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
According to one or more embodiments of the present disclosure, there is provided a sample generation method including: obtaining a training sample, wherein the training sample comprises a sample image and corresponding labeling information, and the labeling information is used for representing positions of a plurality of objects displayed in the sample image; combining at least two objects in the plurality of objects to obtain a combined object; determining the annotation information corresponding to the combined object to obtain combined annotation information; the sample image and the combined annotation information are determined to be a new training sample.
According to one or more embodiments of the present disclosure, combining at least two objects of a plurality of objects to obtain a combined object includes: selecting a target object from a plurality of objects, and executing the following combination steps: determining an object closest to the target object in the plurality of objects; combining the determined object with the target object to obtain a candidate combined object; determining whether other objects exist in a preset range of the candidate combined object; if not, the candidate combined object is determined to be the combined object.
According to one or more embodiments of the present disclosure, at least two objects of the plurality of objects are combined to obtain a combined object, further comprising: if so, the candidate combined object is taken as a target object, a plurality of objects except the target object and the object closest to the target object are taken as a plurality of objects, and the combining step is continuously executed.
According to one or more embodiments of the present disclosure, the method further comprises: and training the target detection model by using the new training sample.
According to one or more embodiments of the present disclosure, the annotation information is a circumscribed rectangular box of each of a plurality of objects displayed in the sample image.
According to one or more embodiments of the present disclosure, there is provided a sample generation apparatus including: the system comprises an acquisition unit, a display unit and a display unit, wherein the acquisition unit is configured to acquire a training sample, the training sample comprises a sample image and corresponding labeling information, and the labeling information is used for representing positions of a plurality of objects displayed in the sample image; a combining unit configured to combine at least two objects of the plurality of objects to obtain a combined object; the annotation information determining unit is configured to determine annotation information corresponding to the combined object and obtain combined annotation information; and a sample generation unit configured to determine the sample image and the combined annotation information as a new training sample.
According to one or more embodiments of the present disclosure, the combining unit may be further configured to: selecting a target object from a plurality of objects, and executing the following combination steps: determining an object closest to the target object in the plurality of objects; combining the determined object with the target object to obtain a candidate combined object; determining whether other objects exist in a preset range of the candidate combined object; if not, the candidate combined object is determined to be the combined object.
According to one or more embodiments of the present disclosure, the combining unit may be further configured to: if so, the candidate combined object is taken as a target object, a plurality of objects except the target object and the object closest to the target object are taken as a plurality of objects, and the combining step is continuously executed.
According to one or more embodiments of the present disclosure, the above method may further include: training unit. Wherein the training unit is configured to train the target detection model with the new training samples.
According to one or more embodiments of the present disclosure, the annotation information is a circumscribed rectangular box of each of a plurality of objects displayed in the sample image.
According to one or more embodiments of the present disclosure, there is provided an electronic device including: one or more processors; and a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement a method as described above.
According to one or more embodiments of the present disclosure, a computer readable medium is provided, on which a computer program is stored, wherein the program, when executed by a processor, implements a method as described in any of the above.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (8)
1. A sample generation method, comprising:
obtaining a training sample, wherein the training sample comprises a sample image and corresponding annotation information, and the annotation information is used for representing positions of a plurality of objects displayed in the sample image;
combining at least two objects in the plurality of objects to obtain a combined object, including: selecting a target object from the plurality of objects: determining an object closest to the target object among the plurality of objects; combining the determined object with the target object to obtain a candidate combined object; determining whether other objects exist in the preset range of the candidate combined object; if not, determining the candidate combined object as the combined object;
determining the annotation information corresponding to the combined object to obtain combined annotation information;
and determining the sample image and the combined labeling information as a new training sample.
2. The method of claim 1, wherein the combining at least two of the plurality of objects results in a combined object, further comprising:
if the candidate combined object is present, the combination step is continued by using the candidate combined object as a target object and using a plurality of objects other than the target object and an object closest to the target object as a plurality of objects.
3. The method of claim 1, wherein the method further comprises:
and training the target detection model by using the new training sample.
4. A method according to any of claims 1-3, wherein the annotation information is a circumscribed rectangular box of each of a plurality of objects displayed in the sample image.
5. A sample generation apparatus comprising:
an acquisition unit configured to acquire a training sample, wherein the training sample comprises a sample image and corresponding annotation information for characterizing positions of a plurality of objects displayed in the sample image;
a combining unit configured to combine at least two objects of the plurality of objects to obtain a combined object, including: selecting a target object from the plurality of objects: determining an object closest to the target object among the plurality of objects; combining the determined object with the target object to obtain a candidate combined object; determining whether other objects exist in the preset range of the candidate combined object; if not, determining the candidate combined object as the combined object;
the annotation information determining unit is configured to determine the annotation information corresponding to the combined object to obtain combined annotation information;
and a sample generation unit configured to determine the sample image and the combined annotation information as a new training sample.
6. The apparatus of claim 5, wherein the combining unit is further configured to:
if the candidate combined object is present, the combination step is continued by using the candidate combined object as a target object and using a plurality of objects other than the target object and an object closest to the target object as a plurality of objects.
7. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.
8. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010252334.3A CN111461227B (en) | 2020-04-01 | 2020-04-01 | Sample generation method, device, electronic equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010252334.3A CN111461227B (en) | 2020-04-01 | 2020-04-01 | Sample generation method, device, electronic equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111461227A CN111461227A (en) | 2020-07-28 |
| CN111461227B true CN111461227B (en) | 2023-05-23 |
Family
ID=71681402
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010252334.3A Active CN111461227B (en) | 2020-04-01 | 2020-04-01 | Sample generation method, device, electronic equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461227B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9373036B1 (en) * | 2015-01-16 | 2016-06-21 | Toyota Motor Engineering & Manufacturing North America, Inc. | Collaborative distance metric learning for method and apparatus visual tracking |
| CN108460414A (en) * | 2018-02-27 | 2018-08-28 | 北京三快在线科技有限公司 | Generation method, device and the electronic equipment of training sample image |
| CN108985208A (en) * | 2018-07-06 | 2018-12-11 | 北京字节跳动网络技术有限公司 | The method and apparatus for generating image detection model |
| CN109035369A (en) * | 2018-07-12 | 2018-12-18 | 浙江工业大学 | A kind of sample extending method merging virtual sample |
| CN109544468A (en) * | 2018-10-26 | 2019-03-29 | 浙江师范大学 | A kind of image data amplification method |
| CN110378400A (en) * | 2019-07-08 | 2019-10-25 | 北京三快在线科技有限公司 | A kind of model training method and device for image recognition |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8471905B2 (en) * | 2009-04-09 | 2013-06-25 | Raytheon Company | Methods and apparatus for imaging |
| US20180053055A1 (en) * | 2016-08-22 | 2018-02-22 | Daqri, Llc | Integrating augmented reality content and thermal imagery |
| US10346721B2 (en) * | 2017-11-01 | 2019-07-09 | Salesforce.Com, Inc. | Training a neural network using augmented training datasets |
-
2020
- 2020-04-01 CN CN202010252334.3A patent/CN111461227B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9373036B1 (en) * | 2015-01-16 | 2016-06-21 | Toyota Motor Engineering & Manufacturing North America, Inc. | Collaborative distance metric learning for method and apparatus visual tracking |
| CN108460414A (en) * | 2018-02-27 | 2018-08-28 | 北京三快在线科技有限公司 | Generation method, device and the electronic equipment of training sample image |
| CN108985208A (en) * | 2018-07-06 | 2018-12-11 | 北京字节跳动网络技术有限公司 | The method and apparatus for generating image detection model |
| CN109035369A (en) * | 2018-07-12 | 2018-12-18 | 浙江工业大学 | A kind of sample extending method merging virtual sample |
| CN109544468A (en) * | 2018-10-26 | 2019-03-29 | 浙江师范大学 | A kind of image data amplification method |
| CN110378400A (en) * | 2019-07-08 | 2019-10-25 | 北京三快在线科技有限公司 | A kind of model training method and device for image recognition |
Non-Patent Citations (2)
| Title |
|---|
| 基于重采样的三维人脸样本扩充;盖赟等;《北京工业大学学报》;第39卷(第05期);全文 * |
| 面向深度学习的遥感图像目标样本生成方法;王敏;陈金勇;王港;高峰;孙康;许妙忠;;国外电子测量技术(04);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111461227A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111738316A (en) | Image classification method and device for zero sample learning and electronic equipment | |
| CN113191257B (en) | Order of strokes detection method and device and electronic equipment | |
| US20240177374A1 (en) | Video processing method, apparatus and device | |
| CN110619028A (en) | Map display method, device, terminal equipment and medium for house source detail page | |
| CN111461227B (en) | Sample generation method, device, electronic equipment and computer readable medium | |
| CN111680754B (en) | Image classification method, device, electronic equipment and computer readable storage medium | |
| CN111401182B (en) | Image detection method and device for feeding rail | |
| CN110334763B (en) | Model data file generation method, model data file generation device, model data file identification device, model data file generation apparatus, model data file identification apparatus, and model data file identification medium | |
| CN113296771A (en) | Page display method, device, equipment and computer readable medium | |
| CN112346630B (en) | State determination method, device, equipment and computer readable medium | |
| CN111835917A (en) | Method, device and equipment for showing activity range and computer readable medium | |
| CN111489286B (en) | Picture processing method, device, equipment and medium | |
| CN114613355B (en) | Video processing method and device, readable medium and electronic equipment | |
| CN110619089A (en) | Information retrieval method and device | |
| CN116974684B (en) | Map page layout method, map page layout device, electronic equipment and computer readable medium | |
| CN111797932B (en) | Image classification method, apparatus, device and computer readable medium | |
| CN112307245B (en) | Method and apparatus for processing image | |
| CN111026983B (en) | Method, device, medium and electronic equipment for realizing hyperlink | |
| CN114359673B (en) | Small sample smoke detection method, device and equipment based on metric learning | |
| CN113448550B (en) | Method and device for realizing collection management of classes, electronic equipment and computer medium | |
| CN110119457B (en) | Method and apparatus for generating information | |
| CN116883659A (en) | Image segmentation method based on pixel-to-multi-operator interaction sensitivity | |
| CN116795954A (en) | Question answering method and device, storage medium and electronic equipment | |
| CN116778452A (en) | Target detection method, device, equipment and storage medium | |
| CN117649459A (en) | Image drawing method, data generating method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: Tiktok vision (Beijing) Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |