Disclosure of Invention
The purpose of the invention is as follows: the invention aims to provide a sample acquisition and rapid labeling method which can quickly acquire a large number of samples, quickly finish accurate labeling and improve the image recognition efficiency and has a relatively fixed target state.
The technical scheme is as follows: the invention discloses a sample acquisition and rapid labeling method with a relatively fixed target state, which comprises the following steps:
(1) arranging a camera for shooting a target;
(2) the picture acquisition equipment is in communication connection with each camera and acquires pictures shot by each camera;
(3) the picture acquisition equipment configures all target information, camera information, associated targets, cameras and preset camera position information;
(4) the picture acquisition equipment starts an automatic snapshot tool to generate a sample picture and a pre-marked file;
(5) and manually checking the automatically-snapped sample pictures, carrying out batch secondary labeling on the pictures with the changed target states, and finally completing sample collection and rapid labeling.
In the step (1), the number of the targets and the cameras is one or more, and the cameras and the targets are in one-to-one or one-to-many relationship.
When the camera and the target are in a one-to-many relationship, the method comprises the steps that a single camera shoots a plurality of different targets at the same preset position, and the single camera rotates to different preset positions to shoot the plurality of different targets.
In the step (2), the picture acquisition equipment is in communication connection with each camera through a wired network or a wireless network.
In the step (2), the image acquisition equipment controls the camera to rotate to a preset position and controls the camera to capture a target image.
In the step (4), before the picture acquisition equipment starts an automatic capturing tool contained in the picture acquisition equipment, a capturing period is set, the picture acquisition equipment periodically and sequentially controls each camera to rotate to each preset position according to configuration information to capture and generate a sample picture containing each target, and then a corresponding pre-marked file is automatically generated.
The sample picture and the corresponding pre-labeled file have the same name, and the file name comprises the current time, the shooting camera and the preset position information.
In the step (5), the sample pictures automatically snapped are manually checked, the pictures with the changed target states are secondarily labeled in batches, and sample collection and rapid labeling are finally completed, and the method specifically comprises the following steps:
(5.1) determining a sample picture with a target state inconsistent with the initially configured target state, and recording the time when the target state changes;
(5.2) sorting and selecting each type of inconsistent pictures and pre-labeled files thereof;
(5.3) selecting one picture in each category and the labeled file thereof, and resetting the category and the position information of the target category by using a batch secondary labeling tool;
and (5.4) completing batch secondary revision of other pre-labeled files of each type in batches through the file name matching rule, and finally completing sample collection and labeling work.
Has the advantages that: compared with the prior art, the invention has the beneficial effects that: (1) aiming at a recognition classification or detection demand scene with a relatively fixed target state, in a sample collection stage, a target picture can be automatically and periodically captured for a long time, sample pictures with different time, different environmental states and different target states are captured, a pre-marked file (2) is automatically and quickly generated by referring to a marked file format and combining configuration information, and a batch secondary marking tool is provided to further reduce the manual processing cost; (3) the continuous snapshot of long period can compensate for little sample, receives the big problem of environmental impact to a certain extent, and quick accurate mark can reduce sample mark processing cycle and promote the mark effect by a wide margin.
Detailed Description
The invention is described in further detail below with reference to specific embodiments and the attached drawings.
As shown in fig. 1 and fig. 2, the method for obtaining and rapidly labeling a sample with a relatively fixed target state according to the present invention includes that the target state is relatively fixed, that is, a target to be photographed is photographed by a certain preset position of a specific camera, the position information of the target in a photographed image is relatively fixed, the target state is allowed to change, but all different states can be classified into a limited number of states, the whole target in all different states is still in the image photographed at the preset position, and the target state changes infrequently, for example, the target state does not exceed 1 day/time in a year; the method specifically comprises the following steps:
(1) arranging a camera for shooting a target; specifically, the number of cameras, the types of the cameras, the installation positions of the cameras and the targets for shooting are reasonably selected according to the shot targets and possible state changes of the shot targets, the positions of the targets and surrounding environment factors, the targets which are taken by the cameras are determined, the targets are finally determined according to which camera is shot at which preset position, and installation or routing inspection design of the cameras is completed. The camera types include but are not limited to a gun camera with a fixed installation angle, a tripod head with a plurality of preset points, an unmanned aerial vehicle capable of fixed-point snapshot, or an orbital robot or a non-orbital robot; the installation position of the cloud platform machine can be shot to different targets by changing the preset points of the cameras, the snapshot routing inspection routes of the unmanned aerial vehicle and the robot mean that the shooting points through which the line inspection passes can be shot to different targets, and for convenience of brief description, specific camera forms are not distinguished below, and the cameras with different preset positions are used as examples for description. The preset position refers to a shooting state of the camera determined by lens information such as focal length, focal point and the like corresponding to the position after the camera holder position or the moving position is set, the camera can set and store one or more preset positions, and the image acquisition equipment can control the camera to rotate to each preset position by calling a related interface;
as shown in fig. 2, the number of the targets and the cameras is one or more, and the cameras and the targets are in one-to-one or one-to-many relationship; when the camera and the target are in a one-to-many relationship, shooting a plurality of different targets at the same preset position by a single camera, and shooting a plurality of different targets by the single camera when the single camera is rotated to different preset positions;
(2) the picture acquisition equipment is in communication connection with each camera and acquires pictures shot by each camera; specifically, the image acquisition device refers to a device including a CPU, a memory, a hard disk, a display, a network interface, and the like, and may be an embedded device or a server computer. The image acquisition equipment is in communication connection with each camera through a wired network or a wireless network; specifically, the image acquisition device and the camera can communicate with each other by means of a switch or a wireless gateway or a Network Video Recorder (NVR). The image acquisition equipment controls the camera to rotate to a preset position and controls the camera to capture a target image, and the captured target image is stored in a local hard disk through network transmission;
(3) the picture acquisition equipment configures all target information, camera information, associated targets, cameras and preset camera position information; specifically, the picture acquisition equipment comprises a configuration tool, and can configure a target name, a camera name, camera network access parameter information, target and camera and preset position association information, an initial target state category and position information of the target in a picture shot corresponding to a preset position of the camera; the position information of the target in the shot picture corresponding to the preset position of the camera refers to: the method comprises the steps that a camera shoots a picture at a preset position, specific position key point information of a target in the picture is configured, a rectangular or polygonal frame is drawn in the picture by using a configuration tool to mark the target position, percentage information of key points of the frame relative to horizontal/vertical coordinates of the whole picture is stored in the configuration, and for example, coordinate information of the upper left corner and the lower right corner needs to be stored in the rectangular frame. The method comprises the following steps that (1) as a target can exist in a plurality of states in subsequent operation and the sizes of the targets in different states are possibly different, an initial target state type and position information thereof can be configured in a configuration stage;
the method comprises the steps that a configuration tool displays a captured target picture or a real-time video in real time according to configured cameras and preset position information, looks up and estimates the positions and shooting angles of all states of a target in the picture, or finally determines satisfactory camera installation and preset position setting by changing the installation position of the cameras or changing the preset position, draws all position frames of the target in the corresponding picture or video and specifies the current state type of the target, obtains key position information of the target in the picture according to picture frame information, and stores all configuration information such as names, associations, positions and initial types into a database or a configuration file;
(4) the image acquisition equipment starts an automatic snapshot tool to generate a sample image and a pre-marked file; before starting an automatic capturing tool contained in the image acquisition equipment, the image acquisition equipment sets a capturing period, periodically and sequentially controls each camera to rotate to each preset position according to configuration information to capture and generate sample images containing each target, then automatically generates corresponding pre-labeled files, and creates a date directory in the image acquisition equipment to store the sample images and the pre-labeled files according to dates; the sample picture and the corresponding pre-labeled file have the same name, the naming rule can be similar to the current time, camera name and preset position number, the labeled file is generated according to the labeled file format required by the training of the image recognition model and is filled with key information, and the key information comprises a target state classification category and target position information;
the file name and the content information of the label file in the generated sample picture/label file come from the configuration information in the step (3), the classification category of the target state in the label file refers to that all targets and different states thereof are considered at the initial stage of configuration to form the naming and the numbering of all classification categories, and when the label file is automatically generated, the classification category of the target state uniformly fills the corresponding initial target state category of the targets in the configuration information;
(5) manually checking automatically-captured sample pictures, carrying out batch secondary labeling on the pictures with the changed target states, and finally completing sample collection and rapid labeling, wherein the sample collection and rapid labeling are performed because the picture acquisition equipment automatically captures a few moments with the changed target states in the whole period, and the corresponding target pictures need to be manually checked; the method specifically comprises the following steps:
(5.1) determining a sample picture with a target state inconsistent with the initially configured target state, and recording the time when the target state changes;
(5.2) sorting and selecting each type of inconsistent pictures and pre-labeled files thereof;
(5.3) selecting one picture in each category and the labeled file thereof, and resetting the category and the position information of the target category by using a batch secondary labeling tool; the batch secondary labeling tool is a software tool with functions of sample picture fast classification/position information browsing, secondary classification, secondary labeling target position and file matching rule setting fast secondary labeling, and can be operated in image acquisition equipment and other computers;
and (5.4) completing batch secondary revision of each type of other pre-labeled files in batches through file name matching rules, and finally completing sample collection and labeling work.
Further, on the basis of the steps, the specific operation process can optimize the man-machine interaction mode of each tool, if the target state is extremely limited (if only two target states exist), if the image acquisition equipment acquires the current real-time states of all targets in real time through other equipment or additional equipment, two different state type classifications and corresponding position coordinate information can be configured for each target, and the current target state type and the corresponding position coordinate information are directly filled in when the pre-marked file is generated, so that the generated sample and the marked file do not need secondary processing, and the labor cost is further liberated.
In the actual engineering field, the image acquisition equipment and the camera are arranged to undertake the work of sample collection and quick labeling during debugging, and also undertake the work of video display and image identification after commissioning.
The following is a specific embodiment of the present invention:
taking an outdoor AIS disconnecting link state identification scene in a power industry transformer substation as an example, assuming that each disconnecting link only needs to identify two states of disconnecting link opening and closing, how to obtain a large number of AIS disconnecting link samples and quickly label are given below. The transformer substation is inspected on site, the number and the types of the disconnecting links are included, all targets to be identified and different states of the targets are divided into a plurality of identification categories, cameras are reasonably selected and arranged at installation positions, and finally all the cameras can be accessed by picture acquisition equipment through networking.
According to the shooting state of the target in the camera, or the physical position and angle of the camera are actually adjusted, or the preset position of a camera holder is changed, all target switches (including potential switch-off states or switch-on states of the switches) can be shot clearly and completely, and the camera preset position corresponding to all the targets is determined for shooting.
The method comprises the steps of simply naming each camera and a disconnecting link target to be identified (each interval disconnecting link comprises 3 single-phase targets), associating the target with the camera and the preset position of the camera, checking pictures shot by the camera corresponding to the target one by one and the preset position of the camera by a configuration tool, setting the identification type corresponding to the current target state, manually drawing target position frame information, finally completing configuration of all naming, association, current type, position information and the like, and storing the configuration in a database.
After configuration is completed, an automatic snapshot tool is opened, each round of snapshot interval time is manually set, then an automatic snapshot function is started, the tool periodically and sequentially controls each Camera to rotate to each preset position for snapshot according to configuration information to generate sample pictures containing each target, corresponding pre-labeled files are automatically generated and are respectively stored in a hard disk according to a date directory, picture naming rules are 20201229_195012_ Camera01_ pos01.jpg, same-name labeled files are generated according to a labeled file format, and the number of targets, the initial state category of each target and the position information of the targets in the pictures are filled in the pictures according to the configuration information.
After enough sample pictures are captured, if the sample pictures are captured for 5 days in one round in each hour, the captured pictures and the pre-labeled files can be copied into other high-performance computers, the background can be monitored by a scada in a station to retrieve and check the time point of the change of the knife switch position signals, then the pictures in the time period are manually checked to check whether the knife switch of the actual picture is inconsistent with the initial target state, the state of the knife switch of the transformer substation is generally changed rarely, inconsistent pictures are not particularly numerous, the knife switch pictures inconsistent with the initial position state and the pre-labeled files thereof can be independently selected, the pictures inconsistent with the initial state of each target knife switch reassign the target state type and the secondary labeled new position frame (each type only needs to be operated once), and then other inconsistent labeled files are quickly modified in batches through the file name matching rule.
And finally completing sample collection and quick labeling after all sample pictures are checked and secondarily labeled.