CN115712746A

CN115712746A - Image sample labeling method and device, storage medium and electronic equipment

Info

Publication number: CN115712746A
Application number: CN202211400674.1A
Authority: CN
Inventors: 边成; 李永会
Original assignee: Douyin Vision Co Ltd
Current assignee: Douyin Vision Co Ltd
Priority date: 2022-11-09
Filing date: 2022-11-09
Publication date: 2023-02-24

Abstract

The method comprises the steps of displaying a target image pair, responding to selection operation of a target image area of any one of the target image pair, synchronously determining image blocks to be annotated corresponding to the target image area in a first image and a second image, and then annotating the image blocks to be annotated to obtain an image annotation result of the target image pair.

Description

Image sample labeling method and device, storage medium and electronic equipment

Technical Field

The present disclosure relates to the field of computer technologies, and in particular, to an image sample labeling method and apparatus, a storage medium, and an electronic device.

Background

Generally, before the model training, a large amount of training sample data needs to be prepared, so as to adjust the model parameters through the training sample data. However, training data samples are typically labeled manually. Particularly for the labeling of the image semantic segmentation data set, a large amount of repetitive image semantic labeling is needed by manually labeling the sample, so that the sample labeling efficiency is reduced.

Disclosure of Invention

This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

In a first aspect, the present disclosure provides an image sample labeling method, including:

displaying a target image pair, wherein the target image pair comprises a first image and a second image which are obtained by shooting the same scene in different shooting modes;

in response to a selection operation of a target image area of any one of the target image pairs, synchronously determining image blocks to be annotated in the first image and the second image, wherein the image blocks to be annotated correspond to the target image area;

and labeling the image blocks to be labeled to obtain an image labeling result of the target image pair.

In a second aspect, the present disclosure provides an image sample labeling apparatus, including:

the display module is configured to display a target image pair, wherein the target image pair comprises a first image and a second image which are obtained by shooting the same scene in different shooting modes;

the selection module is configured to respond to a selection operation of a target image area of any one of the target image pairs, and synchronously determine image blocks to be annotated in the first image and the second image, wherein the image blocks to be annotated correspond to the target image area;

and the marking module is configured to mark the image blocks to be marked and obtain the image marking result of the target image pair.

In a third aspect, the present disclosure provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processing apparatus, performs the steps of the method of the first aspect.

In a fourth aspect, the present disclosure provides an electronic device comprising:

a storage device having a computer program stored thereon;

processing means for executing the computer program in the storage means to carry out the steps of the method of the first aspect.

Based on the technical scheme, the target image pair is displayed, the selection operation of the target image area of any image in the target image pair is responded, the image blocks to be labeled corresponding to the target image area are synchronously determined in the first image and the second image, then the image blocks to be labeled are labeled, and the image labeling result of the target image pair is obtained.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numbers refer to the same or similar elements. It should be understood that the drawings are schematic and that elements and features are not necessarily drawn to scale. In the drawings:

FIG. 1 is a flow diagram illustrating a method for image sample annotation in accordance with an exemplary embodiment.

FIG. 2 is a schematic diagram of a user interface of an annotation tool shown in accordance with an exemplary embodiment.

FIG. 3 is a user interface diagram of an annotation tool shown in accordance with another exemplary embodiment.

Fig. 4 is a schematic diagram illustrating labeling of an image block to be labeled according to an exemplary embodiment.

FIG. 5 is a diagram illustrating image annotation results according to an exemplary embodiment.

FIG. 6 is a flowchart illustrating a method of image sample annotation, according to another exemplary embodiment.

FIG. 7 is a flowchart illustrating a first image block according to an exemplary embodiment.

FIG. 8 is a flow diagram illustrating the determination of a target color according to an exemplary embodiment.

FIG. 9 is a flowchart illustrating an image sample annotation method according to yet another exemplary embodiment.

FIG. 10 is a block diagram illustrating the connection of modules of an image sample annotation device according to an exemplary embodiment.

Fig. 11 is a schematic structural diagram of an electronic device shown in accordance with an example embodiment.

Detailed Description

Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it is to be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein, but rather are provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and the embodiments of the disclosure are for illustration purposes only and are not intended to limit the scope of the disclosure.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the terms "first", "second", and the like in the present disclosure are only used for distinguishing different devices, modules or units, and are not used for limiting the order or interdependence of the functions performed by the devices, modules or units.

It is noted that references to "a" or "an" in this disclosure are intended to be illustrative rather than limiting, and that those skilled in the art will appreciate that references to "one or more" are intended to be exemplary and not limiting unless the context clearly indicates otherwise.

The names of messages or information exchanged between devices in the embodiments of the present disclosure are for illustrative purposes only, and are not intended to limit the scope of the messages or information.

It is understood that before the technical solutions disclosed in the embodiments of the present disclosure are used, the type, the use range, the use scene, etc. of the personal information related to the present disclosure should be informed to the user and obtain the authorization of the user through a proper manner according to the relevant laws and regulations.

For example, in response to receiving a user's active request, prompt information is sent to the user to explicitly prompt the user that the requested operation to be performed would require acquisition and use of personal information to the user. Thus, the user can autonomously select whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operations of the technical solution of the present disclosure, according to the prompt information.

As an optional but non-limiting implementation manner, in response to receiving an active request from the user, the manner of sending the prompt information to the user may be, for example, a pop-up window, and the prompt information may be presented in a text manner in the pop-up window. In addition, a selection control for providing personal information to the electronic device by the user's selection of "agreeing" or "disagreeing" can be carried in the pop-up window.

It is understood that the above notification and user authorization process is only illustrative and not limiting, and other ways of satisfying relevant laws and regulations may be applied to the implementation of the present disclosure.

Meanwhile, it is understood that the data involved in the present technical solution (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws and regulations and the related regulations.

FIG. 1 is a flow diagram illustrating a method for image sample annotation in accordance with an exemplary embodiment. As shown in fig. 1, an embodiment of the present disclosure provides an image sample annotation method, which may be performed by an electronic device, and in particular, may be performed by an image sample annotation apparatus, which may be implemented by software and/or hardware and configured in the electronic device. As shown in fig. 1, the method may include the following steps.

In step 110, a target image pair is displayed, wherein the target image pair includes a first image and a second image obtained by shooting the same scene in different shooting modes.

Here, the target image pair includes a first image and a second image obtained by shooting the same scene in different shooting modes. For example, the first image may be an RGB image obtained by filming a scene with a visible light camera, the second image may be a thermal image (T-image) obtained by filming the scene with a thermal imaging camera, and the target image pair is an RGB image — a thermal image. Of course, the second image may also be a depth image (D-image) obtained by shooting the scene with a depth camera, and the image pair is an RGB image — a depth image. Or the second image may also be a point cloud image of the scene acquired by using a laser radar, and the image pair is an RGB image — a point cloud image.

The annotation tool can respectively display the first image and the second image in the target image pair in different display areas. It should be understood that the annotation tool refers to an application program for implementing the image sample annotation method proposed in the present disclosure. When the image sample is labeled, the target image pair can be loaded through the labeling tool, and the first image and the second image are simultaneously displayed in different image areas.

FIG. 2 is a schematic diagram of a user interface of an annotation tool shown in accordance with an exemplary embodiment. As shown in fig. 2, the annotation tool 200 includes a toolbar 201, a first display area 202, and a second display area 203, and a first image 204 is displayed in the first display area 202, and a second image 205 is displayed in the second display area 203.

In step 120, in response to a selection operation on a target image area of any one of the target image pairs, determining an image block to be annotated in the first image and the second image, which corresponds to the target image area, synchronously.

Here, the selection operation may be performed using a specific selection tool in the toolbar 201 as shown in fig. 2, or may be a click operation, a touch operation, a gesture operation, or the like. The target image area may be an image to be labeled, such as "car", "motorcycle", "bicycle", "person", "lane line", and "traffic light" in the target image pair, and so on.

FIG. 3 is a schematic diagram of a user interface of an annotation tool shown in accordance with another exemplary embodiment. As shown in fig. 3, the annotation tool 200 includes a toolbar 201, a first display area 202, and a second display area 203, and a first image 204 is displayed in the first display area 202, and a second image 205 is displayed in the second display area 203. When the annotator needs to annotate a target image region in the target image pair, the annotator can select the first target image region 206 in the second image 205 and, in synchronization, also the second target image region 207 in the first image 204. The first target image area 206 and the second target image area 207 refer to the consistent pixel areas in the first image 204 and the second image 205. It should be noted that the pixel area does not change with the enlargement and reduction of the image, and is determined by inherent image factors such as image size and resolution, and since the target image pair is an image of the same scene, the image coordinates of the same object on the first image and the second image are identical.

It should be understood that in one annotation process, the annotator can select the target image region on the first image 204 or the second image 205, respectively, and whether the selection operation is performed on one of the first image 204 or the second image 205, the selection operation is synchronized to the other image. Therefore, the annotator can repeatedly compare the first image 204 and the second image 205 to determine the finished image blocks to be annotated in the first image 204 and the second image 205. For example, when a portion of an object is unclear in the first image 204, the annotator may select the unclear portion in the second image 205.

It is noted that the selection operation for the first image 204 or for the second image 205 is synchronized, but the other operations for the first image 204 or for the second image 205 may be independent, i.e. the first display area 202 or the second display area 203 may be performed independently for the other operations for the first display area 202 or for the second display area 203. For example, when the first image 204 in the first display region 202 needs to be enlarged, the first image 204 in the first display region 202 may be individually enlarged, and the second image in the second display region 203 is not enlarged.

In step 130, the image blocks to be labeled are labeled to obtain an image labeling result of the target image pair.

Here, the image labeling result may be obtained by labeling the image blocks to be labeled with different image semantic categories by using different colors. For example, red labeling is used for an image block to be labeled of which the image semantic category is a vehicle, and green labeling is used for an image block to be labeled of which the image semantic category is a person.

Fig. 4 is a schematic diagram illustrating labeling of an image block to be labeled according to an exemplary embodiment. As shown in fig. 4, the annotation tool 200 includes a toolbar 201, a first display area 202, and a second display area 203, and a first image 204 is displayed in the first display area 202, and a second image 205 is displayed in the second display area 203. When the annotator needs to annotate a target image region in the target image pair, the annotator can select the first target image region 206 in the second image 205 and, in synchronization, also the second target image region 207 in the first image 204. Then, the annotator can select a preset color to annotate the first target image area 206 or the second target image area 207. It should be understood that the colors of the annotations are also synchronized, and only one of the first target image area 206 or the second target image area 207 needs to be annotated.

FIG. 5 is a diagram illustrating image annotation results according to an exemplary embodiment. As shown in fig. 5, objects of different image semantic categories, such as "car", "motorcycle", "bicycle", "person", "lane line", and "traffic light", in the target image pair are labeled with different colors.

Of course, in other embodiments, the image blocks to be labeled having different image semantic categories may also be labeled with characters, so as to obtain an image labeling result.

Therefore, by displaying the target image pair, responding to the selection operation of the target image area of any image in the target image pair, synchronously determining the image blocks to be labeled corresponding to the target image area in the first image and the second image, and then labeling the image blocks to be labeled to obtain the image labeling result of the target image pair, the labeling efficiency of the image sample can be improved, and the labeling can be carried out by comparing the first image and the second image, so that the sample labeling result can be more accurate.

FIG. 6 is a flowchart illustrating a method of image sample annotation, according to another exemplary embodiment. As shown in fig. 6, after step 120, the image sample labeling method may further include the steps of:

in step 601, in response to an automatic selection instruction, determining a first pixel similarity between the to-be-annotated image block in the first image and an adjacent image.

Here, the automatic selection instruction may be triggered by a preset operation, which may include, but is not limited to, clicking a preset key, presetting an operation track, and the like. After detecting the automatic selection instruction, the electronic device responds to the automatic selection instruction and calculates a first pixel similarity between the image block to be marked in the first image and the adjacent image.

The adjacent pixels may be pixels adjacent to an edge area of the image to be annotated in the first image, and certainly, the adjacent pixels may also be pixels having a preset pixel distance from the edge area of the image to be annotated.

It is worth mentioning that the first pixel similarity may be obtained by calculating a gray value between pixels.

In step 602, a first image block is determined in the first image based on the first pixel similarity, where the first image block includes the to-be-annotated image block in the first image and a pixel of which the first pixel similarity is greater than a first preset threshold.

Here, after the first pixel similarity is calculated, the image block to be annotated in the first image and the pixels of which the first pixel similarity is greater than a first preset threshold are determined as the first image block. The pixels with the first pixel similarity greater than the first preset threshold are pixels similar to pixels in the edge area of the image to be labeled.

The first preset threshold may be set according to actual conditions, and is not specifically limited in the embodiment of the present disclosure.

FIG. 7 is a flowchart illustrating a first image block according to an example embodiment. As shown in fig. 7, in the first image 701, pixels located inside the first image block 703 and outside the to-be-labeled image block 702 are pixels whose first pixel similarity is greater than a first preset threshold.

In step 603, a second pixel similarity between the image block to be annotated in the second image and an adjacent image is determined.

Here, step 603 is consistent with step 601, and the execution logic of step 603 may refer to the related description of step 601, which is not described herein again.

In step 604, a second image block is determined in the second image based on the second pixel similarity, where the second image block includes the image block to be annotated in the second image and a pixel of which the second pixel similarity is greater than a second preset threshold.

Here, step 604 is consistent with step 602, and the execution logic of step 604 may refer to the related description of step 602, which is not described herein again.

It should be noted that steps 603 to 604 are executed synchronously with steps 601 to 602, that is, when the auto-select command is detected, the electronic device starts to execute

steps

601 and 603 synchronously.

In step 605, based on the first image block and the second image block, a new image block to be annotated is determined in the first image and the second image.

Here, the first image block and the second image block may or may not be identical, depending on the specific image pair. At this time, the first image block and the second image block may be respectively displayed on the first image and the second image, and then a new image block to be annotated may be determined based on a selection instruction of an annotator on the first image block or the second image block. For example, an annotator can select one image from the first image block and the second image block as a new image block to be annotated by comparing the first image block and the second image block.

Therefore, in the embodiment, the image blocks to be annotated can be preliminarily determined in the first image and the second image through selection operation, then more precise image blocks to be annotated can be determined through an automatic selection instruction, and the annotation speed of the image sample can be greatly improved through the automatic selection instruction.

In some implementation embodiments, in step 130, the image block to be annotated may be annotated based on a target color matched with an image semantic category to which the image block to be annotated belongs, so as to obtain an image annotation result of the target image pair.

Here, the image labeling result can be obtained by labeling the image blocks to be labeled with different image semantic categories by using different colors. For example, red labeling is used for an image block to be labeled of which the image semantic category is a car, and green labeling is used for an image block to be labeled of which the image semantic category is a person.

It should be noted that, the annotating person can trigger the electronic device to annotate the image block to be annotated with the target color through the annotation instruction, and the annotating person does not need to select a color for each image block to be annotated.

FIG. 8 is a flow diagram illustrating the determination of a target color according to an exemplary embodiment. As shown in fig. 8, in some implementations, the target color can be determined by:

in step 801, a first similarity between the image block to be annotated of the first image and a preset image in a database is determined.

Here, the preset image in the database refers to an image stored in the database, and images of different image semantic categories, for example, images of different image semantic categories such as "car", "motorcycle", "bicycle", "person", "lane line", and "traffic light" may be stored in the database.

The electronic equipment can respond to the annotation instruction and calculate a first similarity between the image block to be annotated of the first image and a preset image in the database. Wherein the first similarity may be obtained by calculating a gray value, a color, and the like of the pixel.

In step 802, a second similarity between the image block to be annotated of the second image and a preset image in the database is determined.

Here, the electronic device may calculate a second similarity between the image block to be annotated of the second image and the preset image in the database in response to the annotation instruction. Wherein the second similarity may be obtained by calculating a gray value, a color, and the like of the pixel. It is noted that step 802 and step 801 may be performed synchronously.

In step 803, based on the first similarity and the second similarity, an image semantic category to which the image block to be annotated belongs is determined.

Here, after the first similarity and the second similarity are obtained through calculation, the image semantic category to which the preset image corresponding to the maximum value of the first similarity and the second similarity belongs may be determined as the image semantic category to which the labeled image block belongs.

In step 804, the target color is determined according to the image semantic category.

Here, the target color may be determined according to a mapping relationship between different semantic categories of the image and different colors, so that different objects in the target image pair are labeled with different colors.

Therefore, the target color is determined through the similarity, and an annotating person does not need to select a color for each image block to be annotated to annotate, so that the annotation efficiency of the image sample can be greatly improved.

FIG. 9 is a flowchart illustrating an image sample annotation process according to yet another exemplary embodiment. As shown in fig. 9, in some implementations, the image sample labeling method may further include the following steps:

in step 910, in response to a verification instruction, sending the image annotation result of the target image pair to an annotation verification end, so that the annotation verification end verifies the image annotation result of the target image pair.

Here, after completing the annotation of the target image pair, the annotation tool may send the image annotation result of the target image pair to the annotation verification end in response to the verification instruction. And the marking verification end is used for verifying the image marking result of the target image pair.

As some examples, the annotation verifying end may display the image annotation result, and then receive a verification result of the verifier for the image annotation result of the target image pair. The verification personnel can obtain the verification result according to whether the label in the image labeling result displayed by the labeling verification end is matched with the image corresponding to the label.

As another example, the annotation verification terminal may classify the object included in the target image pair through an image recognition algorithm, and then determine whether semantic categories of the images to which the object included in the target image pair belongs are consistent according to the obtained classification result to obtain a verification result.

In step 920, in response to an instruction sent by the annotation checking terminal and used for indicating that the image annotation result of the target image pair passes, storing the image annotation result in an image sample library.

Here, when the annotation checking end determines that the image annotation result of the target image pair is consistent with the object type included in the target image pair, an instruction for indicating that the image annotation result of the target image pair passes is sent to the electronic device, and the electronic device stores the annotation result of the target image pair in the image sample library in response to the instruction.

And if the image annotation result of the target image pair does not pass, prompting an annotation person to annotate the target image pair again in response to an instruction which is sent by the annotation checking terminal and used for indicating that the image annotation result of the target image pair does not pass.

Therefore, the image marking result of the target image pair is verified through the marking verification end, and the marked image sample data can be ensured to be accurate.

FIG. 10 is a block diagram illustrating the connection of modules of an image sample annotation device, according to an exemplary embodiment. As shown in fig. 10, an embodiment of the present disclosure provides an image sample labeling apparatus, where the apparatus 1000 includes:

a presentation module 1010 configured to present a target image pair, wherein the target image pair includes a first image and a second image obtained by shooting a same scene in different shooting manners;

a selecting module 1020, configured to, in response to a selection operation on a target image region of any one of the target image pairs, synchronously determine, in the first image and the second image, an image block to be annotated corresponding to the target image region;

an labeling module 1030 configured to label the image block to be labeled, and obtain an image labeling result of the target image pair.

Optionally, the apparatus 1000 further comprises:

the first determination module is configured to respond to an automatic selection instruction and determine a first pixel similarity between the image block to be annotated in the first image and an adjacent image;

a second determining module, configured to determine a first image block in the first image based on the first pixel similarity, where the first image block includes the image block to be annotated in the first image and a pixel of which the first pixel similarity is greater than a first preset threshold;

the third determining module is configured to determine a second pixel similarity between the image block to be annotated in the second image and an adjacent image;

a fourth determining module, configured to determine a second image block in the second image based on the second pixel similarity, where the second image block includes the image block to be annotated in the second image and a pixel of which the second pixel similarity is greater than a second preset threshold;

and the fifth determining module is configured to determine a new image block to be annotated in the first image and the second image based on the first image block and the second image block.

Optionally, the labeling module 1030 includes:

and the color labeling unit is configured to label the image blocks to be labeled based on a target color matched with the image semantic category to which the image blocks to be labeled belong, and obtain an image labeling result of the target image pair.

Optionally, the color labeling unit is specifically configured to:

determining a first similarity between the image block to be annotated of the first image and a preset image in a database;

determining a second similarity between the image block to be marked of the second image and a preset image in the database;

determining the image semantic category to which the image block to be annotated belongs based on the first similarity and the second similarity;

and determining the target color according to the image semantic category.

Optionally, the apparatus 1000 further comprises:

the sending module is configured to respond to a verification instruction and send the image labeling result of the target image pair to a labeling verification end so that the labeling verification end verifies the image labeling result of the target image pair;

the storage module is configured to respond to an instruction which is sent by the annotation checking terminal and used for indicating that the image annotation result of the target image pair passes, and store the image annotation result in an image sample library.

With respect to the apparatus 1000 in the above embodiment, the method logic executed by each functional module has been described in detail in relation to the method, and is not described herein again.

Referring now to FIG. 1100, a block diagram of an electronic device 1100 suitable for use to implement embodiments of the present disclosure is shown. The terminal device in the embodiments of the present disclosure may include, but is not limited to, a mobile terminal such as a mobile phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a vehicle terminal (e.g., a car navigation terminal), and the like, and a stationary terminal such as a digital TV, a desktop computer, and the like. The electronic device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the use range of the embodiment of the present disclosure.

As shown in fig. 11, the electronic device 1100 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 1101 that may perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM) 1102 or a program loaded from a storage means 1108 into a Random Access Memory (RAM) 1103. In the RAM 1103, various programs and data necessary for the operation of the electronic device 1100 are also stored. The processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.

Generally, the following devices may be connected to the I/O interface 1105: input devices 1106 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; output devices 1107 including, for example, liquid Crystal Displays (LCDs), speakers, vibrators, and the like; storage devices 1108, including, for example, magnetic tape, hard disk, and the like; and a communication device 1109. The communication means 1109 may allow the electronic device 1100 to communicate wirelessly or by wire with other devices to exchange data. While fig. 11 illustrates an electronic device 1100 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may alternatively be implemented or provided.

In particular, the processes described above with reference to the flow diagrams may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program carried on a non-transitory computer readable medium, the computer program containing program code for performing the method illustrated by the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communication device 1109, or installed from the storage device 1108, or installed from the ROM 1102. When executed by the processing device 1101, the computer program performs the above-described functions defined in the methods of the embodiments of the present disclosure.

It should be noted that the computer readable medium in the present disclosure can be a computer readable signal medium or a computer readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer readable signal medium may comprise a propagated data signal with computer readable program code embodied therein, either in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, optical cables, RF (radio frequency), etc., or any suitable combination of the foregoing.

In some implementations, the electronic devices may communicate using any currently known or future developed network Protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communications network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network.

The computer readable medium may be embodied in the electronic device; or may be separate and not incorporated into the electronic device.

The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: displaying a target image pair, wherein the target image pair comprises a first image and a second image which are obtained by shooting the same scene in different shooting modes; in response to a selection operation of a target image area of any one of the target image pairs, synchronously determining image blocks to be annotated in the first image and the second image, wherein the image blocks to be annotated correspond to the target image area; and labeling the image blocks to be labeled to obtain an image labeling result of the target image pair.

Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including but not limited to an object oriented programming language such as Java, smalltalk, C + +, and including conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet service provider).

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

The modules described in the embodiments of the present disclosure may be implemented by software or hardware. Wherein the name of a module does not in some cases constitute a limitation on the module itself.

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems on a chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

The foregoing description is only exemplary of the preferred embodiments of the disclosure and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the disclosure herein is not limited to the particular combination of features described above, but also encompasses other embodiments in which any combination of the features described above or their equivalents does not depart from the spirit of the disclosure. For example, the above features and the technical features disclosed in the present disclosure (but not limited to) having similar functions are replaced with each other to form the technical solution.

Further, while operations are depicted in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single embodiment. Conversely, various features that are described in the context of a single embodiment can also be implemented in multiple embodiments separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims. With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. An image sample labeling method is characterized by comprising the following steps:

2. The method according to claim 1, wherein after the synchronizing determines the image block to be annotated in the first image and the second image corresponding to the target image area, the method further comprises:

responding to an automatic selection instruction, and determining first pixel similarity between the image block to be marked in the first image and an adjacent image;

determining a first image block in the first image based on the first pixel similarity, wherein the first image block comprises the image block to be marked in the first image and pixels with the first pixel similarity larger than a first preset threshold;

determining a second pixel similarity between the image block to be marked in the second image and an adjacent image;

determining a second image block in the second image based on the second pixel similarity, wherein the second image block comprises the image block to be annotated in the second image and pixels of which the second pixel similarity is greater than a second preset threshold;

and determining a new image block to be annotated in the first image and the second image based on the first image block and the second image block.

3. The method according to claim 1, wherein the labeling the image blocks to be labeled to obtain the image labeling result of the target image pair includes:

and labeling the image blocks to be labeled based on the target color matched with the image semantic category to which the image blocks to be labeled belong, and obtaining an image labeling result of the target image pair.

4. The method of claim 3, wherein the target color is determined by:

and determining the target color according to the image semantic category.

5. The method of claim 1, further comprising:

responding to a checking instruction, and sending the image annotation result of the target image pair to an annotation checking end so as to enable the annotation checking end to check the image annotation result of the target image pair;

and responding to an instruction which is sent by the annotation checking terminal and used for indicating that the image annotation result of the target image pair passes, and storing the image annotation result in an image sample library.

6. An image sample labeling apparatus, comprising:

7. The apparatus of claim 6, further comprising:

the first determination module is configured to respond to an automatic selection instruction, and determine a first pixel similarity between the image block to be annotated in the first image and an adjacent image;

a third determining module, configured to determine a second pixel similarity between the image block to be annotated in the second image and an adjacent image;

a fourth determining module, configured to determine a second image block in the second image based on the second pixel similarity, where the second image block includes the image block to be annotated in the second image and a pixel whose second pixel similarity is greater than a second preset threshold;

8. The apparatus of claim 6, wherein the labeling module comprises:

9. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by processing means, carries out the steps of the method of any one of claims 1 to 5.

10. An electronic device, comprising:

a storage device having a computer program stored thereon;

processing means for executing the computer program in the storage means to carry out the steps of the method according to any one of claims 1 to 5.