WO2022267279A1

WO2022267279A1 - Data annotation method and apparatus, and electronic device and storage medium

Info

Publication number: WO2022267279A1
Application number: PCT/CN2021/126182
Authority: WO
Inventors: 牛菜梅
Original assignee: 上海商汤科技开发有限公司
Priority date: 2021-06-24
Filing date: 2021-10-25
Publication date: 2022-12-29
Also published as: CN113407083A

Abstract

The present disclosure relates to a data annotation method and apparatus, and an electronic device and a storage medium, which method is applied to the field of intelligent vehicles. The method comprises: acquiring an object to be annotated; presenting, to a user interaction interface, a target annotation tool, which corresponds to an annotation task type of said object, among a plurality of annotation tools; and in response to an annotation operation that is executed on the basis of the target annotation tool, obtaining annotation information of said object. By means of the embodiments of the present disclosure, an annotation tool can be quickly and accurately provided according to the type of an annotation task, and a data annotation scheme required in the research and development process of intelligent vehicles can be clear and definite for traditional vehicle enterprises, so as to make up for the shortcomings of data annotation during the combination of the traditional vehicle enterprises and artificial intelligence technology, thereby reducing the development resources invested in the research and development process.

Description

A data labeling method and device, electronic equipment and storage medium

This application claims the priority of the Chinese patent application filed on June 24, 2021, with the application number 202110704502.2, and the title of the invention is "a data labeling method and device, electronic equipment, and storage medium", the entire contents of which are incorporated herein by reference. Applying.

technical field

The present disclosure relates to the field of computer technology, and in particular to a data labeling method and device, electronic equipment, and a storage medium.

Background technique

With the application of artificial intelligence technology in traditional industrial fields, the automotive industry has also ushered in new developments. New directions such as autonomous driving and intelligent cockpit are gradually added to the field of automotive research and development. In the research of these directions, data annotation is inseparable.

However, for traditional car companies, due to the lack of systematic artificial intelligence solutions, the enterprise's internal data labeling solutions are too fragmented, and they often only formulate a single labeling solution when facing a single detection and recognition task; Due to the limitations, it is difficult to quickly and accurately create the required labeling tool platform and clarify the data labeling scheme required in the research and development process according to the needs of the company's own autonomous driving and intelligent cockpit research and development. Therefore, a complete and systematic data labeling solution is urgently needed.

Contents of the invention

The present disclosure proposes a data labeling technical solution.

According to an aspect of the present disclosure, a data labeling method is provided, which is applied in the field of smart cars, including:

Get the object to be marked;

Presenting a target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to a user interaction interface;

In response to the labeling operation performed based on the target labeling tool, labeling information of the object to be labelled is obtained.

In a possible implementation manner, the presenting the target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to a user interaction interface includes:

Receive a data field selection instruction input by a user, the data field selection instruction is used to determine the target data field to which the object to be marked belongs, and the target data field includes the smart car field;

According to the determined target data field, a target labeling tool for labeling objects to be labeled in the target data field is determined from the tool set, and the use authority of the target labeling tool is enabled, and displayed on the labeling interface for users to use.

In a possible implementation manner, after acquiring the object to be labeled, the method further includes:

In response to the user's selection operation on the plurality of annotation tools, the annotation tool selected by the user is used as the target annotation tool.

Identifying the labeling task type of the object to be labeled to obtain the target labeling task type;

According to the preset corresponding relationship between the labeling task type and the labeling tool, a labeling tool having a corresponding relationship with the target labeling task type is determined as the target labeling tool.

In a possible implementation manner, the obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the position labeling operation performed based on the target labeling tool, using the marked position as the position of the marked target object;

Use the pre-set default attributes as the attributes of the marked target object.

In a possible implementation manner, the labeling task type of the object to be labeled includes a lane line labeling task, and the target labeling tools corresponding to the lane line labeling task include a line tool and a lane line attribute labeling tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the line drawing operation based on the line tool, using the area indicated by the drawn line as the location of the labeled lane line;

In response to the lane line attribute selection operation based on the lane line attribute labeling tool, the attribute of the marked lane line is determined.

In a possible implementation manner, the tagging task type of the object to be tagged includes a drivable area tagging task, and the target tagging tool corresponding to the drivable area tagging task includes a polygon tool;

In response to the polygon drawing operation based on the polygon tool, the area where the drawn polygon is located is used as the location of the marked drivable area.

In a possible implementation manner, the labeling task type of the object to be labeled includes a target object labeling task, and the target labeling tools corresponding to the target object labeling task include a rectangular frame tool and a target object attribute labeling tool;

In response to the rectangle frame drawing operation based on the rectangle frame tool, the area surrounded by the drawn rectangle frame is used as the location of the marked target object;

In response to the attribute selection operation of the target object attribute labeling tool, the attribute of the target object marked among the objects to be marked is determined.

In a possible implementation, the object to be labeled is a point cloud image, the labeling task type includes a point cloud object labeling task, and the target labeling tool corresponding to the point cloud object labeling task includes a plurality of point cloud objects A frame selection tool, the multiple point cloud object frame selection tools correspond to different point cloud object attributes;

In response to a point cloud object marquee tool selection operation, determine a target point cloud object marquee tool in the plurality of point cloud object marquee tools;

In response to the frame selection operation based on the target point cloud object frame selection tool, the framed area is used as the location of the marked point cloud object, and the point cloud object attribute corresponding to the target point cloud frame selection tool is used as the label Properties of the point cloud object.

In a possible implementation, the labeling task type includes a passenger attribute labeling task, and the target labeling tools corresponding to the passenger attribute labeling task include a rectangular box tool and a passenger attribute labeling tool;

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the position of the marked passenger;

In response to a passenger attribute selection operation based on the passenger attribute annotation tool, at least one attribute of the annotated passenger is determined.

In a possible implementation, the tagging task type includes a face tagging task, and the target tagging tools corresponding to the face tagging task include a rectangular frame tool and a key point tagging tool;

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the location of the marked human face;

In response to the key point labeling operation based on the key point labeling tool, the key points of the human face in the labeled human face are determined.

In a possible implementation manner, the labeling task type includes a gesture labeling task, and the target labeling tools corresponding to the gesture labeling task include a rectangular frame tool and a key point labeling tool;

In response to the rectangle frame drawing operation based on the rectangle frame tool, the area surrounded by the drawn rectangle frame is used as the position of the marked hand;

In response to a key point labeling operation based on the key point labeling tool, key points of the marked hand are determined.

In a possible implementation, the labeling task type includes a driving fatigue labeling task, and the target labeling tools corresponding to the driving fatigue task include a rectangular frame tool and a fatigue attribute labeling tool;

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the position of the marked fatigue detection target;

In response to the key point labeling operation based on the fatigue property labeling tool, the property of the labelled fatigue detection target is determined.

In a possible implementation, the labeling task type includes a dangerous driving labeling task, and the target labeling tools corresponding to the dangerous driving task include a rectangular frame tool and a dangerous attribute labeling tool;

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the position of the marked dangerous driving detection target;

In response to the attribute selection operation based on the dangerous attribute tagging tool, the tagged attribute of the dangerous driving detection target is determined.

According to an aspect of the present disclosure, a data labeling device is provided, which is applied in the field of smart cars, including:

An annotation object acquisition unit, configured to acquire an object to be annotated;

A target labeling tool display unit, configured to present a target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to the user interface;

A labeling information determining unit, configured to obtain labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool.

In a possible implementation manner, the target labeling tool display unit includes:

A data field determination unit, configured to receive a data field selection instruction input by a user, the data field selection instruction is used to determine the target data field to which the object to be marked belongs, and the target data field includes the smart car field;

The target labeling tool display subunit is used to determine the target labeling tool for labeling objects to be labeled in the target data field from the tool set according to the determined target data field, and enable the use authority of the target labeling tool, which is displayed in the labeling The interface is for the user to use.

In a possible implementation manner, the device further includes:

The first target marking tool determination unit is configured to use the marking tool selected by the user as the target marking tool in response to the user's selection operation on the multiple marking tools.

In a possible implementation manner, the device further includes:

The second target labeling tool determining unit is configured to identify the labeling task type of the object to be labeled to obtain the target labeling task type; determine the target labeling task type according to the preset corresponding relationship between the labeling task type and the labeling tool The labeling task type has a corresponding labeling tool as the target labeling tool.

In a possible implementation manner, the marking information determining unit includes:

a target object position determining unit, configured to use the marked position as the marked position of the target object in response to the position marking operation performed based on the target marking tool;

The target object attribute determining unit is used to use the preset default attribute as the attribute of the marked target object.

The marking information determining unit is configured to respond to the line drawing operation based on the line tool, use the area indicated by the drawn line as the location of the marked lane line; respond to the lane of the tool based on the lane line attribute The line attribute selection operation determines the attribute of the marked lane line.

The marking information determining unit is configured to use the area where the drawn polygon is located as the position of the marked drivable area in response to the polygon drawing operation based on the polygon tool.

The marking information determination unit is configured to respond to the drawing operation of the rectangular frame based on the rectangular frame tool, and use the area surrounded by the drawn rectangular frame as the position of the marked target object; The attribute selection operation determines the attribute of the target object marked in the object to be marked.

The label information determining unit is configured to determine a target point cloud object frame selection tool among the plurality of point cloud object frame selection tools in response to a point cloud object frame selection tool selection operation; In the frame selection operation of the frame selection tool, the frame selected area is used as the location of the marked point cloud object, and the point cloud object attribute corresponding to the target point cloud frame selection tool is used as the property of the marked point cloud object.

The labeling information determining unit is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, and use the area surrounded by the drawn rectangular frame as the location of the marked passenger; Passenger attribute selection operation to determine at least one attribute of the marked passenger.

The labeling information determining unit is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, use the area surrounded by the drawn rectangular frame as the location of the marked human face; The key point labeling operation of the tool determines the key points of the face in the marked face.

The labeling information determination unit is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, use the area surrounded by the drawn rectangular frame as the position of the marked hand; The key point labeling operation of the tool determines the key points of the marked hand.

The labeling information determining unit is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, use the area surrounded by the drawn rectangular frame as the position of the fatigue detection target marked; The key point labeling operation of the tool determines the attributes of the labeled fatigue detection target.

The labeling information determining unit is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, and use the area surrounded by the drawn rectangular frame as the position of the marked dangerous driving detection target; The attribute selection operation of the labeling tool determines the attributes of the marked dangerous driving detection target.

According to an aspect of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above-mentioned method.

According to one aspect of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the above method is implemented.

According to an aspect of the present disclosure, there is provided a computer program product, including computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are stored in an electronic device When running in the processor, the processor in the electronic device is used to implement the above method.

In the embodiment of the present disclosure, after obtaining the object to be marked, the target marking tool corresponding to the marking task type of the object to be marked among the multiple marking tools can be presented to the user interaction interface, and the user can perform the marking operation based on the target marking tool, Obtain the labeling information of the object to be labeled. As a result, labeling tools can be quickly and accurately provided according to the type of labeling task, and the data labeling scheme required for the development of smart cars can be clarified for traditional car companies to make up for the shortcomings of data labeling in the combination of traditional car companies and artificial intelligence technology. The development resources invested in the research and development process are reduced.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the accompanying drawings.

Description of drawings

The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.

Fig. 1 shows a flowchart of a data labeling method according to an embodiment of the present disclosure.

Fig. 2 shows a block diagram of a data labeling device according to an embodiment of the present disclosure.

Fig. 3 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

Fig. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure.

detailed description

Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the figures indicate functionally identical or similar elements. While various aspects of the embodiments are shown in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

The word "exemplary" is used exclusively herein to mean "serving as an example, embodiment, or illustration." Any embodiment described herein as "exemplary" is not necessarily to be construed as superior or better than other embodiments.

The term "and/or" in this article is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean: A exists alone, A and B exist simultaneously, and there exists alone B these three situations. In addition, the term "at least one" herein means any one of a variety or any combination of at least two of the more, for example, including at least one of A, B, and C, which may mean including from A, Any one or more elements selected from the set formed by B and C.

In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific implementation manners. It will be understood by those skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, components and circuits that are well known to those skilled in the art have not been described in detail so as to obscure the gist of the present disclosure.

Embodiments of the present disclosure provide a data labeling solution, which can be applied to labeling tasks in various scenarios and obtain labeling information. This method can be widely applied to the labeling requirements in the field of smart cars, and saves time and manpower.

The data labeling method provided by the embodiments of the present disclosure can label the samples used for neural network training in the smart car industry, use the label information obtained by labeling to train the neural network, and realize automatic driving based on the trained neural network and smart cockpits. Specific labeling scenarios may include, for example: lane line labeling, drivable area labeling, target object labeling, point cloud object labeling, passenger attribute labeling, face labeling, gesture labeling, driving fatigue labeling, dangerous driving labeling, etc.

In a possible implementation, the data labeling method may be performed by electronic devices such as terminal devices or servers, and the terminal devices may be user equipment (User Equipment, UE), mobile devices, user terminals, terminals, cellular phones, cordless Phones, personal digital assistants (Personal Digital Assistant, PDA), handheld devices, computing devices, vehicle-mounted devices, wearable devices, etc., the method can be realized by calling the computer-readable instructions stored in the memory by the processor. Alternatively, the method may be performed by a server.

For ease of description, in one or more embodiments of this specification, the execution subject of the identity authentication method may be a labeling platform, and the implementation of the method will be introduced later by taking the execution subject as the labeling platform as an example. It can be understood that the execution subject of the method is an annotation platform, which is only an exemplary description, and should not be understood as a limitation of the method.

The labeling method of the embodiments of the present disclosure may also be applied in other technical fields involving various types of labeling tasks, which is not specifically limited in the embodiments of the present disclosure. The following mainly takes the smart car field as the target field to illustrate.

FIG. 1 shows a flowchart of a data labeling method according to an embodiment of the present disclosure. As shown in FIG. 1 , the data labeling method includes:

In step S11, the object to be marked is obtained.

The user can load the object to be marked through the menu of the user interaction interface, or drag the object to be marked to the designated area of the user interaction interface by directly dragging and dropping, so as to complete the loading of the mark object. The acquisition of the object to be marked is realized by loading the object to be marked.

The object to be labeled can be one or more images; it can also be a single frame or multiple frames in a video. The people, objects, and scenery marked in the object to be marked are the target object, and in the subsequent steps, the marking operation on the target object will be performed on the object to be marked.

The content displayed in the object to be marked can be: road pavement, road signs, road vehicles, road pedestrians, the driver's face, hands, and limbs inside the cabin, and the faces, hands, and limbs of passengers inside the cabin, etc. , the displayed content will contain the target objects that need to be marked. In addition, the target objects to be marked among the objects to be marked may be different in different application scenarios.

In step S12, a target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools is presented to the user interface.

After obtaining the object to be labeled, the user can determine the type of labeling task. The types of labeling tasks here can include: lane line detection, drivable area segmentation, target detection, point cloud labeling, etc. According to the determined labeling task type, a suitable labeling tool can be selected as the target labeling tool.

Various labeling tasks can correspond to at least one target labeling tool. The target marking tool here may be a control for marking the target object in the object to be marked, such as: a line tool, a polygon tool, a rectangular box tool, and the like.

The aforementioned controls may be presented at preset positions in the user interaction interface.

In step S13, the labeling information of the object to be labeled is obtained in response to the labeling operation performed based on the target labeling tool.

The user uses the target labeling tool presented in step S12 to perform labeling operations according to the labeling task among the objects to be labeled obtained in step S11 to label the target object. The labeling operation here can be understood as that the user can use the target labeling tool to determine the points that can identify the target object and the lines between the points, use these points and lines to determine the position or range of the object to be marked, and mark the object properties. After the labeling operation is completed, labeling information such as the position, range, and attribute of the marked object is obtained.

In a possible implementation manner, after acquiring the object to be marked, the method further includes: responding to a user's selection operation on the multiple marking tools, using the marking tool selected by the user as a target marking tool.

After obtaining the object to be labeled, the user can select the target labeling tool among multiple labeling tools according to the shape and other characteristics of the object to be labelled. For example, you can select the rectangular frame tool to mark traffic lights, street signs, signboards, etc.; you can select lines to mark lane lines.

In a possible implementation, after acquiring the object to be labeled, the method further includes: identifying the labeling task type of the object to be labeled to obtain the target labeling task type; according to the preset labeling task type and labeling The corresponding relationship between tools is to determine the labeling tool that has a corresponding relationship with the target labeling task type as the target labeling tool.

In this implementation, the target labeling task type can be determined according to the application scenario of the labeling task set by the user; or, based on the image recognition technology, the target labeling task type can be determined according to the content of the image or video to be labeled. Example 1, if the application scenario of the labeling task set by the user is to detect whether the driver is fatigued driving, it can be determined that the type of the labeling task is: driver fatigue detection. Example 2, the content of the object (image) to be labeled is a road surface, and the labeling task type can be determined as: lane line detection.

In a possible implementation manner, a "labeling task" menu may also be displayed in a tab of the user interaction interface, and then the type of labeling task selected by the user based on the "labeling task" menu may be received as the target labeling task type.

In order to adapt to different labeling task types, the corresponding relationship between target labeling tools and labeling task types can be established in advance. Based on the corresponding relationship, the target labeling tool corresponding to the obtained target labeling task type can be determined, and the target labeling tool can be presented on the user interaction interface.

Exemplarily, when the determined target labeling task type is "drivable area segmentation". You can use the polygon tool as a tool to label drivable areas.

In the embodiment of the present disclosure, by classifying the labeling tasks and associating the labeling task type with the labeling tool, after the target labeling task type is determined, the corresponding target labeling tool can be obtained, which improves the efficiency of labeling.

In a possible implementation manner, the presenting the target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to the user interaction interface includes: receiving a data field selection instruction input by the user, The data field selection instruction is used to determine the target data field to which the object to be marked belongs, and the target data field includes the smart car field; according to the determined target data field, determine the object to be marked for marking the target data field from the tool set target labeling tool, and enable the permission to use the target labeling tool, and display it on the labeling interface for users to use.

The data labeling method provided in this disclosure can be applied to labeling tasks in various data fields. The data field here can be the field where artificial intelligence technology is applied, such as the field of smart cars, smart retail, and smart security. For different data fields, the labeling tools used are often determined. Therefore, in this implementation, the object to be labeled for labeling the target data field can be determined from the tool set according to the target data field to which the object to be labeled belongs. target labeling tool to improve labeling efficiency.

The target data field may be determined based on a data field selection instruction input by the user, so that the user can select the target data field to which the object to be marked belongs. For example, in the case that the object to be labeled belongs to the field of smart cars, the user can select the field of smart cars based on the user operation interface.

In this implementation, the tool set can include a variety of tools, such as line tools, polygon tools, rectangular box tools, attribute labeling tools, etc. In order to facilitate the determination of target labeling tools based on data fields, data fields and labeling tools can be pre-established The corresponding relationship between them, that is, to predetermine the corresponding labeling tools for each data field. Then, after the target data field is determined, the labeling tool corresponding to the target data field can be determined from the tool set as the target labeling tool according to the corresponding relationship.

For the determined target labeling tool, its usage permission will be enabled and displayed in the labeling interface, so that labelers can perform labeling tasks based on the target labeling tool.

In the embodiment of the present disclosure, the target data field is determined based on the data field selection instruction input by the user, and then according to the determined target data field, the target labeling tool for marking the object to be marked in the target data field is determined from the tool set for the user use. As a result, the labeling tool required to perform the labeling task can be quickly provided to the user, greatly reducing labeling time and labor costs, and improving the convenience of the labeling tool.

In a possible implementation manner, the obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes: in response to the position labeling operation performed based on the target labeling tool, Use the marked position as the position of the marked target object; use the preset default attribute as the property of the marked target object.

The position of the target object can be obtained by marking based on tools such as a polygon tool and a rectangular frame tool. The user determines the position of the marked target object by operating the polygon tool or the rectangular frame tool.

In order to reduce labeling time and labor costs, it is possible to combine the acquisition of the target object’s position and the attribute of the target object into one step, that is, to pre-set a default attribute, which can be preset by the developer, or it can also be set by the labeler himself. Setting, when the position of the target object is marked based on tools such as the polygon tool and the rectangular frame tool, the marked target position is automatically given default attributes, thereby greatly reducing the marking time and labor costs, and improving the convenience of the marking tool sex.

Next, according to the types of various labeling tasks, possible implementations of the data labeling method in the embodiments of the present disclosure are exemplarily introduced.

In a possible implementation, the labeling task type of the object to be labeled includes a lane line labeling task, and the target labeling tools corresponding to the lane line labeling task include a line tool and a lane line attribute labeling tool; the response to Obtaining the labeling information of the object to be labeled based on the labeling operation performed by the target labeling tool includes: responding to the line drawing operation based on the line tool, taking the area indicated by the drawn line as the location of the marked lane line Position; in response to the lane line attribute selection operation based on the lane line attribute labeling tool, determine the attribute of the marked lane line.

Exemplarily, when annotating an image or video applied to lane line detection, the image or video frame to be annotated is displayed in a user interaction interface by means of loading or importing. Since the labeled object is a lane line, the corresponding relationship between the labeling task type "lane line labeling task" and the line tool can be set in advance. Based on the corresponding relationship, when it is determined that the labeling task type is "lane line labeling task", the line tool can be presented in the user interaction interface.

The user can select the line tool in the user interface, then draw the line, and then select the attribute of the line, which is used to represent the type of lane line, for example, white solid line, white dashed line, left lane line, right lane line etc. Alternatively, the user may complete the selection of the line attributes first, and then perform the line drawing operation.

When the user is performing the line drawing operation, the user can determine any end of the lane line to be marked as the starting point. Specifically, the midpoint of the starting end in the width direction of the lane line can be used as the starting point, and the user can click the left button of the mouse to The operation method determines the position of the starting point, and the same operation method is used to determine the point representing the lane line along the lane line, and the determined point should be on the lane line. Then, take the other end point of the lane line as the end point. When the end point is selected, you can click the right mouse button to complete the labeling of the lane line.

After marking the lane lines in the image or video, the position of the lane line (the coordinates of each point marking the lane line), attributes and other labeling information are obtained.

In the embodiment of the present disclosure, through the above method, after the position marking of the lane line is completed, the marking information of the lane line can be obtained, which improves the efficiency of marking the lane line.

From the perspective of cost saving, you can also use the polygon tool to mark the lane lines. In this way, the same labeling tool can be shared with other labeling tasks that can use the polygon tool provided by the present disclosure, so as to save development costs.

In a possible implementation, the tagging task type of the object to be tagged includes a drivable area tagging task, and the target tagging tool corresponding to the drivable area tagging task includes a polygon tool; the response is based on the target The marking operation performed by the marking tool to obtain the marking information of the object to be marked includes: responding to the polygon drawing operation based on the polygon tool, using the area where the drawn polygon is located as the position of the marked drivable area.

Exemplarily, when marking an image or video with a drivable area, the image or video frame to be marked is displayed in the user interaction interface by means of loading or importing. In most cases, the drivable area can be a polygon. Then you can pre-set the corresponding relationship between the polygon tool and the "Drivable Area Labeling" labeling task type. Based on this correspondence, when the labeling task type is determined to be "drivable area labeling", the polygon tool can be presented in the user interface. In addition, it is also possible to pre-set the corresponding relationship between the scene segmentation tool and the "Drivable Area Labeling" labeling task type. When the labeling task type is determined to be "Drivable Area Labeling", the scene segmentation tool can be presented in the interactive interface .

The scene segmentation tool here is a pre-integrated module. The user roughly selects the boundary of the object to be segmented, and then the scene segmentation tool can accurately determine the boundary between the object to be segmented and the background according to the boundary, so as to realize the segmentation of the object to be segmented. In the embodiment of the present disclosure, the object to be segmented may be the driving area of the vehicle.

The attributes of the polygon can be: drivable area, non-drivable area. Users can choose according to the labeling task. It can be understood that if only a single type of area needs to be marked in the labeling task, for example: only the drivable area or only the non-drivable area, then if there is only one attribute in the marked polygon, it is not necessary for the user to modify the polygon Instead, a default attribute is directly given, and the user marks the area corresponding to the default attribute.

During the polygon drawing operation, taking the drivable area label as an example, the user can select any point on the drivable area boundary as the starting point, and then sequentially determine multiple points indicating the drivable area, which can be the drivable area boundary The inflection point on the line can also be any point on the boundary line. Click the right mouse button to add the last point, the last point is automatically connected with the starting point to form a polygon, and the polygon drawing operation is completed. The closed circle obtained by the drawing operation The area can then be determined as a drivable area.

According to the above method, each drivable area can be determined on the image, and labeling information such as the range (coordinates) and attributes of the drivable area can be obtained at the same time, which improves the efficiency of marking the drivable area.

In a possible implementation, the labeling task type of the object to be labeled includes a target object labeling task, and the target labeling tools corresponding to the target object labeling task include a rectangular box tool and a target object attribute labeling tool; the response Obtaining the labeling information of the object to be labeled based on the labeling operation performed by the target labeling tool includes: responding to the rectangle frame drawing operation based on the rectangle frame tool, using the area surrounded by the drawn rectangle frame as a label The location of the target object; in response to the attribute selection operation of the target object attribute labeling tool, determine the attribute of the marked target object in the objects to be marked.

In the field of automatic driving of automobiles, object detection may include, for example, the detection of objects such as signal lights, traffic signs, surface signs, pedestrians, and vehicles.

In the embodiment of the present disclosure, the rectangular frame tool may be associated with the labeling task type of "target object labeling" in advance. After the object to be labeled is loaded and the target labeling task type is determined as "target object labeling", the rectangular frame tool can be presented in the user interface. The user can configure the properties of the rectangular box tool, and the properties of the rectangular box can include pedestrians, motor vehicles, non-motor vehicles, etc. The user can first select the properties of the rectangle, and then select the rectangle tool to draw a rectangle; or first select the rectangle tool to draw a rectangle to obtain a rectangle, and then configure the properties of the rectangle.

In the process of drawing a rectangular frame, take drawing a rectangular frame with the attribute of "pedestrian" as an example, click the left mouse button to create a starting point, select the pedestrians in the image, and click the left mouse button again to complete the labeling operation. After the rectangular frame ends, the pedestrians in the rectangular frame are tangent to the upper, lower, left and right sides of the rectangular frame. Taking the Cartesian coordinate system as an example, among the points indicating the position of pedestrians, the point with the largest y value is located on the upper side of the rectangular frame; among the points indicating the position of pedestrians, the point with the smallest y value falls on the lower side of the rectangular frame; Among the points at the position, the point with the largest x value is located on the right side of the rectangle; the point with the smallest x value is located on the left side of the rectangle.

Then, according to the above process, the pedestrians can be selected one by one, and the attribute of the rectangular frame is configured as "pedestrian", and the labeling of pedestrians can be completed.

Through the same method, the labeling of other target objects can be completed, and the respective attributes can be selected to obtain the labeling information of the target object, which improves the efficiency of target object labeling.

In a possible implementation, the object to be labeled is a point cloud image, the labeling task type includes a point cloud object labeling task, and the target labeling tool corresponding to the point cloud object labeling task includes a plurality of point cloud objects A frame selection tool, the multiple point cloud object frame selection tools correspond to different point cloud object attributes; the response to the labeling operation performed based on the target labeling tool is to obtain the labeling information of the object to be labeled, including: response Based on the point cloud object frame selection tool selection operation, determine the target point cloud object frame selection tool in the multiple point cloud object frame selection tools; in response to the frame selection operation based on the target point cloud object frame selection tool, frame The selected area is used as the location of the marked point cloud object, and the point cloud object attribute corresponding to the target point cloud frame selection tool is used as the attribute of the marked point cloud object.

Exemplarily, the user loads or imports the point cloud image into the user interaction interface. A point cloud image is a data set of points in a certain coordinate system. The point cloud image contains the position information of each point, and some point cloud images also contain color information, reflection intensity information, etc.

In the embodiment of the present disclosure, the corresponding relationship between the point cloud object frame selection tool and the labeling task type of "point cloud object labeling task" can be established in advance. Based on the corresponding relationship, after determining the target labeling task type as "point cloud object labeling task", the point cloud object frame selection tool can be presented in the user interface. Then the user can configure the properties of the point cloud object marquee tool, which can include, for example, people, trees, vehicles, and so on.

During the frame selection operation, the user clicks the left mouse button to create a starting point, pulls out a rectangular frame, and selects the target object in the point cloud image, and clicks the left mouse button again to complete the frame selection operation. After the frame selection operation is completed, the rectangle frame is tangent to the frame-selected target object. It can be understood that if the object to be marked is a 3D point cloud image and the rectangular frame is a cube, at this time, each surface of the rectangular frame is tangent to the target object.

According to the above method, each point cloud object in the point cloud image can be framed and selected, and the corresponding attribute can be selected to obtain the labeling information of the point cloud object, which improves the efficiency of point cloud object labeling.

The data labeling method provided in the present disclosure can not only realize the data labeling task in the field of automobile automatic driving, but also can be used to realize the data labeling task in the field of vehicle intelligent cockpit. Several possible implementation methods are described in detail below.

In a possible implementation, the labeling task type includes a passenger attribute labeling task, and the target labeling tools corresponding to the passenger attribute labeling task include a rectangular box tool and a passenger attribute labeling tool; the response is based on the target The labeling operation performed by the labeling tool is to obtain the labeling information of the object to be labeled, including: responding to the rectangular frame drawing operation based on the rectangular frame tool, using the area surrounded by the drawn rectangular frame as the position of the marked passenger ; determining at least one attribute of an annotated passenger in response to a passenger attribute selection operation based on the passenger attribute annotation tool.

In the embodiment of the present disclosure, the corresponding relationship between the "rectangular frame tool and passenger attribute labeling tool" and the labeling task type "passenger attribute labeling task" may be established in advance. Based on this correspondence, when the labeling task type is the passenger attribute labeling task, it can be determined that the target labeling tools corresponding to the passenger attribute labeling task include the rectangular box tool and the passenger attribute labeling tool, and present these tools to the user in the interface. The user can select the rectangular frame tool, and then select the attributes of the rectangular frame in the passenger attribute labeling tool, such as: seating position, age, emotion, whether to wear a mask, etc. In addition, in the passenger attribute labeling tool, multiple secondary attributes of each attribute can also be set. For example, for the "age" attribute, the secondary attributes can be set to 10-20 years old, 20-30 years old, etc.

Then, the user uses the rectangular frame tool to perform a rectangular frame drawing operation to frame the position of the passenger. For the specific process of the rectangular frame drawing operation, please refer to the relevant description of the present disclosure, which will not be repeated here. After the frame selection is completed, the selected passengers are in the corresponding rectangular frame and are tangent to the corresponding rectangular frame. The selected passengers get corresponding attributes at the same time.

In addition, you can use the rectangular box tool to select passengers first, and then set the properties of the rectangular box.

According to the above method, each passenger in the image can be selected by frame, and the corresponding attribute can be selected to obtain the annotation information of the passenger, which improves the efficiency of attribute annotation of the passenger.

In a possible implementation, the tagging task type includes a face tagging task, and the target tagging tools corresponding to the face tagging task include a rectangular box tool and a key point tagging tool; the response is based on the target The labeling operation performed by the labeling tool to obtain the labeling information of the object to be labeled includes: in response to the rectangular frame drawing operation based on the rectangular frame tool, using the area surrounded by the drawn rectangular frame as the area where the marked human face is located Position: in response to the key point labeling operation based on the key point labeling tool, determine the key points of the human face in the marked human face.

In the embodiment of the present disclosure, the corresponding relationship between the "rectangular frame tool and the key point labeling tool" and the labeling task type of "face labeling task" can be established in advance. Based on this correspondence, when the labeling task type is a face labeling task, it can be determined that the target labeling tools corresponding to the face labeling task include a rectangular frame tool and a key point labeling tool, and present these tools on the user interface middle. The user can select the rectangular frame tool, and select the position of the face through the rectangular frame drawing operation. The specific process of the rectangular frame drawing operation can be found in the relevant description of this disclosure, and will not be repeated here. Select the target face by frame to determine the position of the face, so that the rectangular frame is tangent to the edge of the face. Then, open the key point labeling tool and set the number of face key points, for example: 5 points for a face, 106 points for a face. Use the key point labeling tool to mark the position of the facial features on the selected target face.

Through the above method, after marking the position of the face, the marking information of the face can be obtained, which improves the efficiency of face marking.

In a possible implementation, the labeling task type includes a gesture labeling task, and the target labeling tools corresponding to the gesture labeling task include a rectangular box tool and a key point labeling tool; the response is based on the target labeling tool The labeling operation performed to obtain the labeling information of the object to be labeled includes: responding to the rectangle frame drawing operation based on the rectangle frame tool, using the area surrounded by the drawn rectangle frame as the position of the marked hand; In response to a key point labeling operation based on the key point labeling tool, key points of the marked hand are determined.

In the embodiment of the present disclosure, the corresponding relationship between the "rectangular frame tool and key point labeling tool" and the labeling task type of "gesture labeling task" may be established in advance. Based on the corresponding relationship, when the labeling task type is gesture labeling, it can be determined that the target labeling tools corresponding to the gesture labeling task include a rectangular box tool and a key point labeling tool, and present these tools in the user interface. The user can select the rectangle frame tool and select the target object through the rectangle frame drawing operation. In this example, the target object is the target hand, so that the rectangle frame is tangent to the edge of the target hand to determine the position of the hand. Marquee selection ranges from human fingers to wrists.

Then, open the key point labeling tool and set the number of hand key points, for example: 21 points for human hands and 5 points for fingertips. Use the key point labeling tool to mark the joints of the hand on the frame-selected target hand with the key point labeling tool. After the labeling is completed, the corresponding attributes (coordinates, point spacing, etc.) of the key points of the labeling are obtained.

Through the above method, after marking the position of the hand, the marking information of the hand can be obtained, which improves the efficiency of gesture marking.

In a possible implementation, the labeling task type includes a driving fatigue labeling task, and the target labeling tools corresponding to the driving fatigue task include a rectangular frame tool and a fatigue attribute labeling tool; the response is based on the target labeling The marking operation performed by the tool to obtain the marking information of the object to be marked includes: responding to the drawing operation of the rectangular frame based on the rectangular frame tool, using the area surrounded by the drawn rectangular frame as the position of the fatigue detection target marked ; In response to the key point labeling operation based on the fatigue property labeling tool, determine the property of the labelled fatigue detection target.

In the embodiment of the present disclosure, the corresponding relationship between the "rectangular frame tool and fatigue attribute labeling tool" and the labeling task type "driving fatigue labeling task" may be established in advance. Based on the corresponding relationship, when the labeling task is a driving fatigue labeling task, it can be determined that the target labeling tools corresponding to the driving fatigue labeling task include a rectangular frame tool and a fatigue attribute labeling tool, and present these tools in the user interface. The user first selects the rectangle frame tool, and the objects selected by the rectangle frame can be the human eyes and the human mouth. The human eyes and the human mouth are framed by the rectangle frame drawing operation, and then the properties of the rectangle frame are selected to use the human eyes For example, its attributes may include eyes open, eyes closed, uncertain, etc.

In another implementation manner, if the definition of the object to be labeled is limited, image segmentation processing may be performed on the object to be labeled (image or single-frame video). Segment the human eye and mouth frames in the image, and generate eye images and mouth images. Then load the newly generated eye image or mouth image into the interactive interface, and the user can select the attributes of the eye image and the mouth image in the interactive interface.

Through the above method, the position of the fatigue detection target can be marked, and the labeling information of the fatigue detection target can be obtained, which improves the efficiency of the driving fatigue labeling task. When the definition of the labeling object is limited, the accuracy of driving fatigue labeling can be improved.

In another implementation, the tagging task type includes a dangerous driving tagging task, and the target tagging tools corresponding to the dangerous driving task include a rectangular frame tool and a dangerous attribute tagging tool; the response is based on the target tagging tool The labeling operation performed to obtain the labeling information of the object to be labeled includes: responding to the rectangular frame drawing operation based on the rectangular frame tool, using the area surrounded by the drawn rectangular frame as the position of the marked dangerous driving detection target ; Responding to an attribute selection operation based on the dangerous attribute labeling tool, determining the labelled attribute of the dangerous driving detection target.

In the embodiment of the present disclosure, the corresponding relationship between the "rectangular frame tool and dangerous attribute labeling tool" and the labeling task type "dangerous driving labeling task" may be established in advance. Based on this correspondence, when the labeling task is a dangerous driving labeling task, it can be determined that the target labeling tools corresponding to the driving fatigue labeling task include a rectangular box tool and a fatigue attribute labeling tool, and present these tools in the user interface. The user first selects the rectangular frame tool, and determines the position of the target object in the dangerous driving detection through the rectangular frame drawing operation. The target object is usually a human body. After the frame is selected, the attribute of the rectangular frame (for example: smoking, calling , drinking water, etc.). Alternatively, the user may first set the default properties of the rectangular frame, and then use the rectangular frame to select the target object (target human body) in the image, so that the properties of the drawn rectangular frame are the preset default properties.

Through the above method, after marking the position of the dangerous driving detection target, the marking information of the dangerous driving detection target can be obtained, which improves the efficiency of dangerous driving marking.

The data labeling method in the embodiment of the present disclosure can also be applied to other labeling scenarios, and the idea of specific labeling is the same as that of the above-mentioned implementation mode, and will not be introduced one by one due to space limitation. In addition, for the same labeling task, there may also be multiple suitable labeling tools. Therefore, in the embodiments of the present disclosure, different labeling schemes can be given, and a variety of optional labeling tools can be provided to facilitate labeling personnel according to different production requirements. According to operating conditions, trade-offs are made in tool construction in order to maximize the efficiency of labeling and speed up the research and development process.

The systematic data labeling solution for artificial intelligence in the automotive industry provided by this disclosure can be used as an overall R&D solution to support data labeling for traditional car companies to develop intelligent cars, or it can be used as a single combined solution to provide car companies with automatic data labeling solutions. The artificial intelligence implementation of driving and smart cockpit provides data annotation support. Through this solution, it is possible to clarify the data labeling tools that traditional car companies need in the implementation of artificial intelligence, and the detailed data labeling scheme, without the need to integrate all market labeling tools to reduce the development resources invested in the research and development process. In addition, in the process of model training, according to business needs, the research and development schedule can be clarified, and the development of more urgent annotation tools in business is given priority. Or, for some general marking tools that can be applied to a variety of different marking tasks, the tool can be developed first. For example, if there are multiple marking tasks that need to mark irregular objects, the line tool can be discarded and only the polygon tool can be developed. All irregular target objects that need to be marked.

It can be understood that the above-mentioned method embodiments mentioned in this disclosure can all be combined with each other to form a combined embodiment without violating the principle and logic. Due to space limitations, this disclosure will not repeat them. Those skilled in the art can understand that, in the above-mentioned method of the specific implementation mode, the specific execution sequence of each step should be determined by its function and possible internal logic.

In addition, the present disclosure also provides data labeling devices, electronic equipment, computer-readable storage media, and programs, all of which can be used to implement any data labeling method provided in the present disclosure, corresponding technical solutions and descriptions, and corresponding records in the method section ,No longer.

Fig. 2 shows a block diagram of a data labeling device according to an embodiment of the present disclosure. As shown in Fig. 2, the device 20 is applied to the field of smart cars, including:

An annotated object acquisition unit 201, configured to acquire an object to be annotated;

A target labeling tool display unit 202, configured to present a target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to the user interface;

The labeling information determining unit 203 is configured to obtain labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool.

In a possible implementation manner, the object labeling tool display unit 202 includes:

In a possible implementation manner, the device further includes:

In a possible implementation manner, the annotation information determining unit 203 includes:

The marking information determining unit 203 is configured to respond to the line drawing operation based on the line tool, use the area indicated by the drawn line as the location of the marked lane line; Lane line attribute selection operation, determine the attribute of the marked lane line.

The marking information determining unit 203 is configured to use the area where the drawn polygon is located as the position of the marked drivable area in response to the polygon drawing operation based on the polygon tool.

The labeling information determining unit 203 is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, use the area surrounded by the drawn rectangular frame as the location of the marked target object; The attribute selection operation of the tool determines the attribute of the target object marked among the objects to be marked.

The label information determining unit 203 is configured to determine a target point cloud object frame selection tool among the plurality of point cloud object frame selection tools in response to a point cloud object frame selection tool selection operation; In the frame selection operation of the object frame selection tool, the framed area is used as the position of the marked point cloud object, and the point cloud object attribute corresponding to the target point cloud frame selection tool is used as the property of the marked point cloud object.

The labeling information determining unit 203 is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, use the area surrounded by the drawn rectangular frame as the location of the marked passenger; The passenger attribute selection operation of the tool determines at least one attribute of the marked passenger.

The labeling information determining unit 203 is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, use the area surrounded by the drawn rectangular frame as the location of the marked human face; The key point labeling operation of the labeling tool determines the key points of the face in the marked face.

The labeling information determining unit 203 is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, and use the area surrounded by the drawn rectangular frame as the position of the marked hand; The key point labeling operation of the labeling tool determines the key points of the marked hand.

The labeling information determining unit 203 is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, and use the area surrounded by the drawn rectangular frame as the position of the fatigue detection target marked; The key point labeling operation of the labeling tool determines the attributes of the labeled fatigue detection target.

In a possible implementation, the labeling task type includes a dangerous driving labeling task, and the target labeling tools corresponding to the dangerous driving task include a rectangular box tool and a dangerous attribute labeling tool;

The labeling information determining unit 203 is configured to respond to the rectangular frame drawing operation based on the rectangular frame tool, and use the area surrounded by the drawn rectangular frame as the position of the marked dangerous driving detection target; The attribute selection operation of the attribute labeling tool determines the attribute of the marked dangerous driving detection target.

In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the method embodiments above, and its specific implementation can refer to the description of the method embodiments above. For brevity, here No longer.

Embodiments of the present disclosure also provide a computer-readable storage medium, on which computer program instructions are stored, and the above-mentioned method is implemented when the computer program instructions are executed by a processor. Computer readable storage media may be volatile or nonvolatile computer readable storage media.

An embodiment of the present disclosure also proposes an electronic device, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to invoke the instructions stored in the memory to execute the above method.

An embodiment of the present disclosure also provides a computer program product, including computer-readable codes, or a non-volatile computer-readable storage medium carrying computer-readable codes, when the computer-readable codes are stored in a processor of an electronic device When running in the electronic device, the processor in the electronic device executes the above method.

Electronic devices may be provided as terminals, servers, or other forms of devices.

FIG. 3 shows a block diagram of an electronic device 800 according to an embodiment of the present disclosure. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, or a personal digital assistant.

Referring to FIG. 3 , electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input/output (I/O) interface 812, sensor component 814 , and the communication component 816.

The processing component 802 generally controls the overall operations of the electronic device 800, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 802 may include one or more modules that facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802 .

The memory 804 is configured to store various types of data to support operations at the electronic device 800 . Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phonebook data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

The power supply component 806 provides power to various components of the electronic device 800 . Power components 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for electronic device 800 .

The multimedia component 808 includes a screen providing an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 810 is configured to output and/or input audio signals. For example, the audio component 810 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 800 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 804 or sent via communication component 816 . In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

The I/O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of electronic device 800 . For example, the sensor component 814 can detect the open/closed state of the electronic device 800, the relative positioning of components, such as the display and the keypad of the electronic device 800, the sensor component 814 can also detect the electronic device 800 or a Changes in position of components, presence or absence of user contact with electronic device 800 , electronic device 800 orientation or acceleration/deceleration and temperature changes in electronic device 800 . Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. Sensor assembly 814 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor component 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, electronic device 800 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A programmable gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.

In an exemplary embodiment, there is also provided a non-volatile computer-readable storage medium, such as the memory 804 including computer program instructions, which can be executed by the processor 820 of the electronic device 800 to implement the above method.

FIG. 4 shows a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, electronic device 1900 may be provided as a server. Referring to FIG. 4 , electronic device 1900 includes processing component 1922 , which further includes one or more processors, and a memory resource represented by memory 1932 for storing instructions executable by processing component 1922 , such as application programs. The application programs stored in memory 1932 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

Electronic device 1900 may also include a power supply component 1926 configured to perform power management of electronic device 1900, a wired or wireless network interface 1950 configured to connect electronic device 1900 to a network, and an input-output (I/O) interface 1958 . The electronic device 1900 can operate based on the operating system stored in the memory 1932, such as the Microsoft server operating system (Windows Server ^TM ), the graphical user interface-based operating system (Mac OS X ^TM ) introduced by Apple Inc., and the multi-user and multi-process computer operating system (Unix ^™ ), a free and open-source Unix-like operating system (Linux ^™ ), an open-source Unix-like operating system (FreeBSD ^™ ), or the like.

In an exemplary embodiment, there is also provided a non-transitory computer-readable storage medium, such as the memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to implement the above method.

The present disclosure can be a system, method and/or computer program product. A computer program product may include a computer readable storage medium having computer readable program instructions thereon for causing a processor to implement various aspects of the present disclosure.

A computer readable storage medium may be a tangible device that can retain and store instructions for use by an instruction execution device. A computer readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or flash memory), static random access memory (SRAM), compact disc read only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, such as a printer with instructions stored thereon A hole card or a raised structure in a groove, and any suitable combination of the above. As used herein, computer-readable storage media are not to be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.

Computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to a respective computing/processing device, or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in a computer-readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or Source or object code written in any combination, including object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. Computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server implement. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as via the Internet using an Internet service provider). connect). In some embodiments, an electronic circuit, such as a programmable logic circuit, field programmable gate array (FPGA), or programmable logic array (PLA), can be customized by utilizing state information of computer-readable program instructions, which can Various aspects of the present disclosure are implemented by executing computer readable program instructions.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It should be understood that each block of the flowcharts and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams, can be implemented by computer-readable program instructions.

These computer-readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine such that when executed by the processor of the computer or other programmable data processing apparatus , producing an apparatus for realizing the functions/actions specified in one or more blocks in the flowchart and/or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices and/or other devices to work in a specific way, so that the computer-readable medium storing instructions includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks in flowcharts and/or block diagrams.

It is also possible to load computer-readable program instructions into a computer, other programmable data processing device, or other equipment, so that a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process , so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, a portion of a program segment, or an instruction that includes one or more Executable instructions. In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified function or action , or may be implemented by a combination of dedicated hardware and computer instructions.

The computer program product can be specifically realized by means of hardware, software or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (Software Development Kit, SDK) etc. Wait.

Having described various embodiments of the present disclosure above, the foregoing description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and alterations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principle of each embodiment, practical application or improvement of technology in the market, or to enable other ordinary skilled in the art to understand each embodiment disclosed herein.

Claims

A data labeling method, characterized in that it is applied to the field of smart cars, including:

Get the object to be marked;

Presenting a target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to a user interaction interface;

In response to the labeling operation performed based on the target labeling tool, labeling information of the object to be labelled is obtained.
The method according to claim 1, wherein the presenting the target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to the user interface includes:

Receive a data field selection instruction input by a user, the data field selection instruction is used to determine the target data field to which the object to be marked belongs, and the target data field includes the smart car field;

According to the determined target data field, a target labeling tool for labeling objects to be labeled in the target data field is determined from the tool set, and the use authority of the target labeling tool is enabled, and displayed on the labeling interface for users to use.
According to the method according to any one of claims 1 and 2, after acquiring the object to be marked, the method further comprises:

In response to the user's selection operation on the plurality of annotation tools, the annotation tool selected by the user is used as the target annotation tool.
According to the method according to any one of claims 1 and 2, after acquiring the object to be marked, the method further comprises:

Identifying the labeling task type of the object to be labeled to obtain the target labeling task type;

According to the preset corresponding relationship between the labeling task type and the labeling tool, a labeling tool having a corresponding relationship with the target labeling task type is determined as the target labeling tool.
The method according to any one of claims 1-4, wherein the obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the position labeling operation performed based on the target labeling tool, using the marked position as the position of the marked target object;

Use the pre-set default attributes as the attributes of the marked target object.
The method according to any one of claims 1-5, wherein the labeling task type of the object to be labeled includes a lane line labeling task, and the target labeling tool corresponding to the lane line labeling task includes a line tool and a lane line attribute annotation tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the line drawing operation based on the line tool, the area indicated by the drawn line is used as the location of the labeled lane line;

In response to the lane line attribute selection operation based on the lane line attribute labeling tool, the attribute of the marked lane line is determined.
The method according to any one of claims 1-6, wherein the tagging task type of the object to be tagged includes a drivable area tagging task, and the target tagging tool corresponding to the drivable area tagging task includes a polygon tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the polygon drawing operation based on the polygon tool, the area where the drawn polygon is located is used as the location of the marked drivable area.
The method according to any one of claims 1-7, wherein the labeling task type of the object to be labeled includes a target object labeling task, and the target labeling tools corresponding to the target object labeling task include a rectangular frame tool and a target object Attribute annotation tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the location of the marked target object;

In response to the attribute selection operation of the target object attribute labeling tool, the attribute of the target object marked among the objects to be marked is determined.
The method according to any one of claims 1-8, wherein the object to be labeled is a point cloud image, the labeling task type includes a point cloud object labeling task, and the target labeling corresponding to the point cloud object labeling task The tool includes a plurality of point cloud object frame selection tools, and the multiple point cloud object frame selection tools correspond to different point cloud object attributes;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to a point cloud object marquee tool selection operation, determine a target point cloud object marquee tool in the plurality of point cloud object marquee tools;

In response to the frame selection operation based on the target point cloud object frame selection tool, the framed area is used as the location of the marked point cloud object, and the point cloud object attribute corresponding to the target point cloud frame selection tool is used as the label Properties of the point cloud object.
The method according to any one of claims 1-9, wherein the labeling task type includes a passenger attribute labeling task, and the target labeling tools corresponding to the passenger attribute labeling task include a rectangular frame tool and a passenger attribute labeling tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the position of the marked passenger;

In response to a passenger attribute selection operation based on the passenger attribute annotation tool, at least one attribute of the annotated passenger is determined.
The method according to any one of claims 1-10, wherein the labeling task type includes a human face labeling task, and the target labeling tools corresponding to the human face labeling task include a rectangular frame tool and a key point labeling tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the location of the marked human face;

In response to the key point labeling operation based on the key point labeling tool, the key points of the human face in the labeled human face are determined.
The method according to any one of claims 1-11, wherein the labeling task type includes a gesture labeling task, and the target labeling tools corresponding to the gesture labeling task include a rectangular frame tool and a key point labeling tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the rectangle frame drawing operation based on the rectangle frame tool, the area surrounded by the drawn rectangle frame is used as the position of the marked hand;

In response to a key point labeling operation based on the key point labeling tool, key points of the marked hand are determined.
The method according to any one of claims 1-12, wherein the labeling task type includes a driving fatigue labeling task, and the target labeling tools corresponding to the driving fatigue task include a rectangular frame tool and a fatigue attribute labeling tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the position of the marked fatigue detection target;

In response to the key point labeling operation based on the fatigue property labeling tool, the property of the labelled fatigue detection target is determined.
The method according to any one of claims 1-13, wherein the labeling task type includes a dangerous driving labeling task, and the target labeling tools corresponding to the dangerous driving task include a rectangular frame tool and a dangerous attribute labeling tool;

The obtaining the labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool includes:

In response to the rectangular frame drawing operation based on the rectangular frame tool, the area surrounded by the drawn rectangular frame is used as the position of the marked dangerous driving detection target;

In response to the attribute selection operation based on the dangerous attribute tagging tool, the tagged attribute of the dangerous driving detection target is determined.
A data labeling device is characterized in that it comprises:

An annotation object acquisition unit, configured to acquire an object to be annotated;

A target labeling tool display unit, configured to present a target labeling tool corresponding to the labeling task type of the object to be labeled among the multiple labeling tools to the user interface;

A labeling information determining unit, configured to obtain labeling information of the object to be labeled in response to the labeling operation performed based on the target labeling tool.
An electronic device, characterized in that it comprises:

processor;

memory for storing processor-executable instructions;

Wherein, the processor is configured to invoke instructions stored in the memory to execute the method according to any one of claims 1-14.
A computer-readable storage medium on which computer program instructions are stored, wherein the computer program instructions implement the method according to any one of claims 1 to 14 when executed by a processor.
A computer program product, comprising computer readable codes, or a non-volatile computer readable storage medium bearing computer readable codes, when the computer readable codes are run in a processor of an electronic device, the electronic The processor in the device is configured to implement the method of any one of claims 1-14.