CN111291813B

CN111291813B - Image labeling method, device, computer equipment and storage medium

Info

Publication number: CN111291813B
Application number: CN202010090121.5A
Authority: CN
Inventors: 郭泽豪
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2020-02-13
Filing date: 2020-02-13
Publication date: 2023-10-31
Anticipated expiration: 2040-02-13
Also published as: CN111291813A

Abstract

The application relates to an image labeling method, an image labeling device, computer equipment and a storage medium, wherein the method comprises the following steps: taking a target image set, wherein the target image set comprises images corresponding to a plurality of layers obtained by image acquisition of a target part; acquiring position information of the target object in the first image as reference position information; acquiring a relative position relationship between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image; determining second position information of the target object on the second image according to the relative position relation and the reference position information; and carrying out object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, so as to carry out object labeling on the second image according to the recognition result and the second position information. The method can improve the object labeling efficiency and accuracy.

Description

Image labeling method, device, computer equipment and storage medium

Technical Field

The present application relates to the field of image recognition, and in particular, to an image labeling method, an image labeling device, a computer device, and a storage medium.

Background

With the development of technology, images are increasingly used. In many scenarios, it is necessary to identify an object in an image, to determine the position of the object in the image, and to annotate the object on the image.

Currently, the position of an object in an image is usually marked manually. For example, for medical images, it is often necessary for a physician to empirically label the location of a target object in the medical image. However, the number of images to be marked is very large, for example, hundreds of images, and marking personnel need to mark the images one by one, so that the marking efficiency of objects is low.

Disclosure of Invention

In view of the foregoing, it is desirable to provide an image labeling method, apparatus, computer device, and storage medium that address the problem of low object labeling efficiency.

A method of image annotation, the method comprising: acquiring a target image set, wherein the target image set comprises images corresponding to a plurality of layers obtained by image acquisition of a target part, the target image set comprises a first image of the position of a determined target object and a second image of the position of the target object to be determined, the target object is positioned on the target part, and the second image and the first image correspond to different layers of the target part; acquiring position information of the target object in the first image as reference position information; acquiring a relative position relationship between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image; determining second position information of the target object on the second image according to the relative position relation and the reference position information; and carrying out object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, so as to carry out object labeling on the second image according to the recognition result and the second position information.

An image annotation device, the device comprising: the system comprises a target image set acquisition module, a target image acquisition module and a target image processing module, wherein the target image set comprises images which are obtained by image acquisition of a target part and respectively correspond to a plurality of layers, the target image set comprises a first image of the position of a determined target object and a second image of the position of the target object to be determined, the target object is positioned on the target part, and the second image and the first image correspond to different layers of the target part; a reference position information acquisition module, configured to acquire position information of the target object in the first image as reference position information; the relative position relation acquisition module is used for acquiring the relative position relation between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image; a second position information determining module, configured to determine second position information of the target object on the second image according to the relative position relationship and the reference position information; the recognition result obtaining module is used for carrying out object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, and carrying out object labeling on the second image according to the recognition result and the second position information.

In some embodiments, the reference position information includes a first size of the target object on the first image and a first coordinate of the target object on the first image, and the second position information determining module includes: a second size determining unit configured to determine a second size of the target object on the second image according to the first size and the relative positional relationship; and a second position information determining unit configured to determine second position information of the target object on the second image according to the first coordinate and the second size.

In some embodiments, the relative positional relationship comprises a first level distance of the first level and the second level, the second dimension determining unit being configured to: obtaining a size reduction parameter according to the first layer distance, wherein the size reduction parameter and the first layer distance form a positive correlation; and carrying out reduction processing on the first size according to the size reduction parameter to obtain a second size.

In some embodiments, the first dimension includes a first length and a first width, and the second dimension determining unit is configured to: obtaining larger values in the first length and the first width to obtain a reference size; and obtaining the size reduction parameter according to the first layer distance and the reference size, wherein the size reduction parameter and the reference size form a negative correlation.

In some embodiments, the second location information determining unit is configured to; determining edge coordinates of the target object on the second image according to the first coordinates and the second size, wherein the edge coordinates are coordinates corresponding to preset edge positions, and the preset edge positions are multiple; and taking each edge coordinate as second position information of the target object on the second image.

In some embodiments, the recognition result obtaining module is configured to: acquiring an image area corresponding to the second position information from the second image as a target image area; and carrying out object recognition according to the image information of the target image area to obtain a recognition result of the target object in the second image.

In some embodiments, the recognition result obtaining module includes: a pixel threshold obtaining unit, configured to obtain a pixel threshold according to a pixel value of a pixel point in the target image area; the transformation unit is used for transforming the pixel value smaller than the pixel threshold value into a first pixel value and the pixel value larger than the pixel threshold value into a second pixel value in the initial pixel value set corresponding to the target image area to obtain a target pixel value set; the statistics unit is used for carrying out statistics processing on the pixel values in the target pixel value set to obtain pixel statistics values; and the identification result obtaining unit is used for obtaining the identification result of the target object in the second image according to the pixel statistic value.

In some embodiments, the recognition result obtaining unit is configured to: and when the pixel statistic value is determined to be in the preset pixel value range, determining that the target object is included in the second image.

In some embodiments, the obtaining module of the position information of the target object in the first image is configured to: receiving a position incoming request sent by a terminal; extracting the position information of the target object in the first image from the position input request; when receiving an object labeling operation on the first image, the terminal acquires labeling position information corresponding to the object labeling operation, and the labeling position information is used as the position information of the target object in the first image and triggers the position information input request.

In some embodiments, the device further includes a sending module, configured to send the second image and the second location information to the terminal when the identification result indicates that the second image includes the target object, so that the terminal marks the second image according to the second location information, and obtains a marked second image.

In some embodiments, the reference position information includes a first size of the target object on the first image, and the apparatus further includes a second image acquisition module configured to determine a second aspect distance from the first size, the second aspect distance being in positive correlation with the first size; determining a layer surface with a distance from the first layer surface smaller than the distance from the second layer surface as a second layer surface; and acquiring an image corresponding to the second layer from the target image set to be used as a second image of the position of the target object to be determined.

A computer device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to perform the steps of the image annotation method described above.

A computer readable storage medium having stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of the image annotation method described above.

According to the image labeling method, the device, the computer equipment and the storage medium, the target image set comprises images corresponding to a plurality of layers obtained by image acquisition of the target part, the position information of the target object is used as reference position information in a first image of the determined position of the target object in the target image set, and the second position information of the target object on the second image is determined according to the relative position relation between the first layer corresponding to the first image and the second layer corresponding to the second image and the reference position information, so that object identification can be carried out according to the second position information, an identification result of the target object in the second image is obtained, and object labeling is carried out on the second image according to the identification result and the second position information. The position information of the target object in the first image can reflect the relevance of the target object on different layers of the target part, and the relative position relation can reflect the difference of the target object on different layers, so that the position information of the target object in the second image can be accurately obtained based on the relative position relation and the reference position information, the second image is subjected to object labeling based on the identification result and the second position information, and the object labeling efficiency and accuracy can be improved.

Drawings

FIG. 1 is an application environment diagram of an image annotation method provided in some embodiments;

FIG. 2 is a flow chart of an image annotation method in some embodiments;

FIG. 3 is a schematic diagram of a CT apparatus performing a tomographic scan in some embodiments;

FIG. 4 is a schematic diagram of receiving and labeling object labeling operations in some embodiments;

FIG. 5 is a schematic diagram of a second image after label processing in some embodiments;

FIG. 6 is a flowchart of obtaining second position information of a target object on a second image according to a relative position relationship and reference position information in some embodiments;

FIG. 7 is a schematic diagram of the position of an image of a target object on different layers in some embodiments;

FIG. 8 is a schematic representation of the location of a target image region in some embodiments;

FIG. 9 is a block diagram of an image annotation device in some embodiments;

FIG. 10 is a block diagram of the internal architecture of a computer device in some embodiments.

Detailed Description

The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.

It will be understood that the terms "first," "second," and the like, as used herein, may be used to describe various elements, but these elements are not limited by these terms unless otherwise specified. These terms are only used to distinguish one element from another element. For example, a first image may be referred to as a second image, and similarly, a second image may be referred to as a first image, without departing from the scope of the application.

Fig. 1 is a diagram of an application environment of an image labeling method provided in some embodiments, as shown in fig. 1, in the application environment, including a terminal 110 and a server 120. Taking the target site as the lung and the target object as a lung nodule as an example, an image sequence obtained by CT (Computed Tomography ) scanning of the lung of a certain person, for example, 300 images, is stored in the server 120. When a doctor needs to label the lung nodules on the images, the doctor can view the images through the terminal 110, one image (first image) of the lung nodules obtained by CT scanning can be displayed on the terminal 110, when the doctor finds that the first image comprises the lung nodules, the doctor labels the lung nodules on the first image, for example, a label frame is drawn on the first image to represent the positions of the lung nodules, the terminal 110 detects the lung nodule labeling operation, obtains the position information of the lung nodules in the first image according to the lung nodule labeling operation, and sends the position information of the lung nodules in the first image to the server 120. Because a lung nodule generally spans multiple layers, the server 120 may execute the method provided in the embodiment of the present application, derive a location (second location information) corresponding to another image (second image) of the lung nodule marked by the doctor in the image sequence, identify the second image on the second location information, and if the second location information is identified that the lung nodule also exists on the second image, determine that the lung nodule marked by the doctor also exists on the second image, so the terminal 110 or the server 120 may add a marking frame to an image area corresponding to the second location information in the second image, thereby, after the doctor marks the location of the lung nodule in one image, derive the location of the lung nodule in the other image and perform automatic marking. When the doctor switches the image to be viewed as the second image, the terminal 110 displays the second image with the labeling frame and the label of the lung nodule, for example, the labeling serial number, so that the doctor can not only view that the lung nodule exists in the second image, but also determine that the lung nodule on the second image is the same as the lung nodule labeled on the first image.

It may be appreciated that the image labeling method may be performed in the terminal 110, for example, the terminal 110 may obtain an image sequence in advance, and perform the image labeling method provided by the embodiment of the present application.

The server 120 may be an independent physical server, or may be a server cluster formed by a plurality of physical servers, or may be a cloud server that provides basic cloud computing services such as a cloud server, a cloud database, a cloud storage, or a CDN. The terminal 110 may be, but is not limited to, a smart phone, a tablet computer, a notebook computer, a desktop computer, or the like. The terminal 110 and the server 120 may be connected through a communication connection manner such as a network, which is not limited herein.

As shown in fig. 2, in some embodiments, an image labeling method is proposed, and this embodiment is mainly exemplified by the application of the method to the server 120 in fig. 1. The method specifically comprises the following steps:

step S202, a target image set is obtained, wherein the target image set comprises images corresponding to a plurality of layers obtained by image acquisition of a target part, the target image set comprises a first image of the position of a determined target object and a second image of the position of the target object to be determined, the target object is positioned on the target part, and the second image corresponds to different layers of the target part with the first image.

In particular, the target image set may comprise a plurality of images, "plurality" meaning at least two. The target site may be a human or animal site, and may be any site where object recognition is required. Such as the lungs or thyroid gland, etc. The first image and the second image are images acquired by using an imaging device to acquire the same target part, but the corresponding layers are different, for example, the first image and the second image are acquired by using a CT device to scan. The image of the set of target images may be a medical image, for example an image obtained by a cross-sectional scan with a CT imaging device. The CT imaging device can utilize the X-ray beam to scan the layer with a certain thickness on the part of the human body, the detector receives the X-rays transmitted through the layer, the X-ray attenuation coefficient or absorption coefficient is obtained by calculating the information obtained by scanning, and the pixel value of each pixel point can be obtained according to the X-ray attenuation coefficient or absorption coefficient, so that a CT image is obtained. The target object may be any object, for example, a nodule, a blood vessel, or a tissue, etc., such as a lung nodule, thyroid nodule, etc.

The layer plane refers to a layer having a certain thickness in the target site. That is, when the target portion is scanned, the target portion is divided into a plurality of layers, and each layer is scanned to obtain an image as an image corresponding to one layer, so that an image sequence is formed. The image sequence includes a plurality of images. The first image may be one or more images of a sequence of images. The second image is one or more images of the image sequence. The first image is an image in which the position of the target object in the image has been determined, and for example, may be an image in which the position of the target object has been manually noted, while the second image may be an image in which the position of the target object has not been manually noted. It will be appreciated that since multiple target objects may be included in one image, for example multiple lung nodules may be present in one image, the first image in which the location of the target object has been determined may be for a particular lung nodule or nodules. For example, if the location of the A-nodule at the t1 image is determined, then for the A-nodule, the t1 image is the first image of the determined location of the A-nodule. If the position of the B nodule at the t2 image is determined, but the position of the B nodule at the t1 image is not determined, then for the B nodule, the t2 image is the first image of the determined position of the B nodule and the t1 image may be the second image.

As shown in fig. 3, in some embodiments, the CT apparatus performs a tomographic scan, and the arrow indicates the direction of the x-ray, so that the CT apparatus may scan layer by layer, and a plane represents a layer, so that a multi-layer image may be obtained, and an image of a layer may also be referred to as a layer.

Step S204, obtaining the position information of the target object in the first image as reference position information.

Specifically, the position information is used to represent the position of the object. The position information may be represented by coordinates or by a combination of coordinates and dimensions. The dimensions may be expressed in terms of radius or length, etc. For example, assuming that the shape of the labeling frame corresponding to the target object is a rectangle, the position information may include coordinates of a center point of the rectangle, a length of the rectangle, and width information. Assuming that the shape of the labeling frame corresponding to the target object is a circle, the position information may include a rectangular center coordinate and a radius.

In some embodiments, the location information of the target object in the first image may be obtained in advance or may be obtained in real time. For example, manual labeling may be performed in advance, reference position information is obtained and stored, and position information may be obtained in real time according to an object labeling operation of a user.

In some embodiments, a first image may be displayed in the terminal, and when the terminal receives an object labeling operation on the first image, labeling position information corresponding to the object labeling operation is obtained as position information of a target object in the first image, and a position information incoming request is triggered. Therefore, the server can receive the position information input request sent by the terminal, and the position information input request carries the position information of the target object in the first image, so that the position information of the target object in the first image can be extracted from the position input request.

Specifically, the object labeling operation refers to an operation of labeling an object, and may be at least one of a touch operation, a voice operation, an operation through an input device such as a mouse, or a gesture operation. The labeling position information is position information corresponding to the index operation. For example, a start position and an end position of a user touch operation may be acquired. The starting position and the ending position are taken as two opposite corners in the rectangle, so that the rectangle is obtained. The coordinates of the center point of the rectangle and the size of the rectangle are taken as reference position information. For another example, the user's voice "200 th row 400 th column, 50 pixels long, 100 pixels wide" may be received, and coordinates (200, 400), 50 pixels long, and 100 pixels wide may be used as the reference position information.

The location incoming request is for requesting an incoming location, the location incoming information carrying location information of the target object in the first image. The location information of the target object in the first image can thus be extracted from the location incoming request.

In some embodiments, the terminal may obtain a first image from the server in response to an image display operation of the user, and display the first image. For example, when a doctor needs to view a medical image, an acquisition request for acquiring the medical image of a patient may be sent to a server, and the server may acquire an image sequence of a target portion of a target user in response to the acquisition request, or may return the image sequence to a terminal. It is also possible to return part of the image, for example an image, to the terminal first. When the terminal receives the image switching operation of the user, a request for switching the images is sent to the server, and the server returns the images to be switched to in the image sequence.

By automatically marking other images in response to the position information corresponding to the object marking operation, a user such as a doctor can mark any one of the images, and the computer equipment such as a server can automatically acquire the positions of the marked target objects in the other images according to the marked positions, so that the marking time of the user can be reduced, and the marking efficiency can be improved.

Step S206, obtaining the relative position relationship between the first layer surface corresponding to the first image and the second layer surface corresponding to the second image.

Specifically, the first image is an image obtained by image-capturing a first layer of the target portion, and the second image is an image obtained by image-capturing a second layer of the target portion. The relative positional relationship indicates the relative position between the layers. The relative positional relationship of the first level and the second level may be expressed in terms of the distance between the levels. For example, assuming that the first layer is 450 layers and the second layer is 449 layers, the thickness between layers is 1mm, and the relative positional relationship is 1mm. Or a three-dimensional coordinate system is established by taking a certain position point of the first layer as the origin of the coordinate axis, wherein the plane of the first layer is the plane of the x axis and the y axis. The value of the z-axis coordinate of the second level can thus be taken as the distance between the first level and the second level.

Step S208, determining second position information of the target object on the second image according to the relative position relation and the reference position information.

In particular, the second location information may be represented in coordinates. Assume that the label box is rectangular. The leftmost coordinates, rightmost coordinates, uppermost coordinates, and lowermost coordinates of the rectangle may be acquired as the second position information. Or may be a combination of coordinates and a dimensional representation. For example, the second location information may include coordinates of the center point and a size.

Since the target objects are in different layers, the occupied ranges are generally different. For example, a nodule may be progressively outwardly extending so that the nodule is larger on the middle and smaller on the sides. Therefore, in the middle layer, the occupied range is large, and therefore, the size of the range of the target object at the second level can be determined according to the relative positional relationship. The size of the range may be expressed in terms of dimensions, such as length and width. The position of the target object on the second layer is obtained according to the position of the target object on the first image, so that the second position information can be obtained according to the coordinates and the size of the range, for example, the x-axis coordinate and the y-axis coordinate of the middle point of the target object on the first image can be obtained as the coordinates of the middle point of the target object on the second image. And then obtaining second position information of the target object on the second image according to the size of the range of the target object on the second layer and the coordinates of the middle point.

Step S210, performing object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, so as to perform object labeling on the second image according to the recognition result and the second position information.

Specifically, the recognition result may be one of including or not including the target object. Since the second position information is obtained according to the relative position relationship and the reference position information, that is, the position obtained according to the algorithm, but actually, whether the second position has the target object needs to be further determined, after the second position information of the target object is determined, an image area corresponding to the second position information in the second image can be obtained, and object recognition can be performed on the image area to determine whether the target object exists in the image area. The recognition algorithm may be identified using an artificial intelligence model, such as an object recognition model. The calculation may be performed based on the pixel value of the image area corresponding to the second position information, and the recognition result may be determined based on the calculation result.

When the identification result includes that the second image includes the target object, the second image may be marked according to the second position information, the server may be used for marking, or the terminal may be used for marking, for example, when the identification result includes that the second image includes the target object, the server sends the second image and the second position information to the terminal, so that the terminal marks the second image according to the second position information, obtains a marked second image, and the terminal displays the marked second image.

In some embodiments, the image annotation method further comprises: when the identification result is that the second image comprises the target object, acquiring a target object identifier corresponding to the target object; the server or the terminal may add the target object identification on the second image, e.g. the server may return the target object identification to the terminal, which adds the target object identification on the second image. Therefore, the second image after the labeling processing comprises a labeling element for labeling the object and a target object identifier, and the labeling element is generated according to the second position information.

Specifically, object identification is used to identify the object. The object identification may be obtained according to the order of the object labeling operations or according to the user's object identification input operations. For example, each lung nodule may be named sequentially in the order of the objects marked by the user, and if a doctor finds a lung nodule and marks it when viewing the lung nodule image, the lung nodule is marked with 1, and if a lung nodule exists in another layer according to the position information of the lung nodule marked with 1 and the relative position relationship between layers, the lung nodule is marked with 1. For example, when the object labeling operation of the user is received, the terminal may pop up the input box, the user may input the name of the target object in the input box, the terminal carries the name of the target object in the position input request, and when the target object exists in another layer according to the position and the relative position relationship of the target object labeled by the user, the lung nodule identifier is the name input by the user, so when the user switches to the second image of the second layer, the target object and the target object in the first image may be confirmed to be the same object according to the target object identifier displayed on the second image.

In some embodiments, the labeling element in the second image after labeling may be movable, so that a movement operation of the labeling element by the user may be received, and the labeling element is moved according to the movement operation, so that the user may adjust the position of the target object on the second image, so that the labeling position is more accurate.

As shown in FIG. 4, a schematic diagram of a received object annotation operation and annotation is provided in some embodiments. The terminal may have a lung nodule image (first image) of layer 450 displayed thereon. When the user finds a lung nodule on the image, the user may perform an object labeling operation, such as clicking on the location of the lung nodule. After the terminal receives the click operation, labeling elements, such as labeling frame and object identifiers "3" and "4", can be generated at the click position, which represent lung nodules of the 3 rd label and the 4 th label of the user. After labeling by the user, a schematic diagram of the second image after labeling processing is shown in fig. 5 in some embodiments. According to the method provided by the embodiment of the application, the lung nodules 3 are calculated to exist at layers 451 and 452, and the lung nodules 4 are calculated to exist at layer 449. Therefore, a label box and an object identifier can be added to the images of the 449 layer, the 451 layer, and the 452 layer. Wherein, the labeling elements of different layers can be the same or different.

According to the image labeling method, the target image set is the image corresponding to the target part, the obtained images with the multiple layers respectively are used for acquiring the position information of the target object in the first image, which is determined in the target image set, as the reference position information, and the second position information of the target object on the second image is determined according to the relative position relation between the first layer corresponding to the first image and the second layer corresponding to the second image and the reference position information, so that object recognition can be performed according to the second position information, the recognition result of the target object in the second image is obtained, and object labeling is performed on the second image according to the recognition result and the second position information. The position information of the target object in the first image can reflect the relevance of the target object on different layers of the target part, and the relative position relation can reflect the difference of the target object on different layers, so that the position information of the target object in the second image can be accurately obtained based on the relative position relation and the reference position information, the second image is subjected to object labeling based on the identification result and the second position information, and the object labeling efficiency and accuracy can be improved.

In some embodiments, since multiple images may be included in the target image set, when the position information corresponding to one of the images (the first image) in the sequence is received, all other images in the target image set may be used as the second image. It is also possible to take part of the images in the target image set as the second image. For example, the reference location information comprises a first size of the target object on the first image, and determining a second image of the location of the target object to be determined from the set of target images comprises: determining a second layer distance according to the first dimension, wherein the second layer distance and the first dimension form a positive correlation; determining a layer surface with a distance from the first layer surface smaller than a distance from the second layer surface as the second layer surface; and acquiring an image corresponding to the second layer from the target image set to serve as a second image of the position of the target object to be determined.

In particular, the distance between layers refers to the distance between layers, which may be expressed, for example, in terms of the thickness between layers. As a practical example, in tomographic scanning, each layer has a certain thickness, and thus the layer thickness can be multiplied by the number of layers separated from two layers to obtain the distance between the two layers.

The positive correlation means that the correlation is positive, and the larger the first dimension, the larger the second-level distance, with the other parameters unchanged. When there are a plurality of first dimensions, the target deck distance may be determined according to one or more of the dimensions. For example, if the first dimension includes a length and a width, a larger value of the length and the width may be obtained, and the second deck distance may be obtained according to the larger value, e.g., the larger value may be multiplied or added to a preset value, to obtain the second deck distance. Or an average value of the length and the width can be obtained, and the average value is multiplied by a preset value to obtain the second layer distance. The preset value may need to be set, e.g. the preset value may be a factor of less than 1, e.g. 0.5.

After the second layer distance is obtained, a layer with the distance from the first layer smaller than the second layer distance can be used as the second layer, and then an image corresponding to the second layer is obtained to obtain a second image. As a practical example, assuming that the first dimension comprises a length of 5mm and a width of 3mm, the larger value of 5mm may be multiplied by a preset factor of 1/2, yielding 2.5mm. Assuming that the first image is located at the 450 th layer and the thickness of one layer is 1mm, 448 th, 449 th, 451 th and 452 th layers having a distance from the 450 th layer of less than 2.5mm may be used as the second layer. Since the extension of the target object such as a nodule at different levels is not infinite, by using a level smaller than the second level distance as the second level, the amount of position calculation can be reduced, and the calculation efficiency can be improved. The reference dimension is in positive correlation with the first dimension, so that the second-level distance can be adjusted adaptively according to the dimension of the target object such as a nodule in the first image, and the flexibility and the accuracy are high. For example, the larger the size of a nodule marked by a doctor in a first image, the more levels the nodule extends, and the more distant levels may have nodules, so the greater the second level distance.

In some embodiments, the reference position information includes a first size of the target object on the first image and a first coordinate of the target object on the first image, and determining, based on the relative positional relationship and the reference position information, the second position information of the target object on the second image includes:

step S602, determining a second size of the target object on the second image according to the first size and the relative positional relationship.

Specifically, the relative positional relationship may be the first-level distance. The first deck distance refers to the distance separating the first deck from the second deck. For example, assuming a layer thickness of 2mm (millimeters) separated by 2 layers, the layer distance is 4mm. The change in the second dimension relative to the first dimension may be determined based on the first face distance. The second dimension is thus derived from the first dimension and the variation, which may be determined in particular according to the situation of the particular target object. For example, the size of a nodule is not constant, but gradually decreases, and the physician marks the location, typically where the nodule is apparent, and thus it is considered that the size of the nodule in images at other levels decreases. The change in dimension can thus be determined from the first deck distance, the larger the change in dimension.

In some embodiments, the size reduction parameter may be obtained according to the first layer distance, and the first size is reduced according to the size reduction parameter to obtain the second size. The size reduction parameter is used for performing a size reduction process, and the larger the size reduction parameter is, the more the size is reduced. The downsizing parameter is in positive correlation with the first face distance, so the larger the first face distance is, the larger the downsizing parameter is. I.e. the larger the first level distance, the larger the downsizing parameter and thus the more downsizing, i.e. the smaller the second size of the target object. The correspondence between the first-level distance and the downsizing parameter may be determined in advance according to a change condition of the target object. In the embodiment of the application, when the position of the target object of the first image is marked or identified by the user, the position is generally the position where the target object is obvious, namely the position with larger size, so that the size of the target object in the images of other layers can be considered to be reduced along with the increasing of the distance from the first layer, and the size reduction parameter corresponding to the first size is in positive correlation with the distance from the first layer, so that the accuracy of the obtained second size can be improved.

In some embodiments, the downsizing parameter may also be derived in combination with the first size, e.g. the first size may be inversely related to the downsizing parameter. I.e. the larger the first size, the smaller the size reduction relationship may be, with the other parameters unchanged.

In some embodiments, the first dimension includes a first length and a first width, and deriving the downscaling parameter from the first deck distance, the downscaling parameter having a positive correlation with distance includes: acquiring larger values in the first length and the first width to obtain a reference size; and obtaining a size reduction parameter according to the first layer distance and the reference size, wherein the size reduction parameter and the reference size form a negative correlation.

Specifically, the first length and the first width refer to the length and the width of the target object on the first image, respectively. The larger value refers to a larger value of the first length and the first width. For example, assuming that the first length is greater than the first width, the first length is the reference dimension. The downsizing parameters may be derived in combination with the reference dimension and the first level distance. For example, a ratio of the first-layer distance divided by the reference size may be used as the size reduction parameter. The square of the ratio of the first-layer distance divided by the reference size may also be obtained as a size reduction parameter. In the embodiment of the application, the size reduction parameter and the reference size are in a negative correlation, so that the larger the reference size is, the smaller the size reduction parameter is, and the slower the nodule is reduced. I.e. the larger the lung nodule, the slower its size is reduced, which results in a more accurate second size due to the consideration of the reference size determining size reduction parameter.

After the size reduction parameter is obtained, the first size may be reduced by using the size reduction parameter. For example. The size reduction parameter may be subtracted from the preset parameter, and the resulting value may be multiplied by the first size as a coefficient to obtain the second size. The size reduction parameter may be subtracted from a preset parameter, and then the square is opened, and the obtained value is multiplied by the first size as a coefficient to obtain the second size. The preset parameter is, for example, 1.

For example, for a lung nodule, the formulas for calculating the second dimension may be represented by formulas 1 and 2, where the second dimension includes a second length and a second widthI.e. the length and width of the target object in the second image. Assuming that the second image is located at the j-th layerRepresenting the length of the target object in the image of the i-j th layer (second layer) (second image), +.>The length of the target object in the image of the i-th layer (first layer) (first image) is represented. X represents the X-axis. />Representing the width of the target object in the image of the i-j layer (second layer) (second image). />The width of the target object in the image (first image) of the i-th layer (first layer) is represented, and y represents the y-axis. z _i-j Representing the distance between the i-j layer and the j layer, wherein the z-axis value of the first layer can be set to 0 when the coordinate axis is established, then z _i-j The coordinate values of the z-axis of the i-j-th layer may be represented. /> Representation +.>And->Is a larger value of (a).

In step S604, second position information of the target object on the second image is determined according to the first coordinates and the second size.

Specifically, the first coordinate may be a coordinate of a preset point, for example, a coordinate of a center point or a coordinate of a certain point located at an edge of the target object in the first image, and may be specifically set according to needs. For a nodule, the coordinates of its center may be considered constant, extending along the center line to both sides. The first coordinates can thus be taken as coordinates of the target object on the second image. Since the first coordinates of the preset point have been determined. The position information of the target object on the second image can thus be derived from the first coordinates and the second size.

In some embodiments, determining, according to the first coordinates and the second dimensions, edge coordinates of the target object on the second image, the edge coordinates being coordinates corresponding to preset edge positions, the preset edge positions being a plurality of; and taking the respective edge coordinates as second position information of the target object on the second image.

Specifically, a plurality means at least two. The edge coordinates are coordinates corresponding to the edge position of the target object, for example, assuming that the target object is a rectangle, the edge may include coordinates corresponding to four sides, and may include coordinates of the leftmost side, the rightmost side, the uppermost side, and the lowermost side of the rectangle. Therefore, a corresponding calculation formula can be preset according to the position of the first coordinate in the target object, and the first coordinate and the second size are calculated according to the calculation formula to obtain the edge coordinate.

In some embodiments, assuming that the user annotates at the ith layer of the lung nodule image sequence, the ith layer of the lung nodule sequence is denoted as N _i The x-axis coordinate of the center point of the rectangular frame marked by the user at the ith layer (first layer) isThe y-axis coordinate is +.>The x-axis coordinate of the target object at the leftmost (x-axis minimum) level in the i-j-th level (second level) is denoted +.>The rightmost x-axis coordinate (x-axis maximum) is expressed as +.>The uppermost y-axis coordinate (y-axis maximum) is expressed asThe lowest y-axis coordinate (y-axis minimum) is expressed as +.>The length of the nodule of the ith layer in the x-axis is marked +.>The length on the y-axis is marked +.>The z-axis coordinate of the i-th layer is 0. The z-axis seating of the ith-jth layer is marked z _i-j The parameter c represents the larger value of the length and width of the marked frame marked by the user at the ith layer, and is marked as +.>The calculation formulas of the values of the respective edge coordinates of the second layer can be expressed as formulas (3) to (6). Where i is a positive integer and j may be a positive integer or a negative integer.

FIG. 7 is a schematic diagram of the location of images of a target object on different layers in some embodiments. In fig. 7, the region formed by the plurality of lattices represents the region occupied by the target object. As shown in fig. 7, the coordinates of the center points of the nodules in the layers are consistent, and are the x-axis coordinate value and the z-axis coordinate value corresponding to the center position of the target object in the i-th layer, that is, the positions where the dashed lines pass through in fig. 7. Since the position of the target object of the i-j layer is obtained from the position of the i-layer, and the size of the object marked by the user is generally the largest, the size of the target object of the i-layer is larger than the size of the target object in the i-j layer.

In some embodiments, performing object recognition on the second image according to the second position information, and obtaining a recognition result of the target object in the second image includes: acquiring an image area corresponding to the second position information from the second image as a target image area; and carrying out object recognition according to the image information of the target image area to obtain a recognition result of the target object in the second image.

In particular, the image information may include pixel values of the image. After the second position information is obtained, an image of the region corresponding to the second position information can be used as the image region to be identified. The target image area is input into the model for recognition. The image area may also be processed, for example binarized. And obtaining a recognition result according to the binarization processing result. For example, as shown in fig. 8, if the region corresponding to the second position information in the second image is obtained as the region a, the region a in the second image is taken as the target image region.

In some implementations, performing object recognition according to image information of the target image area, and obtaining a recognition result of the target object in the second image includes: obtaining a pixel threshold according to the pixel value of the pixel point in the target image area; transforming the pixel value smaller than the pixel threshold value in the initial pixel value set corresponding to the target image area into a first pixel value, and transforming the pixel value larger than the pixel threshold value into a second pixel value to obtain a target pixel value set; carrying out statistical processing on pixel values in the target pixel value set to obtain pixel statistical values; and identifying the target object in the second image according to the pixel statistic value.

Specifically, the pixel value in the initial pixel value set is the pixel value of each pixel point in the target image area, and may be a gray value. It may be to acquire a median value or an average value of pixel values of the target image area as the pixel threshold value. The pixel value may be obtained from the difference between the maximum value and the minimum value of the pixel value. For example, the pixel value adjustment value may be added to the minimum value so that the pixel value threshold value is located between the maximum value and the minimum value. The pixel value adjusting value is obtained according to the difference value between the maximum value and the minimum value in the target image area, so that the pixel threshold value can be adjusted in a self-adaptive mode. For example, the difference between the maximum value and the minimum value and the preset coefficient may be multiplied, and the preset coefficient may be, for example, 0.3 as the pixel value adjustment value.

For example, for the i-j th layer, the pixel values of the target image region may be obtained to generate a pixel matrixThe value of the a-th row and the b-th column in the target image area is the pixel value of the pixel point of the a-th row and the b-th column. By pixel matrix->Is calculated as a binarization threshold value +.>The formula of (2) is shown as formula (7). Wherein (1)>Representing the minimum of pixel values in the target image area of the i-j-th layer. / >Representing the maximum value of the pixel values in the target image area of the i-j-th layer.

The first pixel value and the second pixel value may be set as needed, the second pixel value being greater than the first pixel value. For example, the first pixel value may be 0 and the second pixel value may be 1. Thus according to pixel thresholdPixel matrix comprising initial set of pixel values>Binary matrix formed by converting into target pixel value set>The formula of (2) is shown as formula (8)

The pixel statistics may be an average. The formula for deriving the pixel statistics may be as shown in formula (9),length of target object in image (second image) representing the i-j layer (second layer), +.>The width of the target object representing the image (second image) of the i-j-th layer (second layer). g _i-j Representing the i-j th layerAnd (3) counting the pixel corresponding to the target image area. />Representation pair->The pixel values in the matrix are summed.

In some embodiments, it may be determined that the target object is included in the second image when the pixel statistics are determined to be within a preset range of pixel values. The preset pixel value range may be set as desired, and may be, for example, greater than a preset pixel threshold. For example, it may be experimentally determined that a lung nodule is present when the pixel statistics are between [0.2,0.94], i.e. between 0.2 and 0.94. Since the gray value of a target object, such as a nodule, is generally large in an image obtained by a CT scan, a pixel value larger than a pixel threshold is converted into a second pixel value, and a pixel value smaller than the threshold is converted into a first pixel value. Therefore, the overall situation of the pixel value in the target image area can be determined through counting the obtained pixel statistic value, and the identification result can be accurately obtained according to the pixel statistic value.

Taking a target object as a lung nodule, carrying out CT imaging on the lung of a patient to obtain images of a plurality of layers to form an image sequence, and automatically labeling other images of the image sequence according to the labeling of a doctor as an example, so as to explain the method provided by the embodiment of the application, which comprises the following steps:

1. and acquiring an image file corresponding to the lung nodule.

Specifically, an image of a lung nodule is generally an imaging manner of capturing an image of the lung nodule from top to bottom, and an average of one examination may generate more than 300 image files, which are called Dicom (Digital Imaging and Communications in Medicine, digital imaging and communication in medicine) files, one Dicom file corresponding to one layer of the lung nodule, so that a lung nodule containing K layers may contain K Dicom files, so that K Dicom files may be acquired.

2. And processing the image file to obtain an array file.

Specifically, the Dicom file may be read and loaded through a third party library to generate a matrix of k×512×512 and a thickness between layers, where one matrix of k×512 corresponds to one layer of the lung nodule sequence, and the image is represented as 512×512 pixels, and the matrix and the thickness of k×512×512 may be saved as an npy (numpy) file, which may be also referred to as an array file, and the matrix value may be a gray value of the image. The npy file may be encrypted by an encryption algorithm and named with the id (i.e., identification) of the image file.

Steps 1 and 2 may also be referred to as a data preprocessing process, which is offline, is responsible for processing image files that process sequences of lung nodules to generate a matrix, and may store the matrix in an encrypted file named sequence ID. The lung nodule sequence represents a layer sequence of lung nodules, i.e. lung nodules are scanned in layers.

3. And generating a first image according to the array file, and displaying the first image on the terminal.

Specifically, when an image sequence corresponding to the viewing sequence id is received, an encrypted npy file may be obtained and decrypted to obtain a matrix of k×512×512 and a thickness of a layer, and when a doctor needs to view the first image, the first image may be sent to the terminal.

4. Position information of a target object in a first image is acquired as reference position information.

Specifically, after a doctor views a certain image, if a lung nodule is found, the nodule position can be marked on the image, at this time, the marking platform receives the position of the marking frame as a reference position, and the sequence ID, the identification of the image layer and the information of the reference position are transmitted to a corresponding server.

5. A second image is acquired.

Specifically, the second layer may be determined according to the size of the labeling frame labeled by the doctor. For example, if the length of the callout box is greater than 1mm, layers within 3 layers apart may be used as the second layer. And if the length of the labeling frame is less than 1mm, acquiring layers within 2 layers apart as a second layer. And after the second image layers are obtained, obtaining images corresponding to the second image layers as second images.

6. And acquiring the relative position relationship between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image.

Specifically, the thickness of the layer can be obtained, and the distance between the first layer surface and the second layer surface is obtained according to the thickness of the layer and the number of layers which are separated.

7. And obtaining second position information of the target object on the second image according to the relative position relation and the reference position information.

In particular, the reference position information may include coordinates of an intermediate point of the target object in the first image, as well as a length and a width. And the coordinates of the intermediate point are used as the coordinates of the intermediate point of the target object on the second image, and the length and the width of the target object on the second image are calculated according to the length and the width of the target object in the first image and the distance between the first layer surface and the second layer surface. Assuming that the labeling frame surface of the lung nodule is rectangular, the maximum value and the minimum value of the lung nodule in the x axis can be obtained according to the coordinates of the middle point and the length and the width of the lung nodule in the second image, and the maximum value and the minimum value in the y axis are the edge coordinates.

8. And carrying out object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image.

Specifically, the pixel value of the image area corresponding to the rectangle may be obtained, the binarization processing is performed on the target image area, and if the average value of the gray values obtained by the binarization processing is 0.6 and the preset pixel value range is 0.2 to 0.94, it is confirmed that the lung nodule exists in the target image area corresponding to the second position information.

9. And adding a labeling frame on the target image area on the second image. And returning the second image with the marked frame when the request for displaying the second image is received.

Specifically, after labeling the first image, the doctor can switch the next image in the image sequence to be the image to be displayed currently, and if the image is the second image and the lung nodule exists, the label frame and the mark of the lung nodule, for example, 3, are added at the second position information of the image. In this way, the physician can quickly determine that a lung nodule is present on the second location information and is the same lung nodule as on the first image.

The method provided by the embodiment of the application can be applied to the labeling of the lung nodules, the labeling of the lung nodules is always a problem which puzzles a labeling doctor, an imaging mode of scanning from top to bottom is generally adopted, 1mm is used for cutting 1 image, more than 300 images can be generated after 1-time examination, a doctor needs to find out the nodules from the inside and label the nodules, the nodules are generally three-dimensional and penetrate through a plurality of image layers, if each layer needs to be manually labeled, the workload of the doctor is great, and the three-dimensional labeling of the lung nodules is realized by establishing a mathematical model for calculating the positions of the nodules and automatically deducing the positions of the upper layer and the lower layer according to the positions of the doctor labels, so that the doctor can automatically identify the positions of the lung nodules in the upper layer and the lower layer in any layer and the positions of the lung nodules are identified and field labeled by an algorithm. According to statistics, according to the method provided by the embodiment of the application, the average time for marking one examination sequence by a doctor is reduced from 30min (minutes) to 18min, so that the marking efficiency is improved.

The image sequence provided by the embodiment of the application can be presented in a 3D (three dimensional, three-dimensional) mode, so that a three-dimensional lung nodule image is presented, and when the three-dimensional lung nodule image is presented, the nodule can be marked in the three-dimensional lung nodule image according to the position information of the nodule.

As shown in fig. 9, in some embodiments, an image labeling apparatus is provided, and the image labeling apparatus may be integrated in the server 120, and specifically may include a target image set acquisition module 902, a reference location information acquisition module 904, a relative location relationship acquisition module 906, a second location information determination module 908, and an identification result obtaining module 910.

The target image set obtaining module 902 is configured to obtain a target image set, where the target image set includes images corresponding to multiple layers obtained by image capturing on a target portion, and the target image set includes a first image of a position of a determined target object and a second image of a position of the target object to be determined, the target object is located on the target portion, and the second image corresponds to different layers of the target portion with respect to the first image.

A reference position information obtaining module 904, configured to obtain position information of the target object in the first image as reference position information.

A relative position relationship obtaining module 906, configured to obtain a relative position relationship between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image;

a second position information determining module 908 is configured to determine second position information of the target object on the second image according to the relative position relationship and the reference position information.

The recognition result obtaining module 910 is configured to perform object recognition on the second image according to the second position information, obtain a recognition result of the target object in the second image, and perform object labeling on the second image according to the recognition result and the second position information.

In some embodiments, the reference location information includes a first size of the target object on the first image and a first coordinate of the target object on the first image, and the second location information determination module 908 includes: a second size determining unit configured to determine a second size of the target object on the second image based on the first size and the relative positional relationship; and a second position information determining unit for determining second position information of the target object on the second image according to the first coordinates and the second size.

In some embodiments, the relative positional relationship comprises a first level distance of a first level and a second level, the second dimension determination unit being for: obtaining a size reduction parameter according to the first layer distance, wherein the size reduction parameter and the first layer distance form a positive correlation; and carrying out reduction processing on the first size according to the size reduction parameters to obtain a second size.

In some embodiments, the first dimension includes a first length and a first width, and the second dimension determining unit is to: acquiring larger values in the first length and the first width to obtain a reference size; and obtaining a size reduction parameter according to the first layer distance and the reference size, wherein the size reduction parameter and the reference size form a negative correlation.

In some embodiments, the second location information determining unit is for; determining edge coordinates of the target object on the second image according to the first coordinates and the second size, wherein the edge coordinates are coordinates corresponding to preset edge positions, and the preset edge positions are multiple; and taking the respective edge coordinates as second position information of the target object on the second image.

In some embodiments, the recognition result obtaining module includes: a pixel threshold value obtaining unit, configured to obtain a pixel threshold value according to a pixel value of a pixel point in the target image area; the transformation unit is used for transforming the pixel value smaller than the pixel threshold value into a first pixel value and the pixel value larger than the pixel threshold value into a second pixel value in the initial pixel value set corresponding to the target image area to obtain a target pixel value set; the statistics unit is used for carrying out statistics processing on the pixel values in the target pixel value set to obtain pixel statistics values; the recognition result obtaining unit is used for obtaining the recognition result of the target object in the second image according to the pixel statistic value.

In some embodiments, the recognition result obtaining unit is configured to: and when the pixel statistic value is determined to be within the preset pixel value range, determining that the target object is included in the second image.

In some embodiments, the obtaining module of the position information of the target object in the first image is configured to: receiving a position incoming request sent by a terminal; extracting the position information of the target object in the first image from the position input request; when receiving an object labeling operation on a first image, the terminal acquires labeling position information corresponding to the object labeling operation, and takes the labeling position information as the position information of a target object in the first image and triggers a position information input request.

In some embodiments, the device further includes a sending module, configured to send the second image and the second location information to the terminal when the identification result indicates that the second image includes the target object, so that the terminal marks the second image according to the second location information, and obtains a second image after marking.

In some embodiments, the reference location information comprises a first size of the target object on the first image, the apparatus further comprising a second image acquisition module for determining a second aspect distance from the first size, the second aspect distance being in positive correlation with the first size; determining a layer surface with a distance from the first layer surface smaller than a distance from the second layer surface as the second layer surface; and acquiring an image corresponding to the second layer from the target image set to serve as a second image of the position of the target object to be determined.

FIG. 10 illustrates an internal block diagram of a computer device in some embodiments. The computer device may be specifically the server 120 of fig. 1. As shown in fig. 10, the computer device includes a processor, a memory, and a network interface connected by a system bus. The memory includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system, and may also store a computer program that, when executed by a processor, causes the processor to implement an image annotation method. The internal memory may also store a computer program that, when executed by the processor, causes the processor to perform the image annotation method.

It will be appreciated by those skilled in the art that the structure shown in FIG. 10 is merely a block diagram of some of the structures associated with the present inventive arrangements and is not limiting of the computer device to which the present inventive arrangements may be applied, and that a particular computer device may include more or fewer components than shown, or may combine some of the components, or have a different arrangement of components.

In some embodiments, the image labeling apparatus provided by the present application may be implemented in the form of a computer program that is executable on a computer device as shown in fig. 10. The memory of the computer device may store various program modules constituting the image labeling apparatus, such as the target image set acquisition module 902, the reference position information acquisition module 904, the relative position relationship acquisition module 906, the second position information determination module 908, and the recognition result obtaining module 910 shown in fig. 9.

The computer program of each program module causes a processor to execute the steps in the image labeling method of each embodiment of the present application described in the present specification.

For example, the computer device shown in fig. 10 may obtain, by using the target image set obtaining module 902 in the image labeling apparatus shown in fig. 9, a target image set, where the target image set is an image corresponding to a plurality of layers obtained by performing image collection on a target portion, and the target image set includes a first image in which a position of a target object is determined and a second image in which a position of the target object is to be determined, where the target object is located on the target portion, and the second image corresponds to a different layer of the target portion from the first image. The position information of the target object in the first image is acquired as reference position information by the reference position information acquisition module 904. Acquiring a relative position relationship between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image through a relative position relationship acquisition module 906; second positional information of the target object on the second image is determined by the second positional information determination module 908 based on the relative positional relationship and the reference positional information. The recognition result obtaining module 910 performs object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, so as to perform object labeling on the second image according to the recognition result and the second position information.

In some embodiments, a computer device is provided, comprising a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the steps of the image annotation method described above. The steps of the image labeling method herein may be the steps of the image labeling method of the above embodiments.

In some embodiments, a computer readable storage medium is provided, storing a computer program which, when executed by a processor, causes the processor to perform the steps of the image annotation method described above. The steps of the image labeling method herein may be the steps of the image labeling method of the above embodiments.

It should be understood that, although the steps in the flowcharts of the embodiments of the present application are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in various embodiments may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor do the order in which the sub-steps or stages are performed necessarily performed in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.

Those skilled in the art will appreciate that implementing all or part of the above-described methods may be accomplished by way of a computer program, which may be stored on a non-transitory computer readable storage medium and which, when executed, may comprise the steps of the above-described embodiments of the methods. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.

The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples illustrate only a few embodiments of the application and are described in detail herein without thereby limiting the scope of the application. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the application, which are all within the scope of the application. Accordingly, the scope of protection of the present application is to be determined by the appended claims.

Claims

1. A method of image annotation, the method comprising:

acquiring a target image set, wherein the target image set comprises images corresponding to a plurality of layers obtained by image acquisition of a target part, the target image set comprises a first image of the position of a determined target object, and the target object is positioned on the target part;

Acquiring position information of the target object in the first image as reference position information; the reference position information includes a first size of the target object on the first image;

determining a second deck distance from the first dimension;

determining a layer surface, of which the distance between a first layer surface corresponding to the first image is smaller than that between the second layer surface, as a second layer surface;

acquiring an image corresponding to the second layer from the target image set, and taking the image as a second image of the position of the target object to be determined;

acquiring a relative position relationship between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image; the relative positional relationship between the first and second aspects is represented by a distance between the first and second aspects;

determining second position information of the target object on the second image according to the relative position relation and the reference position information;

and carrying out object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, so as to carry out object labeling on the second image according to the recognition result and the second position information.

2. The method of claim 1, wherein the reference position information includes a first size of the target object on the first image and a first coordinate of the target object on the first image, and wherein determining the second position information of the target object on the second image based on the relative positional relationship and the reference position information includes:

determining a second size of the target object on the second image according to the first size and the relative position relation;

and determining second position information of the target object on the second image according to the first coordinates and the second size.

3. The method of claim 2, wherein the relative positional relationship comprises a first range of the first range and the second range, and wherein determining the second size of the target object on the second image based on the first size and the relative positional relationship comprises:

obtaining a size reduction parameter according to the first layer distance, wherein the size reduction parameter and the first layer distance form a positive correlation;

and carrying out reduction processing on the first size according to the size reduction parameter to obtain a second size.

4. A method according to claim 3, wherein the first dimension comprises a first length and a first width, and wherein deriving the dimension reduction parameter from the first deck distance comprises:

obtaining larger values in the first length and the first width to obtain a reference size;

and obtaining the size reduction parameter according to the first layer distance and the reference size, wherein the size reduction parameter and the reference size form a negative correlation.

5. The method of claim 2, wherein the determining second location information of the target object on the second image based on the first coordinates and the second size comprises;

determining edge coordinates of the target object on the second image according to the first coordinates and the second size, wherein the edge coordinates are coordinates corresponding to preset edge positions, and the preset edge positions are multiple;

and taking each edge coordinate as second position information of the target object on the second image.

6. The method according to claim 1, wherein the performing object recognition on the second image according to the second location information, and obtaining a recognition result of the target object in the second image includes:

Acquiring an image area corresponding to the second position information from the second image as a target image area;

and carrying out object recognition according to the image information of the target image area to obtain a recognition result of the target object in the second image.

7. The method according to claim 6, wherein the performing object recognition according to the image information of the target image area, and obtaining the recognition result of the target object in the second image includes:

obtaining a pixel threshold according to the pixel value of the pixel point in the target image area;

transforming the pixel value smaller than the pixel threshold value in the initial pixel value set corresponding to the target image area into a first pixel value, and transforming the pixel value larger than the pixel threshold value into a second pixel value, so as to obtain a target pixel value set;

carrying out statistical processing on pixel values in the target pixel value set to obtain pixel statistical values;

and obtaining the identification result of the target object in the second image according to the pixel statistic value.

8. The method of claim 7, wherein obtaining the recognition result of the target object in the second image according to the pixel statistics comprises:

And when the pixel statistic value is determined to be in the preset pixel value range, determining that the target object is included in the second image.

9. The method according to claim 1, wherein the obtaining step of the position information of the target object in the first image includes:

receiving a position incoming request sent by a terminal;

extracting the position information of the target object in the first image from the position incoming request;

when receiving an object labeling operation on the first image, the terminal acquires labeling position information corresponding to the object labeling operation as the position information of the target object in the first image, and triggers the position information input request.

10. The method according to claim 9, wherein the method further comprises:

and when the identification result is that the second image comprises the target object, the second image and the second position information are sent to the terminal, so that the terminal marks the second image according to the second position information, and a marked second image is obtained.

11. An image annotation device, the device comprising:

The system comprises a target image set acquisition module, a target image acquisition module and a target image acquisition module, wherein the target image set comprises images corresponding to a plurality of layers obtained by image acquisition of a target part, the target image set comprises a first image of the position of a determined target object, and the target object is positioned on the target part;

a reference position information acquisition module, configured to acquire position information of the target object in the first image as reference position information; the reference position information includes a first size of the target object on the first image;

the relative position relation acquisition module is used for acquiring the relative position relation between a first layer surface corresponding to the first image and a second layer surface corresponding to the second image; the relative positional relationship between the first and second aspects is represented by a distance between the first and second aspects;

a second image acquisition module for determining a second deck distance according to the first size; determining a layer surface with a distance from the first layer surface smaller than the distance from the second layer surface as a second layer surface; acquiring an image corresponding to the second layer from the target image set, and taking the image as a second image of the position of the target object to be determined;

A second position information determining module, configured to determine second position information of the target object on the second image according to the relative position relationship and the reference position information;

the recognition result obtaining module is used for carrying out object recognition on the second image according to the second position information to obtain a recognition result of the target object in the second image, and carrying out object labeling on the second image according to the recognition result and the second position information.

12. The apparatus of claim 11, wherein the reference position information comprises a first size of the target object on the first image and a first coordinate of the target object on the first image, the second position information determining module comprising:

a second size determining unit configured to determine a second size of the target object on the second image according to the first size and the relative positional relationship;

and a second position information determining unit configured to determine second position information of the target object on the second image according to the first coordinate and the second size.

13. The apparatus of claim 12, wherein the relative positional relationship comprises a first level distance of the first level and the second level, the second dimension determination unit further to:

14. The apparatus of claim 13, wherein the first dimension comprises a first length and a first width, the second dimension determining unit further to:

15. The apparatus of claim 12, wherein the second location information determining unit is further configured to:

16. The apparatus of claim 11, wherein the recognition result obtaining module is further configured to:

17. The apparatus of claim 16, wherein the recognition result obtaining module is further configured to:

18. The apparatus of claim 17, wherein the recognition result obtaining module is further configured to:

19. The apparatus of claim 11, wherein the apparatus is further configured to:

receiving a position incoming request sent by a terminal;

20. The apparatus of claim 19, wherein the apparatus is further configured to:

21. A computer device comprising a memory and a processor, the memory having stored therein a computer program which, when executed by the processor, causes the processor to perform the steps of the image annotation method according to any of claims 1 to 10.

22. A computer readable storage medium, characterized in that the computer readable storage medium has stored thereon a computer program which, when executed by a processor, causes the processor to perform the steps of the image annotation method according to any of claims 1 to 10.