CN117218633A

CN117218633A - Article detection method, device, equipment and storage medium

Info

Publication number: CN117218633A
Application number: CN202310952434.0A
Authority: CN
Inventors: 张建安; 刘微; 王昕�
Original assignee: Hisense Group Holding Co Ltd
Current assignee: Hisense Group Holding Co Ltd
Priority date: 2023-07-31
Filing date: 2023-07-31
Publication date: 2023-12-12

Abstract

The application relates to the technical field of computer vision, in particular to an article detection method, an article detection device, an article detection equipment and a storage medium, which are used for solving the problem of low article detection accuracy, and the method comprises the following steps: inputting original images obtained by shooting a plurality of articles to be detected in a visual field area into a rotary target detection model to obtain detection area positioning information corresponding to each article to be detected, wherein the rotary target detection model is obtained by training the rotary target detection model to be trained based on a plurality of sample images and detection area marking information of each sample article in the sample images, and each sample article comprises at least one article with a part of outer contour outside the shooting visual field area of the corresponding sample image; if the object to be detected with partial outline outside the visual field area exists in the original image based on the positioning information of each detection area, pixel filling is carried out on the original image, and the image of each object to be detected is segmented from the obtained target image; thus, the accuracy of article detection is improved.

Description

Article detection method, device, equipment and storage medium

Technical Field

The present application relates to the field of computer vision, and in particular, to a method, an apparatus, a device, and a storage medium for detecting an article.

Background

In an actual scene, for special occasions, such as customs security inspection, inbound security inspection and the like, security inspection is often required to be carried out on personal belongings of passengers. Rotational object detection models are often used in the above-described scenarios where security checks are required. Based on the rotation target detection model, target detection is performed on images containing a plurality of articles, so that images corresponding to the articles are obtained, and analysis, recognition and the like of the articles are completed.

However, current rotating object detection models are typically only able to detect items that are completely displayed in an image. However, in an actual scene, the detected image often contains incompletely displayed objects due to the arrangement of part of objects and other reasons, so that when the current rotating target detection model is adopted to detect the objects, the model usually does not detect the objects incompletely displayed in the image, thereby causing information loss and further causing the problem of low detection accuracy in the actual use model.

Disclosure of Invention

The embodiment of the application provides an article detection method, device, equipment and storage medium, which are used for improving the detection accuracy of a rotating target detection model.

The specific technical scheme provided by the embodiment of the application is as follows:

in a first aspect, an embodiment of the present application provides an article detection method, including:

shooting a plurality of objects to be detected in the visual field area to obtain an original image;

inputting the original image into a preset rotating target detection model to obtain detection area positioning information corresponding to each article to be detected, wherein the rotating target detection model is obtained by training the rotating target detection model to be trained based on a plurality of sample images and detection area labeling information of each sample article in the sample images, and each sample article comprises at least one article with a part of outer contour located outside a shooting visual field area of the corresponding sample image;

if the detection area positioning information corresponding to each object to be detected is based on the detection area positioning information, determining that the object to be detected with partial outline outside the visual field area exists in the original image, performing pixel filling on the original image to obtain a target image, and dividing the image of each object to be detected from the target image.

According to the article detection method provided by the embodiment of the application, the rotation target detection model to be trained is trained, so that the rotation target detection model after training can smoothly detect the articles to be detected, the outer contours of which are positioned outside the shooting visual field area, contained in the original image, and therefore, the original image corresponding to the articles to be detected is subjected to pixel filling to obtain the target image, and therefore, the images of the articles to be detected can be smoothly separated from the target image in a traditional image segmentation mode, the probability of information loss is reduced, the detection accuracy of the model in actual use is improved, and preparation is made for subsequent safety analysis, identification and the like based on the images of the articles to be detected.

In one possible implementation manner, the detection area positioning information of any article to be detected includes a plurality of coordinate information corresponding to a predicted rotation detection frame of the any article to be detected;

determining that an object to be detected with partial outline outside the visual field area exists in the original image by the following method:

selecting an abscissa maximum value and an abscissa minimum value, and an ordinate maximum value and an ordinate minimum value from the detection area positioning information corresponding to each object to be detected;

if at least one reference value among the abscissa maximum value, the abscissa minimum value, the ordinate maximum value and the ordinate minimum value meets a preset condition, determining that an object to be detected with part of the outer contour outside the visual field area exists in the original image, wherein the preset condition comprises:

the minimum value of the abscissa is smaller than a first preset value;

the maximum value of the abscissa is larger than the width of the original image;

the minimum value of the ordinate is smaller than a second preset value;

the ordinate maximum is greater than the height of the original image.

According to the method, by setting the preset conditions, whether the to-be-detected objects with partial outer contours outside the visual field area exist in the to-be-detected objects can be determined based on the detection area positioning information of the to-be-detected objects output by the rotation target detection model, so that the next computer vision processing flow is determined: the method comprises the steps of directly calling a traditional image segmentation mode to segment images of all objects to be detected from an original image, or firstly filling pixels of the original image to obtain a target image, and then calling the traditional image segmentation mode to segment the images of all the objects to be detected from the target image.

In one possible implementation manner, before the pixel filling is performed on the original image, the method further includes:

determining a filling rule corresponding to the attribute of the reference value meeting the preset condition based on the corresponding relation between the attribute of the reference value and the filling rule;

determining a filling area and the number of filling pixels based on the reference value and the filling rule;

the pixel filling is carried out on the original image to obtain a target image, which comprises the following steps:

and filling the pixels with the number of the filling pixels in the filling region outside the original image by adopting a preset pixel value to obtain the target image.

After determining that the to-be-detected objects with partial outer contours outside the visual field area exist in each to-be-detected object, determining a specific filling area and the number of filling pixels, and playing a guiding role for subsequent pixel filling; and then filling by adopting preset pixel values so as to ensure that the image of the object to be detected, of which part of the outer contour is positioned outside the visual field area, can be smoothly segmented from the target image when the image segmentation is carried out subsequently.

In one possible implementation manner, after the obtaining the target image, before the separating the image of each object to be measured from the target image, the method further includes:

Adjusting the detection area positioning information of any one to-be-detected object in the original image based on the coordinate information of the preset angle of the target image aiming at any one to-be-detected object in each to-be-detected object to obtain target detection area positioning information of any one to-be-detected object in the target image;

the step of dividing the image of each object to be detected from the target image comprises the following steps:

and dividing the image of any one to-be-detected object from the target image according to the target detection area positioning information of the any one to-be-detected object in the target image aiming at any one to-be-detected object in the each to-be-detected object.

In one possible implementation, before training the rotation target detection model to be trained, the method further includes:

determining whether the number of coordinate information included in the detection area labeling information of each sample article is larger than a first number, wherein the first number is the number of coordinate information of a rotation detection frame included in the detection area positioning information of any sample article, and the number is output by the rotation target detection model to be trained;

screening at least two candidate coordinate information from the coordinate information included in the detection area labeling information of the target sample article if the number of the coordinate information included in the detection area labeling information of the target sample article is larger than the first number, wherein the candidate coordinate information is coordinate information corresponding to an intersection point of an image edge line of a target sample image including the target sample article and a rotation detection frame of the target sample article;

And determining target coordinate information corresponding to the rotation detection frame of the target sample object outside the shooting visual field area of the target sample image based on the at least two pieces of candidate coordinate information, and replacing the at least two pieces of candidate coordinate information with the target coordinate information.

According to the method, the target sample articles are marked in the marking mode with more than the first quantity, the areas of the articles can be accurately framed to the maximum extent, before the rotating target detection model to be trained is trained, the quantity of coordinate information included in the marking information of the detection areas of the sample articles is unified, the rotating target detection model to be trained can learn the characteristics of the target sample articles, and therefore the trained rotating target detection model can smoothly detect articles to be detected, the outer contours of which are located outside the visual field area of the original image, in the model using stage, and therefore the information loss probability is reduced, and the detection accuracy of the model in actual use is improved.

In a possible implementation manner, the determining, based on the at least two candidate coordinate information, the target coordinate information corresponding to the rotation detection frame in which the target sample object is located outside the shooting field of view of the target sample image includes:

Selecting reference coordinate information from the coordinate information based on any one of the at least two pieces of alternative coordinate information, wherein the reference coordinate information is adjacent to the number of the point corresponding to the any one piece of alternative coordinate information and does not comprise other pieces of alternative coordinate information in the at least two pieces of alternative coordinate information;

and determining target coordinate information corresponding to the rotation detection frame of the target sample object outside the shooting visual field area based on the at least two pieces of alternative coordinate information and the reference coordinate information corresponding to the at least two pieces of alternative coordinate information.

In one possible implementation manner, the screening at least two candidate coordinate information from the coordinate information included in the detection area labeling information of the target sample object includes:

and carrying out multi-round screening on the coordinate information based on a preset extension number set until all numbers in the extension number set are selected to obtain the at least two candidate coordinate information, wherein one round of screening process comprises the following steps:

the number corresponding to the preset ordering information is used as the first number selected by the round in the numbers of the points corresponding to the second number of the coordinate information selected by the previous round, wherein the numbers of the points corresponding to the second number of the coordinate information are continuous;

Determining the number of the second number selected by the current round based on the extended number set and the first number, and selecting the second number coordinate information corresponding to the current round screening from the coordinate information based on the number of the second number selected by the current round;

and if the included angle value of the target included angle in each included angle formed by the points is determined to be larger than the included angle threshold value based on the points corresponding to the second number of coordinate information, and the coordinate information of the vertex of the target included angle is not the determined alternative coordinate information, determining the coordinate information of the vertex of the target included angle as the alternative coordinate information.

In a second aspect, an embodiment of the present application provides an article detection apparatus, including:

the shooting module is used for shooting a plurality of objects to be detected in the visual field area to obtain an original image;

the target detection module is used for inputting the original image into a preset rotary target detection model to obtain detection area positioning information corresponding to each article to be detected, wherein the rotary target detection model is obtained by training the rotary target detection model to be trained based on a plurality of sample images and detection area marking information of each sample article in the sample images, and each sample article comprises at least one article with a part of outer contour located outside a shooting visual field area of the corresponding sample image;

And the pixel filling module is used for carrying out pixel filling on the original image to obtain a target image if the to-be-detected objects with partial outer contours outside the visual field area exist in the original image based on the detection area positioning information corresponding to each to-be-detected object, and dividing the image of each to-be-detected object from the target image.

the pixel filling module is specifically configured to determine that an object to be detected whose partial outline is located outside the field of view area exists in the original image by:

The minimum value of the abscissa is smaller than a first preset value;

the minimum value of the ordinate is smaller than a second preset value;

the ordinate maximum is greater than the height of the original image.

In one possible implementation, before the pixel filling of the original image, the pixel filling module is further configured to:

the pixel filling module is specifically configured to:

In one possible implementation manner, after the obtaining the target image, before the dividing the image of each object to be measured from the target image, the pixel filling module is further configured to:

The pixel filling module is specifically configured to:

In one possible implementation, before training the rotational target detection model to be trained, the target detection module is further configured to:

In one possible implementation manner, the target detection module is specifically configured to:

In a third aspect, an embodiment of the present application provides an electronic device, including:

a memory for storing a computer program or instructions;

a processor for executing a computer program or instructions in the memory such that the method as in any of the first aspects above is performed.

In a fourth aspect, embodiments of the present application provide a computer readable storage medium, which when executed by a processor, causes the processor to perform the method of any of the first aspects described above.

In addition, the technical effects caused by any implementation manner of the second aspect to the fourth aspect may refer to the technical effects caused by different implementation manners of the first aspect, which are not described herein.

Drawings

FIG. 1 is a schematic diagram of an application scenario in an embodiment of the present application;

FIG. 2 is a schematic flow chart of an article detection method according to an embodiment of the application;

FIG. 3 is a schematic flow chart of a method for preprocessing detection region labeling information in an embodiment of the application;

FIG. 4 is a schematic diagram of a conventional labeling effect of a sample object according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an original image according to an embodiment of the present application;

FIG. 6 is a schematic diagram of an article marking effect according to an embodiment of the present application;

FIG. 7 is a flowchart of a method for determining alternative coordinate information according to an embodiment of the present application;

FIG. 8 is a schematic diagram illustrating a process for determining object coordinate information according to an embodiment of the present application;

FIG. 9 is a flowchart of a method for determining target coordinate information according to an embodiment of the present application;

FIG. 10 is a schematic flow chart of determining whether an original image includes an object to be tested with a part of its outline outside a field of view according to an embodiment of the present application;

FIG. 11 is a flowchart of a method for determining a filling area and a filling pixel number according to an embodiment of the present application;

FIG. 12 is a schematic diagram of an image coordinate system corresponding to an image according to an embodiment of the present application;

FIG. 13 is a schematic diagram of a process of segmenting an image of an object 8 from a target image according to an embodiment of the present application;

FIG. 14 is a schematic diagram of a logic architecture of an article detection apparatus according to an embodiment of the present application;

fig. 15 is a schematic diagram of an entity architecture of an electronic device according to an embodiment of the application.

Detailed Description

The following description of the embodiments of the present application will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments, but not all embodiments of the present application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to be within the scope of the application.

It should be noted that the terms "first," "second," "third," and the like in the description and the claims of the present application and in the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be capable of operation in sequences other than those illustrated or otherwise described.

The following briefly describes the design concept of the embodiment of the present application.

The application relates to the technical field of computer vision, in particular to an article detection method, an article detection device, an article detection equipment and a storage medium.

In an actual scene, for special occasions, such as customs security inspection, station-entering security inspection and the like, security inspection is often required to be carried out on personal belongings of personnel. Rotational object detection models are often used in the above-described scenarios where security checks are required. Based on the rotation target detection model, target detection is performed on images containing a plurality of articles, so that images corresponding to the articles are obtained, and analysis, recognition and the like of the articles are completed.

In the related art, the current rotating object detection model can only detect the object to be detected which is completely displayed in the image. However, in some actual situations, some articles are often not fully placed in the preset area, so that the collected image contains articles that are not fully displayed, and thus, when the current rotation target detection model is adopted to detect articles, the model generally does not detect targets of the articles that are not fully displayed in the image, thereby causing information loss, and further causing a problem of low detection accuracy in the actual use model.

In view of this, in order to solve the problem that the detection accuracy of the model in actual use is low, the embodiment of the application provides an article detection method, through a plurality of sample images including at least one article with a part of outline located outside the shooting field area of the corresponding sample image, and detection area labeling information of each sample article in the sample image, a rotation target detection model to be trained is trained, and a rotation target detection model is obtained, and in specific implementation, the rotation target detection model is utilized to smoothly detect an article to be detected with a part of outline located outside the shooting field area of the image, and then pixel filling is performed on an original image corresponding to the article to be detected, so as to obtain a target image, thus, the image of each article to be detected can be successfully segmented from the target image by adopting a traditional image segmentation mode, the probability of information loss is reduced, the detection accuracy in actual use of the model is improved, and preparation is made for subsequent safety analysis, recognition and the like based on the image of each article to be detected.

An application scenario of an optional method for detecting an article according to an embodiment of the present application is described below with reference to the accompanying drawings.

As shown in fig. 1, the inspection apparatus 10 is included in the application scenario. The inspection apparatus 10 includes a table top for placing an object to be inspected and an image acquisition device; the image acquisition device is mounted on top of the table top (the position shown in fig. 1 is for example only, and the application is not limited in detail) for acquiring images in the field of view on the table top.

In an exemplary customs security inspection scene, personnel place personal objects (the object to be inspected in the embodiment of the application) on a table surface of inspection equipment 10, the inspection equipment 10 executes an object detection method in the embodiment of the application, that is, a plurality of objects to be inspected in a visual field area are shot to obtain an original image, the original image is input into a preset rotating target detection model to obtain detection area positioning information corresponding to each object to be inspected, wherein the rotating target detection model is obtained by training the rotating target detection model to be trained based on a plurality of sample images and detection area labeling information of each sample object in the sample images, and each sample object comprises at least one object with a part of outline outside the shooting visual field area of the corresponding sample image; if the detection area positioning information corresponding to each object to be detected is used for determining that the object to be detected with partial outline outside the visual field area exists in the original image, the original image is subjected to pixel filling to obtain a target image, and therefore the image of each object to be detected is segmented from the target image by adopting a traditional image segmentation mode, so that the subsequent security analysis, recognition and the like of each object can be conveniently carried out.

In some possible embodiments, the above-mentioned inspection device may also be any type of device with a facial recognition function, for example, may be an intelligent terminal, an intelligent mobile terminal, a tablet computer, a notebook computer, an intelligent palm device, a personal computer (Personal Computer, PC), a computer, a monitoring device, an intelligent screen, various wearable devices, a personal digital assistant (Personal Digital Assistant, PDA), and so on.

In some preferred embodiments, the mesa should ensure that the portion falling within the field of view should have a solid background, such as black or white, to avoid the photographic background from affecting subsequent item detection.

In some possible embodiments, the image capturing device may be a camera, a video camera, or other electronic devices with a camera or a shooting function, such as a tablet computer or a smart phone.

Of course, the method provided by the embodiment of the application is not limited to the application scenario shown in fig. 1, and can be used in other possible application scenarios, such as the application scenario further includes a server connected with the inspection device 10 through a limited or wireless connection, the inspection device 10 sends the collected facial image to the server, and the server executes an article detection method in the embodiment of the application to detect the collected original image to obtain the image of each article to be detected in the original image; as another example, any application scenario in which the rotating target detection model is used to detect an article may be understood, and the application scenario is not specifically limited in the embodiment of the present application.

Having described an optional application scenario of embodiments of the present application, a further detailed description of preferred embodiments of the present application will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present application only, and are not intended to limit the present application, and embodiments of the present application and features of the embodiments may be combined with each other without conflict.

Referring to fig. 2, an embodiment of the present application provides an article detection method, which includes the following specific steps:

step 200: and shooting a plurality of objects to be detected in the visual field area to obtain an original image.

In the embodiment of the application, as shown in fig. 1, the plurality of to-be-measured objects are randomly placed on the table surface, the number of to-be-measured objects placed each time is not particularly limited, and can reach more than ten to even tens of to-be-measured objects, each to-be-measured object can not exceed the shot visual field area, and part of to-be-measured objects can also exist to exceed the shot visual field area. Meanwhile, the object to be measured can be placed on the table top at any angle, and the application is not particularly limited.

In specific implementation, when step 200 is executed, the inspection device shown in fig. 1 may control the image acquisition device to photograph the table top according to a preset period, or may respond to a photographing instruction, and may trigger the inspection device to control the image acquisition device to photograph the table top when a preset photographing condition is met, so as to obtain an original image, a plurality of objects to be measured placed on the table top are recorded in the original image, and a photographing range of a lens of the image acquisition device is a field of view area in step 200.

Step 210: inputting the original image into a preset rotating target detection model to obtain detection area positioning information corresponding to each article to be detected, wherein the rotating target detection model is obtained by training the rotating target detection model to be trained based on a plurality of sample images and detection area marking information of each sample article in the sample images, and each sample article comprises at least one article with a part of outer contour located outside a shooting visual field area of the corresponding sample image.

In a specific implementation, when executing step 210, the original image is input into a preset rotation target detection model, and based on the rotation target detection model, each object to be detected in the original image is detected to obtain detection area positioning information corresponding to each object to be detected, where the rotation target detection model is obtained by training a rotation target detection model to be trained based on a plurality of sample images and detection area labeling information of each sample object in each sample image.

In the embodiment of the application, the rotation target detection model to be trained can be constructed based on any algorithm such as YOLOv5, YOLOv7, YOLOv8 and the like, and can also be other types of detection models. In the embodiment of the application, the rotational target detection model to be trained is preferably constructed by YOLOv 5-OBB.

Specifically, before training the above-mentioned rotating target detection model to be trained, referring to fig. 3, the following steps are further executed:

step 300: determining whether the number of the coordinate information included in the detection area labeling information of each sample article is larger than a first number, wherein the first number is the number of the coordinate information of the rotation detection frame included in the detection area positioning information of any sample article, and the number is output by the rotation target detection model to be trained.

In the related art, the rotation target detection algorithm is an extension of the target detection algorithm. Conventional object detection algorithms typically employ rectangular label boxes to represent an object, while rotating object detection algorithms employ rotating rectangular boxes to represent an object. Because the rotating target detection algorithm uses a rotating rectangular box (in the embodiment of the present application, the rotating rectangular box is denoted as a rotating detection box) to represent a target, the rotating target detection algorithm usually uses a four-point labeling mode when data is labeled.

As shown in fig. 4, 1 object is included in the image a, and is denoted as object 1. When a conventional rotation detection frame is used to represent an object, the coordinate information of four vertexes of the rotation detection frame that frames the object 1 is used to represent the object 1, and the coordinate information is denoted as [ x ], and P1', P2', P3', and P4', respectively, and the corresponding coordinate information can be denoted as [ x ] ₁ ,y ₁ ,x ₂ ,y ₂ ,x ₃ ,y ₃ ,x ₄ ,y ₄ ]Wherein (x) _i ,y _i ) I=1, 2,3,4 denote coordinate information of four points in the clockwise direction from the upper left corner, respectively.

However, in a practical scenario, an image as shown in fig. 5 (i.e., an original image shown in fig. 1) often appears, where the original image includes 8 items to be measured, where the items to be measured 8 are not completely displayed in the original image, and the upper left corner of the items to be measured 8 is located outside the field of view of the image acquisition device, and is an item with a partial outline located outside the field of view of the image. If the object 8 to be measured is framed by adopting a conventional four-point marking mode, the object cannot be completely covered.

In order to enable the trained rotating target detection model to successfully identify the object 8 to be detected, in the embodiment of the application, the object is marked in a five-point marking mode, so that the area of the object 8 to be detected can be accurately framed. As shown in fig. 6, the object 8 to be measured is labeled by a five-point labeling method, and P1, P2, P3, P4, and P5 are sequentially obtained.

In the embodiment of the present application, the first number in step 300 may be a value of 4, so that the first number may be consistent with the labeling manner used for training the rotation target detection model in the related art. However, it should be noted that, in order to improve the accuracy of model training, the first number may take other values, and other labeling modes, such as a labeling mode with more than five points, may be adopted for the article 8 to be tested, which is not particularly limited in the present application.

In the following embodiments, only the first number value is 4, and the article to be tested 8 is represented by a five-point labeling manner, which is taken as an example, to introduce a design idea of the article detection method provided by the embodiment of the present application.

In the embodiment of the application, before training a trained rotating target detection model, a large number of sample images are required to be collected, wherein each sample image comprises at least one object with a partial outline outside a shooting visual field area of the corresponding sample image. Then, marking each sample image in each sample image by adopting the four-point marking mode or the five-point marking mode to obtain marking information of detection areas of each sample object in each sample image; and performing model training on the rotating target detection model to be trained based on the collected massive sample images and the detection area labeling information of each sample object in each sample image.

In a specific implementation, before the model training is performed on the rotating target detection model to be trained, the detection area labeling information of each sample object is further preprocessed by executing steps 300 to 330.

Specifically, when step 300 is executed, determining whether the number of coordinate information included in the detection region labeling information of each sample article is greater than a first number, where the first number is the number of coordinate information of a rotation detection frame included in the detection region positioning information of any sample article output by the rotation target detection model to be trained, that is, the number of coordinate information included in the detection region positioning information output by the rotation target detection model in specific implementation; if the number of coordinate information included in the detection region labeling information of the target sample article is greater than the first number, step 310 is executed, otherwise, that is, if the number of coordinate information included in the detection region labeling information of the sample article is not greater than the first number, no processing is executed.

Step 310: and screening at least two pieces of alternative coordinate information from the coordinate information included in the detection area labeling information of the target sample article if the number of the coordinate information included in the detection area labeling information of the target sample article is larger than the first number, wherein the alternative coordinate information is coordinate information corresponding to an intersection point of an image edge line of a target sample image containing the target sample article and a rotation detection frame of the target sample article.

In the embodiment of the present application, when step 310 is executed, if the number of coordinate information included in the detection region labeling information of the target sample object is greater than the first number, it is indicated that the target sample object is an object whose partial outline is located outside the shooting field of view region of the target sample image including the target sample object, and referring to fig. 7, when step 310 is executed, the following steps are specifically executed:

step 3101: based on a preset extension number set, carrying out multi-round screening on each coordinate information until all numbers in the extension number set are selected, and obtaining the at least two candidate coordinate information, wherein one round of screening process is as follows:

step 31011: and presetting a number corresponding to the ordering information in the numbers of points corresponding to the second number of coordinate information selected in the previous round as the first number selected in the previous round, wherein the numbers of the points corresponding to the second number of coordinate information are continuous.

In the embodiment of the present application, when step 3101 is executed, an extension number set is first constructed; and then, carrying out multi-round screening on each coordinate information based on the extended number set until all numbers in the extended number set are selected to obtain at least two pieces of alternative coordinate information, wherein in the one-round screening process, steps 31011 to 31014 are sequentially executed.

In specific implementation, when step 31011 is executed, the number of the point corresponding to the second number of coordinates information selected in the previous round is determined, and then the number of the second name represented by the sorting information in the determined number of the point is used as the first number selected in the previous round, where the number of the point corresponding to the second number of coordinates information is continuous. In the embodiment of the present application, the second number may take a value of 3.

For example, referring still to fig. 6, assume that the target sample article (i.e., the article 8 to be measured shown in fig. 6) is marked by a five-point marking method, and the obtained points are P1, P2, P3, P4, and P5 in sequence.

Then, an extension number set may be constructed using the subscripts of the coordinate information of the above points, denoted as [1,2,3,4,5,1,2].

It is further assumed that the number of the second number (e.g., 3) selected in the previous round is 3,4, and 5 in turn, that is, the points corresponding to the second number (e.g., 3) of coordinate information corresponding to the previous round of screening are P3, P4, and P5 in turn.

Then, in executing step 31011, the ranking information of 3,4,5 is characterized as the first number of the second name 4 chosen for this round.

Step 31012: and determining the number of the second number selected in the round based on the extended number set and the first number.

For example, still referring to fig. 6, it is still assumed that the target sample article (i.e. the article 8 to be measured shown in fig. 6) is marked by a five-point marking method, and the obtained points are sequentially P1, P2, P3, P4, and P5, and an extended number set is constructed by using the subscripts of the coordinate information of the points, and is marked as [1,2,3,4,5,1,2]; the number of the second number (3) selected in the previous round is 3,4,5 in turn, and the first number obtained by executing step 31011 is 4.

Then, in executing step 31012, it is determined that the 3 numbers selected in this round are 4,5, and 1 in order based on the extension number set [1,2,3,4,5,1,2], and the first number 4.

Step 31013: and selecting the coordinate information of the second number corresponding to the current round screening from the coordinate information based on the number of the second number selected by the current round.

For example, still referring to fig. 6, it is still assumed that the target sample article (i.e. the article 8 to be measured shown in fig. 6) is marked by a five-point marking method, and the obtained points are sequentially P1, P2, P3, P4, and P5, and an extended number set is constructed by using the subscripts of the coordinate information of the points, and is marked as [1,2,3,4,5,1,2]; the number of the second number (3) of the previous round of selection is 3,4,5 in order, and the number of the 3 present round of selection determined by executing step 31012 is 4,5,1 in order.

Then, in the execution stepAt step 31013, selecting a second number of coordinate information corresponding to the current round of screening from the coordinate information based on the numbers 4, 5, and 1, and sequentially determining the second number as [ x ] ₄ ,y ₄ ](P4)、[x ₅ ,y ₅ ](P5)、[x ₁ ,y ₁ ](P1)。

Step 31014: if the included angle value of the target included angle in each included angle formed by the points is determined to be larger than the included angle threshold value based on the points corresponding to the second number of coordinate information, and the coordinate information of the vertexes of the target included angle is not the determined alternative coordinate information, the coordinate information of the vertexes of the target included angle is determined to be the alternative coordinate information.

In some possible embodiments, after performing step 31013 to obtain the second number of coordinate information corresponding to the present round of screening, end-to-end connection may be performed according to the ranking information of the numbers of the points to obtain the target graph, and determine the included angle value of each included angle in the target graph, and determine whether the included angle value of each included angle is greater than the included angle threshold; if so, marking the included angle with the included angle value larger than the included angle threshold as a target included angle, and determining the coordinate information of the vertex of the target included angle as alternative coordinate information when the coordinate information of the vertex of the target included angle is not determined alternative coordinate information, such as recording the number of the point corresponding to the alternative coordinate information; when the coordinate information of the vertex of the target included angle is determined to be the determined candidate coordinate information, no operation is executed, or the number of the point corresponding to the determined candidate coordinate information is recorded secondarily, and the like.

In some preferred embodiments, for a target sample article marked by adopting a five-point marking mode, after selecting a second number (e.g. 3) of coordinate information corresponding to the present round of screening, two vectors can be constructed by taking a middle point in each point corresponding to the 3 coordinate information as a core, then, an included angle value of the two vectors can be calculated by adopting a cosine included angle mode, the calculated included angle value is compared with an included angle threshold, if the included angle value is greater than the included angle threshold, the middle point is indicated to be a rotation detection frame of the target sample article, and if the middle point is an intersection point of an image edge line of a target sample image containing the target sample article, the coordinate information corresponding to the middle point is determined to be alternative coordinate information; if the included angle value is not greater than the included angle threshold, indicating that the intermediate point is a point located within the target sample image containing the target sample object, no processing is performed.

Because the rotation detection frame in the embodiment of the application adopts a rotation rectangular frame, if a certain point is an intermediate point, the calculated angle value of the two vectors is greater than 90 degrees, and the point is usually the point located on the image edge line of the target sample image containing the target sample object. In order to avoid calculation errors, in the embodiment of the present application, the value of the included angle threshold may be 98 °, and it should be noted that the included angle threshold may also be specifically set based on actual situations, and the present application is not limited specifically.

For example, referring to fig. 8, the article 8 to be measured (i.e., the target sample article in this example) shown in fig. 5 and 6 is taken as an example.

Still assume that the target sample object is marked by adopting a five-point marking mode, the obtained points are sequentially P1, P2, P3, P4 and P5, and the coordinate information of a second number (such as 3) corresponding to the selection of the current round of screening is sequentially [ x ] ₄ ,y ₄ ](P4)、[x ₅ ,y ₅ ](P5)、[x ₁ ,y ₁ ](P1)。

In some preferred embodiments, in performing step 31014, two vectors, denoted v, are constructed with the intermediate point P5 of the 3 points as the core ₁ ,v ₂ The two vectors can be expressed by the following formula:

v ₁ ＝P ₅ -P ₄ ＝(x ₅ -x ₄ ,y ₅ -y ₄ )

v ₂ ＝P ₁ -P ₅ ＝(x ₁ -x ₅ ,y ₁ -y ₅ )

then, the angle value θ of the two vectors is calculated using the cosine angle formula:

wherein arccos () means to obtain the radian value corresponding to the cosine function, and deviee () means to convert the radian value into the angle value, i.e., the angle value θ.

Again, the calculated θ is compared to an included angle threshold.

Assume that the included angle threshold takes the value 98 °, θ=120°.

Then, still referring to fig. 8, since θ is greater than 98 °, indicating that P5 is the intersection of the rotation detection frame of the target sample article and the image edge line of the target sample image containing the target sample article, the coordinate information corresponding to P5 is determined as the candidate coordinate information.

Then, steps 31011 to 31014 are performed to determine whether or not alternative coordinate information exists in the 3 coordinate information corresponding to the next round of screening.

Still referring to fig. 8, the 3 coordinate information corresponding to the next round of screening is [ x ] in turn ₅ ,y ₅ ](P5)、[x ₁ ,y ₁ ](P1)、[x ₂ ,y ₂ ]And (P2) executing the process, taking P1 as a core, constructing new two vectors, and calculating the included angle value of the two vectors by adopting a cosine included angle formula, so as to obtain P1 which is also the intersection point of the rotation detection frame of the target sample article and the image edge line of the target sample image containing the target sample article, thereby determining the coordinate information corresponding to P1 as the alternative coordinate information.

In this way, the rotation detection frames of the target sample object can be selected from P1, P2, P3, P4, and P5, and the coordinate information corresponding to the intersection of the image edge lines of the target sample image including the target sample object, that is, the coordinate information corresponding to P5 and P1 can be obtained. In the process of flow processing, after the candidate coordinate information is screened out, only the numbers of the P5 and the P1 (namely, the subscripts of the coordinate information) are required to be recorded.

In some possible embodiments, the rotation detection frame corresponding to the rotation target detection model to be trained may not be a rectangular frame, so that the detection area positioning information output by the model for any sample article may also include coordinate information of four points or coordinate information of multiple points, and accordingly, by adopting the above multi-round screening provided by the embodiment of the present application, at least two candidate coordinate information may be screened out from each coordinate information of the target sample article, and then, by executing the subsequent step 320, the target coordinate information corresponding to the rotation detection frame of the target sample article located outside the shooting field area of the target sample image is determined.

Step 320: and determining target coordinate information corresponding to the rotation detection frame of the target sample object outside the shooting visual field area of the target sample image based on the at least two pieces of candidate coordinate information.

In specific implementation, referring to fig. 9, in executing step 320, the following steps 3201 to 3202 are specifically executed:

step 3201: for any one of the at least two candidate coordinate information, selecting reference coordinate information from the coordinate information based on the candidate coordinate information, wherein the reference coordinate information is adjacent to the number of the point corresponding to the candidate coordinate information and does not include other candidate coordinate information in the at least two candidate coordinate information.

For example, referring still to fig. 8, the article 8 to be measured (i.e., the target sample article in this example) shown in fig. 5 and 6 is taken as an example.

Still assume that the target sample object is marked by adopting a five-point marking mode, and the obtained points are P1, P2, P3, P4 and P5 in sequence.

In the embodiment of the present application, the points corresponding to the candidate coordinate information may be obtained by executing step 310 as P1 and P5, and when executing step 3201, the number of the point of the reference coordinate information corresponding to each candidate coordinate information may be obtained by the following formula:

Wherein, the point corresponding to the two alternative coordinate information is compiledThe numbers are ordered from small to large and marked as [ id ] ₁ ,id ₂ ]，id _1n And id ₂ Respectively represent id ₁ And id ₂ Subscript (i.e., number) of the coordinate information of adjacent points; mod () represents a modulo operation; in the embodiment of the application, when the value after taking the mode is zero, the default replacement is 5; when id ₁ =1 and id ₂ When the number of the samples is =5,

still referring to fig. 8, for P1 and P5, the obtained reference coordinate information corresponds to points numbered 2 and 4, respectively, that is, corresponding points P2 and P4, respectively.

Step 3202: and determining target coordinate information corresponding to the rotation detection frame of the target sample object outside the shooting visual field area based on the at least two pieces of alternative coordinate information and the reference coordinate information corresponding to the at least two pieces of alternative coordinate information.

In the embodiment of the present application, when step 3202 is executed, a straight line may be obtained specifically based on the candidate coordinate information and the reference coordinate information corresponding to the candidate coordinate information, and then at least two straight lines may be obtained based on the at least two candidate coordinate information and the reference coordinate information corresponding to the at least two candidate coordinate information, and then coordinate information corresponding to an intersection point of the at least two straight lines is used as target coordinate information corresponding to a rotation detection frame where the target sample object is located outside the shooting field of view.

For example, still referring to fig. 8, the object 8 to be measured (i.e., the target sample object in this example) shown in fig. 5 and 6 is still taken as an example.

In the embodiment of the present application, the points corresponding to the candidate coordinate information may be obtained by executing step 310 as P1 and P5, and P2 corresponding to P1 and P4 corresponding to P5 may be obtained by executing step 3201.

Then for two points p on the target sample image ₁ ＝(x ₁ ,y ₁ ) And p ₂ ＝(x ₂ ,y ₂ ) Wherein the points are represented in pixel coordinates, i.e. x ₁ ,x ₂ ,y ₁ ,y ₂ Are integers.

Let the linear equation passing through the two points P1, P2 be y=ax+b.

Then according to the two points p ₁ ＝(x ₁ ,y ₁ ) And p ₂ ＝(x ₂ ,y ₂ ) The coefficient of the linear equation passing through the two points P1 and P2 can be calculated by using a coefficient method to be determined.

Since the two points may lie on the same horizontal line (x in this case ₁ ＝x ₂ ) Then the conventional calculation method fails. Therefore, in the embodiment of the application, the classification calculation is performed according to different conditions, namely

When x is ₁ ＝x ₂ The coefficients are:

when x is ₁ ≠x ₂ The coefficients are:

b＝y ₁ -a×x ₁

in the embodiment of the present application, for P1 and P2, and P4 and P5, two straight lines may be obtained, such as straight line 1 and straight line 2 shown in fig. 8.

The coefficients corresponding to the linear equations of the two straight lines are assumed to be (a) ₁ ,b ₁ ) And (a) ₂ ,b ₂ ) And the coordinate information of the intersection of the two straight lines (i.e., the coordinate information of Pt shown in fig. 8) is (x) _inter ,y _inter )。

The coordinate information of the intersection point of the two straight lines satisfies the following equation set:the solution of the equation set can be obtained by calculation as +.>The coordinate information corresponding to the intersection point of the two straight lines, that is, the target coordinate information corresponding to the rotation detection frame of the target sample object located outside the shooting visual field area, can be obtained through the coefficients of the two straight line equations.

It should be noted that a is determined before the abscissa of the intersection point is calculated based on the formula corresponding to the solution ₁ ＝a ₂ If so, directly returning the parallel prompt information, namely the prompt information without the intersection point, and if not, calculating the coordinate information of the intersection point, namely the target coordinate information, based on the formula corresponding to the solution.

Step 330: the at least two candidate coordinate information are replaced with the target coordinate information.

In the embodiment of the present application, after the target coordinate information corresponding to the rotation detection frame of the target sample object located outside the shooting field of view is obtained by executing steps 300 to 320, step 330 is executed, and the at least two candidate coordinate information are replaced by the target coordinate information. Referring to fig. 8, the coordinate information corresponding to the point P1 and the point P5 is replaced with the coordinate information of the point Pt.

In this way, the number of coordinate information included in the detection area labeling information of each sample object in each sample image is unified, and then, based on the unified detection area labeling information of each sample object and a plurality of sample images, the rotation target detection model to be trained is trained, so that the rotation target detection model to be trained can learn the characteristics of the target sample object; after the model reaches the convergence condition, outputting a rotating target detection model corresponding to the last round of training, and taking the rotating target detection model corresponding to the last round of training as a rotating target detection model used in the implementation of a subsequent model, namely, when the step 210 is executed, the original image is input into the rotating target detection model to obtain the positioning information of the detection area corresponding to each object to be detected in the original image, and if some objects to be detected with the outer contour outside the visual field area exist in each object to be detected, the objects to be detected can be detected smoothly, so that the detection accuracy of the model in actual use is improved.

Step 220: if the detection area positioning information corresponding to each object to be detected is used for determining that the object to be detected with partial outline outside the visual field area exists in the original image, pixel filling is carried out on the original image, and a target image is obtained.

In the embodiment of the present application, as can be seen from the foregoing, the detection area positioning information output by the rotation target detection model for any object to be detected includes a plurality of coordinate information corresponding to the predicted rotation detection frame of the object to be detected, and the rotation target detection model used in the implementation can detect the object to be detected with a part of its outer contour located outside the field of view area, so that the coordinate information outside the field of view area of the original image may exist in the detection area positioning information output by the rotation target detection model, and thus, when the conventional image segmentation method is adopted, the object to be detected with a part of its outer contour located outside the field of view area cannot be successfully segmented from the original image, so before executing step 220, referring to fig. 10, it is required to determine whether the object to be detected with a part of its outer contour located outside the field of view area exists in the original image or not first:

step 1000: and selecting an abscissa maximum value and an abscissa minimum value, and an ordinate maximum value and an ordinate minimum value from the detection area positioning information corresponding to each object to be detected.

Step 1010: if at least one reference value among the abscissa maximum value, the abscissa minimum value, the ordinate maximum value and the ordinate minimum value meets a preset condition, determining that an object to be detected with part of the outer contour outside the visual field area exists in the original image; the preset conditions comprise: the minimum value of the abscissa is smaller than a first preset value; the maximum value of the abscissa is larger than the width of the original image; the ordinate minimum value is smaller than the second preset value. The ordinate maximum is greater than the height of the original image.

In some possible embodiments, if the maximum value of the abscissa, the minimum value of the abscissa, the maximum value of the ordinate and the minimum value of the ordinate do not meet the preset conditions, it is determined that no part of the to-be-detected objects with outer contours outside the field of view area exist in the original image, and the image corresponding to each to-be-detected object is segmented from the original image by adopting a traditional image segmentation method directly based on the positioning information of the detection area corresponding to each to-be-detected object.

In the embodiment of the present application, referring to fig. 11, after determining that an object to be tested whose outer contour is outside the field of view exists in the original image, before performing the pixel filling of the original image in step 220, the following steps are further performed:

step 1100: and determining the filling rule corresponding to the attribute of the reference value meeting the preset condition based on the corresponding relation between the attribute of the reference value and the filling rule.

Step 1110: based on the reference value and the filling rule, a filling area and a number of filling pixels are determined.

In the embodiment of the application, k articles to be detected are assumed to exist in the original image.

Then, the predicted rotation detection frames corresponding to the k articles to be detected are marked as [ Det ] ₁ ,Det ₂ ,Det _k ]Then(representing the predicted rotation detection frame of the ith article to be tested in k articles to be tested).

In the embodiment of the application, the minimum value, the maximum value, the minimum value and the maximum value of the abscissa of the predicted rotation detection frames of the k articles to be detected are expressed by the following formulas:

wherein j represents the number of coordinate information included in the detection area positioning information of any article to be detected output by the rotation target detection model.

In practice, it can be determined whether the original image needs to be filled (i.e. step 1010), and the number of filled regions and pixels (steps 1100-1110) by the following formula:

in this embodiment of the present application, the first preset value may be 0, and the second preset value may also be 0, where the preset conditions include: x is x _min ＜0；x _max ＞W；y _min ＜0；y _max W is the width of the original image, H is the height of the original image;

if the reference value x of the preset condition is satisfied _min If the attribute of the reference value is the minimum abscissa, the filling rule corresponding to the attribute of the reference value is: x is x _left ＝-x _min ；

If the reference value x of the preset condition is satisfied _max If the attribute of the reference value is the maximum abscissa, the filling rule corresponding to the attribute of the reference value is: x is x _right ＝x _max -W；

If the preset value is satisfiedReference value y of condition _min The attribute of (2) is the minimum on the ordinate, the filling rule corresponding to the attribute of the reference value is: y is _top ＝-y _min ；

If the reference value y meeting the preset condition _max The attribute of (2) is the ordinate is the largest, and the filling rule corresponding to the attribute of the reference value is: y is _down ＝y _max -H。

In the embodiment of the present application, after the filling area and the number of filling pixels are obtained, the pixel filling of the original image in step 220 is performed, and when the target image is obtained, specifically, a preset pixel value may be adopted, and the number of pixels of the filling pixels are filled in the filling area outside the original image, so as to obtain the target image, where the preset pixel value is a constant pixel value, for example, if the original image is a three-channel RGB image, the preset pixel value may be (0, 0).

If a plurality of filling areas are determined, filling is sequentially performed in the original image, respectively, to obtain the target image. Specifically, if the filled regions are determined to be a left filled region, a right filled region, an upper filled region, and a lower filled region, x is sequentially extended in the left filled region _left Column pixels, extending y in upper fill area _top Row pixels, extending x in right fill area _right Column pixels extending y in the lower fill area _down And row pixels.

In the embodiment of the application, since the original image is subjected to pixel expansion, the detection area positioning information of each object to be detected in the original image is obtained by detecting and outputting the rotation target detection model based on the original image, and the detection area positioning information of each object to be detected in the original image is not matched with the image area of the object to be detected in the target image, so that after the target image is obtained in the step 220, before the image of each object to be detected is segmented from the target image in the step 230, the detection area positioning information of each object to be detected in the original image is adjusted based on the coordinate information of the preset angle of the target image, and the target detection area positioning information of the object to be detected in the target image is obtained.

In the related art, when an image is processed, the upper left corner of the image is generally taken as the origin of coordinates, the origin of coordinates is downward in the positive y-axis direction, and the origin of coordinates is rightward in the positive x-axis direction, as shown in fig. 12. Correspondingly, in the embodiment of the application, the preset angle is set as the upper left angle along the basic knowledge. In specific implementation, when adjusting the positioning information of the detection area of any object to be detected, the positioning information of the target detection area of the corresponding object to be detected in the target image can be obtained through the following formula, namely, the coordinate information of the upper left corner is added to each coordinate information:

wherein i is [1, k ]],j∈[1,2,3,4]K represents the number of the coordinate information included in the detection area positioning information output by the model, wherein k represents k articles to be detected in the original image; x is x _left 、y _top For specific values of (a) can be found in the calculation results of the aforementioned formulas.

Step 230: and dividing the image of each object to be detected from the target image.

In the embodiment of the present application, after the target detection area positioning information of each object to be detected in the target image is obtained, step 230 is executed, and for any object to be detected in each object to be detected, the image of the object to be detected is segmented from the target image based on the target detection area positioning information of any object to be detected in the target image by adopting a conventional image segmentation method.

As shown in fig. 13, the original image shown in fig. 1 contains 8 to-be-measured objects, where the to-be-measured object 8 is an object to be measured whose partial outer contour is located outside the field of view of the original image. According to the method, the detection area positioning information of each object to be detected is detected through the rotary target detection model, the object to be detected, of which part of the outline is located outside the visual field area, is determined to exist in the original image based on the detection area positioning information of each object to be detected, then pixel filling is carried out on the original image, and as only the object to be detected 8 in the original image is the object to be detected of which part of the outline is located outside the visual field area of the original image, pixels need to be filled in the upper filling area of the original image, and a target image is obtained, as shown in fig. 13.

Still referring to fig. 13, the upper left corner area in the image of the object 8 to be measured, which is segmented from the target image, is a filled pixel; therefore, the image of each object to be detected in the original image can be obtained by adopting a traditional image segmentation mode, so that analysis, identification and the like of each object can be conveniently completed later.

Based on the same inventive concept, referring to fig. 14, an embodiment of the present application provides an article detecting device, including:

The shooting module 1410 is configured to shoot a plurality of objects to be detected in the field of view area to obtain an original image;

the target detection module 1420 is configured to input the original image into a preset rotating target detection model to obtain detection area positioning information corresponding to each article to be detected, where the rotating target detection model is obtained by training the rotating target detection model to be trained based on a plurality of sample images and detection area labeling information of each sample article in the sample images, and each sample article includes at least one article with a partial outline outside a shooting field area of the corresponding sample image;

and the pixel filling module 1430 is configured to, if it is determined that, based on the detection area positioning information corresponding to each object to be detected, there is an object to be detected whose partial outline is outside the field of view in the original image, perform pixel filling on the original image to obtain a target image, and divide the image of each object to be detected from the target image.

The pixel filling module 1430 is specifically configured to determine that an object to be detected whose partial outline is located outside the field of view area exists in the original image by:

the minimum value of the abscissa is smaller than a first preset value;

the minimum value of the ordinate is smaller than a second preset value;

the ordinate maximum is greater than the height of the original image.

In one possible implementation, before the pixel filling the original image, the pixel filling module 1430 is further configured to:

the pixel filling module 1430 is specifically configured to:

In one possible implementation manner, after the obtaining the target image and before the segmenting the image of each object to be tested from the target image, the pixel filling module 1430 is further configured to:

the pixel filling module 1430 is specifically configured to:

In one possible implementation, the object detection module 1420 is further configured to, prior to training the rotational object detection model to be trained:

In one possible implementation, the object detection module 1420 is specifically configured to:

Based on the same inventive concept, referring to fig. 15, an embodiment of the present application provides an electronic device, including:

a memory 151 for storing a computer program or instructions;

a processor 152 for executing computer programs or instructions in the memory 151 to cause a method as in any of the various embodiments described above to be performed.

The processor 152 may include one or more central processing units (Central Processing Unit, CPU), or digital processing units, etc. for executing computer programs or instructions in the memory 151, such that any of the methods as in the various embodiments described above are performed.

The specific connection medium between the memory 151 and the processor 152 is not limited in the embodiment of the present application. In fig. 15, the memory 151 and the processor 152 are connected by a bus 153, and the connection manner between other components is only schematically illustrated and not limited to the above. The bus 153 may be classified into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in fig. 15, but not only one bus or one type of bus.

Based on the same inventive concept, embodiments of the present application provide a computer-readable storage medium, which when executed by a processor, causes the processor to perform any one of the methods of the respective embodiments described above. Since the principle of solving the problem by the computer readable storage medium is similar to that of the article detection method, the implementation of the computer readable storage medium may refer to the implementation of the method, and the repetition is omitted.

It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.

Claims

1. An article detection method, comprising:

2. The method of claim 1, wherein the detection area positioning information of any one of the items to be detected includes a plurality of coordinate information corresponding to a predicted rotation detection frame of the any one of the items to be detected;

The minimum value of the abscissa is smaller than a first preset value;

the minimum value of the ordinate is smaller than a second preset value;

the ordinate maximum is greater than the height of the original image.

3. The method of claim 2, wherein prior to pixel filling the original image, further comprising:

4. The method of claim 3, wherein after the obtaining the target image, before the segmenting the image of each object to be tested from the target image, further comprises:

5. The method of any of claims 1-4, further comprising, prior to training the rotational object detection model to be trained:

6. The method of claim 5, wherein determining, based on the at least two candidate coordinate information, target coordinate information corresponding to the rotation detection frame for which the target sample object is located outside a captured field of view of the target sample image comprises:

7. The method according to claim 5, wherein the screening at least two candidate coordinate information from the coordinate information included in the detection region labeling information of the target sample article includes:

8. An article detection device, comprising:

9. An electronic device, comprising:

a memory for storing a computer program or instructions;

A processor for executing a computer program or instructions in the memory, such that the method according to any of claims 1-7 is performed.

10. A computer readable storage medium, characterized in that instructions in the storage medium, when executed by a processor, enable the processor to perform the method of any one of claims 1-7.