KR20210147443A - Method and apparatus for labeling multi-channel image training data based on crowd verification - Google Patents

Abstract

According to an embodiment of the present invention, it is possible to provide a method and device for labeling learning data capable of efficiently and quickly improving an image inference machine learning model. Disclosed is the method for labeling multi-channel image learning data based on crowd verification which is running on a computer system. The disclosed method comprises: a step of receiving multi-channel image data; a step of generating a dataset based on the same time reference for the multi-channel image data, and inferring posture of individual livestock in an image based on machine learning for each multi-channel image data of corresponding dataset; a step of comparing and analyzing an inference result to compare and determine whether the inference result in the same dataset matches; and a step of performing labeling on the multi-channel image data when the inference result in the same dataset matches.

Description

Crowd verification-based multi-channel image training data labeling method and apparatus {Method and apparatus for labeling multi-channel image training data based on crowd verification}

The present disclosure relates to a training data labeling method, and more particularly, to a crowd validation-based multi-channel image training data labeling method and apparatus.

Learning data is needed to learn artificial intelligence. To make a competent AI, high-quality learning data is needed, and in order to train the AI, it is necessary to partially process the data into a form that can be learned by a machine, rather than using the data as it is. In the field of image recognition, labeling for AI training, that is, a bounding box and label input, and a training dataset are required for object recognition within an image. The task to create was expensive and time consuming.

Recently, a technique to increase the accuracy of a model by repeatedly performing such a cycle of labeling, learning, and inference has been developed, which reduces the part that needs to be performed by humans by using an automated solution and platform and reduces the learning process It helps to prepare a high-accuracy model in a short time by automating it as much as possible. However, the training data labeling automation technique that has been disclosed so far still has a lot of work to be performed by humans. For example, inference is made by copying an existing image data set that has completed training, and a human needs to input an additional recognized bounding box and label into the image by comparing and confirming the inference result, the label result, and the original label result. In addition, the object to be additionally recognized still requires the existing labeling operation.

In other words, in order to train a new image, it is necessary to check all the results of manual labeling or inferred from the existing model as before, and the labeling of additional objects to be recognized still requires a lot of time and effort because manual labeling is required as before. It requires effort.

Therefore, there is a need for an effective and efficient method of generating an inference model with high accuracy based on image data.

According to one aspect of the present disclosure, there is provided a method for labeling data, which is performed on a computer system, based on crowd validation, for multi-channel image learning. The method includes receiving multi-channel image data; generating a dataset based on the same time reference for the multi-channel image data, and inferring the posture of individual livestock in the image based on machine learning for each multi-channel image data of the corresponding dataset; comparing and analyzing the inference results to compare and determine whether the inference results in the same dataset match; and performing labeling on the multi-channel image data when the inference results in the same dataset match.

According to an embodiment of the present disclosure, in the method, if the inference results in the same dataset do not match, k number of image data included in the same dataset (k>1, k is an odd number) verifier terminals The method may further include the step of transmitting to , receiving verification results of the image data included in the same dataset from the k verifier terminals, and determining whether the verification results match.

According to an embodiment of the present disclosure, the method may include performing labeling on the multi-channel image data when the verification results match.

According to an embodiment of the present disclosure, when the verification results do not match, the method transmits image data included in the same dataset to m (m>k, m is an odd number) number of verifier terminals, and The method may further include receiving verification results of image data included in the same dataset from m verifier terminals, and determining whether the verification results match.

According to an embodiment of the present disclosure, the method may further include retraining the machine learning model based on new training data labeled with respect to the multi-channel image data.

According to an embodiment of the present disclosure, the method may further include paying a reward to the verifier terminal.

According to another feature of the present disclosure, there is provided a computer-readable recording medium containing one or more computer-readable instructions executable by a computer, wherein the one or more computer-readable instructions, when executed by the computer, cause the computer to , a computer-readable recording medium for performing any one of the methods described above is provided.

According to another feature of the present disclosure, a crowd-verification-based multi-channel image learning data labeling apparatus is provided. The apparatus includes: a multi-channel image acquisition unit configured to receive a plurality of multi-channel image data; a multi-channel image inference unit configured to infer an object included in an image based on a machine learning model for each of the plurality of multi-channel image data; a multi-channel image inference result comparison and analysis unit configured to compare and analyze the inferred result; and a labeling file generator configured to generate a labeling file for each of the plurality of multi-channel image data.

According to an embodiment of the present disclosure, the device transmits the plurality of multi-channel image data to a plurality of verification terminals when the results of comparative analysis of the inferred results do not match, and transmits the plurality of multi-channel image data to the plurality of verification terminals. It may further include a crowd verification unit configured to receive the verification result and determine whether the verification result from the verifier is consistent.

According to an embodiment of the present disclosure, the apparatus may further include a learning unit for re-learning the machine learning model based on the new training data labeled with respect to the multi-channel image data.

According to an embodiment of the present disclosure, it is possible to provide a training data labeling method and apparatus capable of efficiently and quickly improving an image inference machine learning model.

According to an embodiment of the present disclosure, it is possible to provide a training data labeling method and apparatus capable of generating and verifying training data based on an automated platform through a plurality of crowd validators even when the training data is not completely prepared.

According to an embodiment of the present disclosure, it is possible to provide a learning data labeling method and apparatus capable of improving an image inference machine learning model based on an automated platform by using an inference result of a new image using a plurality of multi-channel images. have.

In improving the image inference machine learning model, the labeling task is one of the most time-consuming and expensive parts in reality, so increasing the efficiency of the labeling task based on an automated platform can very effectively improve the image inference machine learning model. have.

1 is a diagram schematically illustrating an overall configuration of a crowd-based multi-channel image labeling system according to an embodiment of the present disclosure.
2 is a functional block diagram illustrating an exemplary functional configuration of the image labeling server of FIG. 1 according to an embodiment of the present disclosure.
3 is an exemplary operation flowchart illustrating a flow in which an image labeling server processes image labeling according to an embodiment of the present disclosure.
4 is an exemplary operation flowchart illustrating a learning flow of an image labeling server according to an embodiment of the present disclosure.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Hereinafter, when it is determined that there is a risk of unnecessarily obscuring the subject matter of the present disclosure, detailed descriptions of already known functions and configurations will be omitted. In addition, it should be understood that the contents described below are only related to one embodiment of the present disclosure, and the present disclosure is not limited thereto.

The terms used herein are used only to describe specific embodiments and are not intended to limit the present disclosure. For example, an element expressed in a singular should be understood as a concept including a plurality of elements unless the context clearly means only the singular. In addition, in the specification of the present disclosure, terms such as 'comprise' or 'have' are only intended to designate that the features, numbers, steps, operations, components, parts, or combinations thereof described in the specification exist, and such The use of the term is not intended to exclude the possibility of the presence or addition of one or more other features or numbers, steps, operations, components, parts, or combinations thereof.

In the embodiments described herein, a 'block' or 'unit' means a functional part that performs at least one function or operation, and may be implemented as hardware or software, or a combination of hardware and software. In addition, the plurality of 'blocks' or 'units' may be integrated into at least one software module and implemented with at least one processor, except for 'blocks' or 'units' that need to be implemented with specific hardware. .

In addition, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. It should be noted that commonly used terms defined in the dictionary should be interpreted as having a meaning consistent with the meaning in the context of the related art, and should not be construed as unduly limited or expanded unless explicitly defined otherwise in the specification of the present disclosure. should know

Hereinafter, with reference to the accompanying drawings, an embodiment of the present disclosure will be described in detail.

1 is a diagram schematically illustrating an overall configuration of a crowd-based multi-channel image labeling system 100 according to an embodiment of the present disclosure. As shown, the crowd-based multi-channel image labeling system includes a plurality of cameras (120a, 120b, ..., 120n), a plurality of verifier terminals (130a, 130b, ..., 130n), an image labeling server ( 140), and the communication network 150 may be included.

According to an embodiment of the present disclosure, the cameras (120a, 120b, ..., 120n) are for photographing the appearance of livestock managed by the livestock farm, an image photographing camera, a video recording camera having a wired or wireless communication function , may be any electronic device such as a CCTV camera.

According to an embodiment of the present disclosure, the cameras 120a, 120b, ..., 120n are images of livestock themselves, gait images of livestock, sleep images, motion images, amount of feed ingested, shoulder posture image, excretion You can shoot video, vomit video, and the cleaning status of the livestock.

According to an embodiment of the present disclosure, the cameras 120a, 120b, ..., 120n store recorded image data in a network video recorder (NVR:110), and transmit the recorded image data through the communication network 150 . can As shown in this figure, the crowd-based multi-channel image labeling system 100 is illustrated as having four cameras 120a, 120b, 120c, and 120d, but the present disclosure is not limited thereto.

According to an embodiment of the present disclosure, the cameras 120a, 120b, ..., 120n may photograph the same space. To explain, as shown in FIG. 1 , a plurality of cameras, for example, four cameras at each corner, are installed in one barn, so that the entire barn can be photographed at various locations.

According to an embodiment of the present disclosure, each of the plurality of verifier terminals 130a, 130b, ..., 130n may be a communication-capable terminal possessed by each verifier performing crowd verification. According to an embodiment of the present disclosure, each of the plurality of verifier terminals 130a, 130b, ..., 130n is, for example, various types of electronic terminals such as a mobile phone, a smartphone, a tablet PC, a desktop, and a laptop of the verifier. can The verifier communicates with the server 140 through the communication network 150 using his/her verifier terminals 130a, 130b, ..., 130n to provide or obtain various types of information.

According to an embodiment of the present disclosure, the image labeling server 140 may receive image data from the cameras 120a, 120b, ..., 120n and infer an image based on the collected image data. According to an embodiment of the present disclosure, the image labeling server 140 may map the postures of individual livestock based on the collected image data. According to an embodiment of the present disclosure, the image labeling server 140 clusters the image data collected by each of the plurality of cameras 120a, 120b, ..., 120n into one set of data captured at the same time, The analysis results can be compared by inferring each image belonging to the same set.

Table 1 is a table exemplarily showing the result of inferring the image data collected by receiving the image data from the cameras 120a, 120b, ..., 120n from the image labeling server 140 .

number hour camera
number Analysis One 2020-03-24 14:04:32 One [{“label”:“Standing_Pig”, “bndbox”:{“xmin”:54, “ymin”:46, “xmax”:128, “ymax”:241}}, {“label”:“Sitting_Pig”, “bndbox”:{“xmin”:564, “ymin”:80, “xmax”:945, “ymax”:312}}] 2 2020-03-24 14:04:32 2 [{“label”:“Standing_Pig”, “bndbox”:{“xmin”:68, “ymin”:86, “xmax”:382, “ymax”:514}}, {“label”:“Lying_Pig”, “bndbox”:{“xmin”:165, “ymin”:169, “xmax”:255, “ymax”:324}}] 3 2020-03-24 14:04:32 3 [{“label”:“Standing_Pig”, “bndbox”:{“xmin”:272, “ymin”:154, “xmax”:365, “ymax”:297}}, {“label”:“Sitting_Pig”, “bndbox”:{“xmin”:457, “ymin”:134, “xmax”:631, “ymax”:246}}] 4 2020-03-24 14:04:32 4 [{“label”:“Standing_Pig”, “bndbox”:{“xmin”:1729, “ymin”:232, “xmax”:1822, “ymax”:361}}, {“label”:“Sitting_Pig”, “bndbox”:{“xmin”:1488, “ymin”:173, “xmax”:1596, “ymax”:291}}]

According to an embodiment of the present disclosure, the image labeling server 140 collects images from a plurality of cameras 120a, 120b, ..., 120n at a constant cycle, for example, every second, periodically targeting a new image acquired By analyzing the image, information on livestock included in the image can be inferred based on machine learning. In one embodiment, the image labeling server 140 analyzes the images taken by cameras 1 to 4 at the time of 2020-03-24 14:04:32 through a machine learning model to determine the posture of the livestock included in the image can be inferred. In one embodiment, when the image labeling server 140 receives images of 4 channels in a multi-channel time zone, that is, every 1 second, the image labeling server 140 processes a new set of 4 images per second. Inference can be performed on the target. In one embodiment, the image labeling server 140 for the image data of the same time period collected by each of the four cameras (120a, 120b, 120c, 120d), a standing pig in the first image collected by the first camera It is inferred that there is one pig, one pig sitting, inferred that there is one standing pig and one pig sitting in the second image collected from the second camera, and in the third image collected by the third camera, there is one pig sitting. It can be inferred that there is one standing pig and one sitting pig, and that there are one standing pig and one sitting pig in the fourth image collected by the fourth camera.

According to an embodiment of the present disclosure, the image labeling server 140 analyzes the image data collected by each of the plurality of cameras 120a, 120b, ..., 120n, and images belonging to the same set to obtain images of the same time. of inference results can be comparatively analyzed. For example, in the above example, by analyzing the inference results of the first to fourth images, it may be determined that the inference results match.

According to an embodiment of the present disclosure, the image labeling server 140 analyzes the inference result and determines that the result matches each image, for example, a corresponding posture image of a pig included in the first to fourth images. Labels of postures can be labeled. To explain, a labeling file called a standing pig may be generated for the image of a standing pig, and a labeling file called a pig who is sitting on the shoulder may be generated for the part of the image of the pig sitting on the shoulder. In an embodiment, the image labeling server 140 may receive an instruction to analyze a specific object (a specific posture of livestock) and perform analysis to increase inference accuracy of the corresponding object. For example, when focusing on the sitting pig (Sitting_Pig) to increase the inference accuracy, the image labeling server 140 analyzes whether the image inference result includes the sitting pig image, and analyzes the inference result including the sitting pig image can perform

According to an embodiment of the present disclosure, when the image labeling server 140 analyzes the inference result and determines that the results do not match, crowd-based image labeling may be performed for more accurate review. In one embodiment, the image labeling server 140 provides a plurality of image datasets, that is, four images, with inconsistent inference results to the inspector terminal, and receives the labeling results for the image dataset from the plurality of inspector terminals, It can be determined whether the analysis results of the inspector are consistent.

According to an embodiment of the present disclosure, the image labeling server 140 may adjust the number of inspectors receiving the labeling result. In an embodiment, the image labeling server 140 may provide a plurality of images with inconsistent results to three inspectors, and receive the labeling results to determine whether the analysis results match. In an embodiment, when the analysis results received from all inspectors match, the image labeling server 140 may label a corresponding posture label with respect to a posture image of a pig included in a plurality of provided images. In another embodiment, when the analysis results received from the inspectors are inconsistent, the image labeling server 140 increases the number of inspectors to provide a plurality of images with inconsistent results, that is, four images, and the labeling results from the plurality of inspectors can be received to determine whether the analysis results match. In one embodiment, the image labeling server 140 may increase the number of inspectors until the labeling result is the same, and when the number of inspectors exceeds a threshold, the image may not be used without further inspection. . In another embodiment, the image labeling server 140 may increase the number of inspectors until the labeling results are the same. For a posture image of a pig included in the image, a label for that posture can be labeled.

According to an embodiment of the present disclosure, the image labeling server 140 may pay a reward to an inspector who inputs a correct labeling result. In an embodiment, the image labeling server 140 may accumulate and deduct points for settlement based on a result verified by each verification person. In an embodiment, the image labeling server 140 may award points if the verifier correctly verifies, and may deduct points if the verifier is not correct. For example, the image labeling server 140 can earn 1 point when the verifier correctly verifies one image or one object, and can deduct more points, for example, 20 points, if the verifier is verified differently from a majority of opinions. can In one embodiment, the image labeling server 140 pays a reward to the inspector who enters the correct labeling result for inputting image labeling based on a crowd, and if the verification is not accurate, the points subtracted for the correct verification It may be advantageous to increase the quality of the verification result by inducing careful verification to the crowd validator if it is higher than the points awarded as a reward.

When the image labeling server 140 needs to subtract the points of a specific inspector, the image labeling server 140 may subtract the inspector's points after receiving the administrator's confirmation. In one embodiment, the image labeling server 140 determines that the verification is inaccurate, but if the administrator does not input a point deduction permission, the image labeling server 140 may not deduct the inspector's points. . For example, if the image is very difficult to read, the administrator may not deduct the inspector's points even if the verification is not accurate. In one embodiment, the image labeling server 140 may pay a point to the inspector according to the manager's instructions.

According to an embodiment of the present disclosure, the communication network 150 may include any wired or wireless communication network, for example, a TCP/IP communication network. According to an embodiment of the present disclosure, the communication network 150 may include, for example, a Wi-Fi network, a LAN network, a WAN network, an Internet network, and the like, but the present invention is not limited thereto. According to an embodiment of the present disclosure, the communication network 150 is, for example, Ethernet, GSM, EDGE (Enhanced Data GSM Environment), CDMA, TDMA, OFDM, Bluetooth, VoIP, Wi-MAX, Wibro and any other various wired or wireless wired or wireless It can be implemented using a communication protocol.

2 is a functional block diagram illustrating an exemplary functional configuration of the image labeling server of FIG. 1 according to an embodiment of the present disclosure.

As shown, the image labeling server 140 includes a multi-channel image acquisition unit 201, a multi-channel image inference unit 203, a multi-channel image inference result comparison analysis unit 205, a crowd verification unit 207, It may include a labeling file generating unit 209 , a learning unit 211 , a storage unit 213 , and a communication unit 215 .

According to an embodiment of the present disclosure, the multi-channel image acquisition unit 201 may acquire an image by receiving image data. In an embodiment, the multi-channel image acquisition unit 201 may acquire an image by receiving image data from the cameras 120a, 120b, ..., 120n. In an embodiment, the multi-channel image acquisition unit 201 may acquire multi-channel images by receiving multi-channel image data from a plurality of cameras 120a, 120b, ..., 120n that photograph the same area. Here, the multi-channel image data may refer to image data obtained by photographing the same area at different locations at the same time.

According to an embodiment of the present disclosure, the multi-channel image inference unit 203 may infer the posture of individual livestock from the image data collected based on the machine learning model. According to an embodiment of the present disclosure, the multi-channel image inference unit 203 sets the data captured at the same time with respect to the image data collected by each of the plurality of cameras 120a, 120b, ..., 120n as one set. We can cluster and infer the poses of individual animals for each image belonging to the same set.

Table 1 described above is a table exemplarily showing a result of inferring multi-channel image data by the multi-channel image inference unit 203 .

According to an embodiment of the present disclosure, the multi-channel image inference unit 203 collects images from a plurality of cameras 120a, 120b, ..., 120n at a constant cycle, for example, every second, and applies a new image acquired In this way, the information of livestock included in the image can be periodically inferred based on machine learning. For example, the multi-channel image inference unit 203 analyzes the images taken by cameras 1 to 4 at the time of 2020-03-24 14:04:32 to infer the posture and position information of the livestock included in the image. can In one embodiment, when the multi-channel image acquisition unit 201 receives images of 4 channels in the multi-channel time zone, that is, every 1 second, the multi-channel image inference unit 203 receives data of 4 images per second Inference can be performed on new images in the set. Here, the set may mean a set of multi-channel image data captured by each of the plurality of cameras 120a, 120b, ..., 120n in the same area at different locations at the same time. In one embodiment, the multi-channel image inference unit 203 is a result of inferring the image data captured by each of the four cameras 120a, 120b, 120c, and 120d at the same time and at a different location in the same area. 1 It is inferred that there are 1 standing pig and 1 sitting pig in the first image collected from the camera, and 1 standing pig and 1 sitting pig in the second image collected from the second camera. inferring, inferring that there are 1 standing pig and 1 pig sitting in the third image collected from the third camera, and 1 standing pig and 1 pig sitting in the fourth image collected from the fourth camera Each can be inferred that there is one.

According to an embodiment of the present disclosure, the multi-channel image inference result comparison analysis unit 205 may compare and analyze the inference result of images belonging to the same dataset. In an embodiment, the multi-channel image inference result comparison analysis unit 205 may compare and analyze each of the inference results to determine that the inference results of the multi-channel images match. In an embodiment, the multi-channel image inference result comparison analysis unit 205 may transmit the multi-channel image data having the same inference result to the labeling file generation unit 209 .

According to an embodiment of the present disclosure, the crowd verification unit 207 may receive an image dataset and perform verification on each of the multi-channel images included in the set. In one embodiment, the crowd verifier 207 receives an image dataset in which the inference result of the images included in the same dataset does not match, and compares the image to a plurality of verifier terminals 130a, 130b, ..., 130n ) to receive the inspection result. For example, in one embodiment in the above example, the crowd verifying unit 207 provides a plurality of images, that is, four images, with inconsistent results to the verifier terminals 130a, 130b, ..., 130n, and a plurality of The verification result may be received from the verifier terminals 130a, 130b, ..., 130n, and it may be determined whether the results received from the verifier terminals 130a, 130b, ..., 130n match. In an embodiment, the crowd verifying unit 207 may transmit multi-channel image data whose verification results match to the labeling file generating unit 209 .

According to an embodiment of the present disclosure, the crowd verifier 207 may adjust the number of verifier terminals 130a, 130b, ..., 130n. In an embodiment, if the verification results from the plurality of verifier terminals 130a, 130b, ..., 130n do not match each other, the crowd verifier 207 may increase the number of verifier terminals requesting verification. have. For example, in the first verification, a request is made to three verifier terminals, and when the verification results received from the three verifier terminals do not match, the second verification requests verification to five verifier terminals and receives the verification result. can do. In an embodiment, the crowd verifier 207 may increase the number of verifier terminals until the verification results match, and when the number of verifier terminals exceeds a threshold, the image is used without further verification. may not In another embodiment, the crowd verifier 207 may increase the number of verifier terminals until the verification results are the same, and when the number of verifier terminals exceeds a threshold, in a majority rule, that is, The plurality of verification results and the multi-channel image dataset may be transmitted to the labeling file generating unit 209 by considering a result that a plurality of verification results agree with each other as a correct verification result.

According to an embodiment of the present disclosure, the crowd verifier 207 may pay a reward to a verifier terminal that inputs a correct labeling result. In an embodiment, the crowd verifier 207 may accumulate and deduct points for settlement based on a result verified by each verifier terminal. In an embodiment, the crowd verifier 207 may pay points when the verifier terminal correctly verifies, and deduct points when the verifier is not correct. For example, the crowd verification unit 207 may accumulate 1 point if the verifier terminal correctly verifies one image or one object, and may deduct 20 points if the verification is different from a majority of opinions. In one embodiment, the crowd verification unit 207 pays a reward to the verifier terminal for inputting the correct labeling result is to input to perform the verification based on the crowd, and if the verification is not accurate, the point to be subtracted is correct It may be advantageous to increase the quality of the verification result by inducing careful verification to the crook verifier if it is higher than the points awarded according to verification.

When the crowd verifier 207 deducts the points of a specific verifier terminal, the crowd verifier 207 may subtract the points of the verifier terminal after receiving the manager's confirmation. In an embodiment, the crowd verifier 207 determines that the verification result of the specific verifier terminal is not accurate, but if the manager does not allow the point deduction, the crowd verifier 207 determines the point of the verifier terminal. may not be deducted. In an embodiment, the crowd verifier 207 may provide points to the verifier terminal according to the manager's instruction.

According to an embodiment of the present disclosure, the labeling file generation unit 209 may generate a labeling file based on the multi-channel image inference result comparative analysis result and the corresponding image data. In one embodiment, when labeling the posture of the livestock in the image, the label (class) is a standing pig (Standing_Pig), a lying pig (Lying_Pig), a stacked pig (Multiple_Pig), a sitting posture pig (Sitting_Pig), a vomiting pig (Vomiting_Pig), Diarrhea_Pig, and Cough_Pig.

In one embodiment, the labeling file generating unit 209 analyzes the inference result and determines that the result matches the corresponding posture for each image, for example, a posture image of a pig included in the first to fourth images. You can label it with a label. To explain, a labeling file called a standing pig may be generated for the image of a standing pig, and a labeling file called a pig who is sitting on the shoulder may be generated for the part of the image of the pig sitting on the shoulder. In an embodiment, the labeling file may include, but is not limited to, a path in which an image file is stored, a folder name in which an image is stored, an image file name, a size of an image, and a list of object information. In an embodiment, the object information list may mean a label name and a bound box (xmin, ymin, xmat ymax), but is not limited thereto.

According to an embodiment of the present disclosure, the learning unit 211 may perform a role of inferring an image by performing machine learning based on the generated model. In an embodiment, the learner 211 may update the machine learning model with new training data, that is, newly labeled image data. The learning unit 211 may include an artificial neural network, for example, a deep neural network (DNN) such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a deep belief network (DBN).

According to an embodiment of the present disclosure, the storage unit 213 may be any storage medium in which various programs executable on the server 140 and related data are stored. According to an embodiment of the present disclosure, the storage unit 213 may store sensor data, image data, and data related to the execution of the analysis module. According to an embodiment of the present disclosure, the storage unit 213 may be configured to include various types of volatile or nonvolatile memory such as DRAM, SRAM, DDR, RAM, ROM, magnetic disk, optical disk, flash memory, and the like.

According to an embodiment of the present disclosure, the communication unit 215 may support the image labeling server 140 to communicate with the outside through the communication network 150 . According to an embodiment of the present disclosure, the communication unit 215 may receive data from the communication network 150 according to a predetermined protocol, and transmit data from the image labeling server 140 to the outside through the communication network 150 . You can perform the necessary procedures.

3 is an exemplary operation flowchart illustrating a flow in which an image labeling server processes image labeling according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the image labeling server 140 may collect image learning data in advance and generate at least one classification model through a learning process such as data labeling, and to improve the performance of the generated model. For this purpose, additional labeling of the training data can be performed.

As shown, in step S302, the image labeling server 140 may receive a predetermined multi-channel image. Here, the multi-channel image data may refer to image data obtained by photographing the same area at different locations at the same time. Then, in step S304, the image labeling server 140 may infer the posture of individual livestock in the image based on machine learning based on the collected multi-channel image data. According to an embodiment of the present disclosure, the image labeling server 140 clusters data captured at the same time for multi-channel image data into one dataset, and determines the posture of individual livestock for each image belonging to the same data set. can be inferred. Here, the data set may mean a set of multi-channel image data captured by each of the plurality of cameras 120a, 120b, ..., 120n in the same area at different locations at the same time. For example, the image labeling server 140 infers the image data captured by the four cameras 120a , 120b , 120c , and 120d at the same time and at a different location from the first camera. Inferring that there are 1 standing pig and 1 pig sitting in the first image collected, and inferring that there are 1 standing pig and 1 pig sitting in the second image collected from the second camera, It is inferred that there are 1 standing pig and 1 ponytail pig in the third image collected from the 3rd camera, and 1 standing pig and 1 croup pig in the 4th image acquired from the 4th camera. It can be inferred that each

Thereafter, in step S306 , the image labeling server 140 may compare and analyze the inference results of images belonging to the same dataset. For example, it can be determined that the inference results of the multi-channel images are consistent by comparing and analyzing the above inference results.

If the inference results match, the image labeling server 140 may generate a labeling file based on the corresponding multi-channel image data (step S308). The image labeling server 140 may retrain the machine learning classification model by using the newly created labeling file as new learning data.

If the inference results do not match, the image labeling server 140 may perform crowd verification on the corresponding dataset (step S312). In an embodiment, the crowd verification may mean receiving verification results for a specific image dataset from a plurality of verifiers, that is, whether there is a detection target in the image and manually labeling the classification of the detection target. In one embodiment, the image labeling server 140 transmits the dataset in which the inference result does not match to the plurality of verifier terminals 130a, 130b, ... , 130n, and again to the plurality of verifier terminals 130a, 130b, ..., 130n) may receive the verification result. In an embodiment, the image labeling server 140 provides a dataset with inconsistent inference results, for example, four images to the verifier terminals 130a, 130b, ... , 130n, and a plurality of verifier terminals ( By receiving the verification results from 130a, 130b, ..., 130n), it is possible to determine whether the results received from the verifier terminals 130a, 130b, ..., 130n match.

According to an embodiment of the present disclosure, in step S314, the image labeling server 140 may adjust the number of verifier terminals 130a, 130b, ..., 130n. In an embodiment, if the verification results from the plurality of verifier terminals 130a, 130b, ..., 130n do not match each other, the image labeling server 140 may increase the number of verifier terminals requesting verification. have. In one embodiment, the image labeling server 140 may increase the number of verifier terminals until the verification results match, and when the number of verifier terminals exceeds a threshold, the image is used without further verification. may not In another embodiment, the image labeling server 140 may increase the number of verifier terminals until the verification results are the same, and when the number of verifier terminals exceeds a threshold, many of the verification results sent by the verifier terminals are Considering the sent result as a verification result, step S308 may be performed on the image.

Next, in step S316 , the image labeling server 140 may pay a reward to the verifier terminal inputting the correct labeling result. In an embodiment, the image labeling server 140 may accumulate and deduct points for settlement based on a result verified by each verifier terminal.

4 is an exemplary operation flowchart illustrating a learning flow of an image labeling server according to an embodiment of the present disclosure.

Steps S401 to S409 are similar to the general machine learning flow for object recognition in an image. A training dataset is prepared in step S401, and primary labeling is performed on the training dataset (S403). In general, the labeling of datasets for machine learning learning, that is, the input of bounding boxes and labels, is performed by humans. Thereafter, in step S405, machine learning is performed based on the training dataset prepared in step S403 to generate a learning result model, and in step S407, the actual image dataset is received and the corresponding learning result model is used. to perform object recognition on the actual image dataset (step S409).

In one embodiment of the present disclosure, in step S411, the object recognition result obtained by performing machine learning on multi-channel image data, that is, the result of step S409, may be compared and analyzed to determine the accuracy of the recognition result. If it is determined that the recognition result is correct, the multi-channel image data may be labeled as training data (step S413), and the machine learning model may be updated.

As will be appreciated by those skilled in the art, the present disclosure is not limited to the examples described herein, and various modifications, reconstructions, and substitutions may be made without departing from the scope of the present disclosure. For example, the various techniques described herein may be implemented by hardware or software, or a combination of hardware and software. Accordingly, certain aspects or portions of the analysis machine for software safety analysis according to the present disclosure may be implemented as one or more computer programs executable by a general-purpose or dedicated microprocessor, micro-controller, or the like. A computer program according to an embodiment of the present disclosure includes a storage medium readable by a computer processor or the like, such as EPROM, EEPROM, nonvolatile memory such as a flash memory device, a magnetic disk such as an internal hard disk and a removable disk, a magneto-optical disk, and It may be implemented in a form stored in various types of storage media including a CDROM disk and the like. In addition, the program code(s) may be implemented in assembly language or machine language, and may be implemented in a form transmitted through electric wiring, cabling, optical fiber, or any other type of transmission medium.

Although exemplary embodiments have been primarily described herein with reference to various drawings, other similar embodiments may be utilized. All modifications and variations falling within the true spirit and scope of the present disclosure are intended to be embraced by the following claims.

110: NVR
120: camera
130: verifier terminal
140: image labeling server
150: communication network

Claims (10)

A crowd validation-based multi-channel image learning data labeling method performed on a computer system, comprising:
receiving multi-channel image data;
generating a dataset based on the same time reference for the multi-channel image data, and inferring the posture of individual livestock in the image based on machine learning for each multi-channel image data of the corresponding dataset;
comparing and analyzing the inference results to compare and determine whether the inference results in the same dataset match; and
performing labeling on the multi-channel image data when the inference results in the same dataset match
A crowd validation-based multi-channel image training data labeling method comprising a. According to claim 1,
the method
When the inference results in the same dataset do not match, the image data included in the same dataset is transmitted to k (k>1, k is an odd number) verifier terminals, and the k verifier terminals Crowd-verification-based multi-channel image learning data labeling method further comprising the step of receiving a verification result of the image data included in the same dataset, and determining whether the verification result is consistent. 3. The method of claim 2,
the method
and performing labeling on the multi-channel image data when the verification results match. 4. The method of claim 3,
the method
When the verification results do not match, the image data included in the same dataset is transmitted to m (m>k, m is an odd number) number of validator terminals, and included in the same dataset from the m validator terminals. Crowd verification-based multi-channel image learning data labeling method further comprising the step of receiving the verification result of the image data, and determining whether the verification result is consistent. 3. The method of claim 2,
The method further comprises the step of retraining a machine learning model based on the new training data labeled for the multi-channel image data. 4. The method of claim 3,
The method is a crowd-verification-based multi-channel image learning data labeling method further comprising the step of providing a reward to the verifier terminal. A computer-readable recording medium containing one or more computer-readable instructions executable by a computer, wherein the one or more computer-readable instructions, when executed by the computer, cause the computer to: A computer readable recording medium for performing the method according to any one of the preceding claims. A crowd validation-based multi-channel image learning data labeling device, comprising:
a multi-channel image acquisition unit configured to receive a plurality of multi-channel image data;
a multi-channel image inference unit configured to infer an object included in an image based on a machine learning model for each of the plurality of multi-channel image data;
a multi-channel image inference result comparison and analysis unit configured to comparatively analyze the inferred result; and
A labeling file generating unit configured to generate a labeling file for each of the plurality of multi-channel image data
Crowd validation-based multi-channel image learning data labeling device comprising a. 9. The method of claim 8,
If the results of comparison and analysis of the inferred results do not match, the plurality of multi-channel image data is transmitted to a plurality of verification terminals, the verification results are received from the plurality of verifier terminals, and the verification results from the verifiers are Crowd verification-based multi-channel image training data labeling apparatus further comprising a crowd verification unit configured to determine whether they match. [Claim 10] The apparatus of claim 9, further comprising a learning unit for re-learning a machine learning model based on the new training data labeled with respect to the multi-channel image data.

Images