JP6700373B2 - Apparatus and method for learning object image packaging for artificial intelligence of video animation - Google Patents

Description

  The present invention relates to a learning target image packaging device and method for artificial intelligence of video animation.

  In recent years, the trend of artificial intelligence (AI) has been technically changing from voice and text recognition to video image recognition. Since the recognition of video images can be used for controlling autonomous vehicles, analyzing black box images, CCTV images, interpreting medical images, etc., it has a great ripple effect in various industries. Among them, in the research and development of four-stage autonomous vehicles, it is judged that 160 billion km of video image data is required worldwide and 100 million km of image is required for each city. In addition, recognition of video images requires a separate processing operation in order to use the collected video image data to make the images intelligent, but since the processing operations are performed manually based on human cognitive ability, It takes a lot of time and cost.

Korean Published Patent Publication No. 10-2004-0026825

  Therefore, the present invention has been made in view of the above circumstances, and an object thereof is to provide a learning target image packaging apparatus and method for artificial intelligence of video moving images.

  Problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned above should be clearly understandable by a person skilled in the art from the following description.

  A learning target image packaging method for artificial intelligence of a video moving picture according to an aspect of the present invention for solving the above-mentioned problems, includes an original moving picture receiving step of receiving an original road running video, and the original road running video. The primary sampling step of determining the number of extractions based on the crowded variable of the area to be extracted, and extracting the determined number of frame images from the original road running image, and each of the determined number of frame images are the first learning. A secondary sampling step of extracting at least one learning target frame image from the determined number of frame images based on whether the target object is included and the amount of movement of the first learning target object. The congestion variable of the area associated with the original road driving image includes at least one of demographic data or vehicle statistical data of the area associated with the original road driving image.

  In one embodiment, the primary sampling step further includes a metadata extraction step of extracting metadata of the original road driving image, and the extraction number is determined based on the metadata of the original road driving image. ..

  In one embodiment, the metadata extraction step uses one or more of GPS information of the vehicle, moving speed information of the vehicle, and a shooting time of the original road driving image, which are associated with the original road driving image. Extract the metadata of the original road driving video.

  In one embodiment, the secondary sampling step uses an object detection algorithm to generate one or more first learning target objects of one or more predetermined types on each of the determined number of frame images. A first learning target object detecting step of detecting the position of the one or more first learning target objects detected on the n-th (n is a natural number of 2 or more) frame image and the position immediately before the n-th frame image. The position of each of the one or more first learning target objects detected on the (n-1)th frame image is compared to determine the position of each of the one or more first learning target objects detected on the nth frame image. A first learning target object movement amount measuring step of measuring the movement amount is included.

  In one embodiment, the secondary sampling step further includes a background removal step of removing a background excluding the first learning target object on each of the determined number of frame images.

  In one embodiment, the method further includes a learning data set generation step of processing the one or more learning target frame images to generate a learning data set.

  In one embodiment, the learning data set generating step includes a second learning target object detecting step of detecting one or more second learning target objects of one or more predetermined types on the learning target frame image. , And a learning target object annotating step of performing an annotation process on the detected one or more second learning target objects on the learning target frame image.

  In one embodiment, the learning data set generating step further includes a validity verification step of verifying the validity of the learning data set.

  A learning target image packaging device for artificial intelligence of a video moving image according to another aspect of the present invention for solving the above-mentioned problems is an original road running image receiving unit for receiving an original road running image, the original road running image receiving unit. A first sampling unit that determines the number of extractions based on the crowded variable of the region associated with the image, and extracts the determined number of frame images from the original road driving image, and the determined number of frame images, respectively. A second sampling unit that extracts one or more learning target frame images from the determined number of frame images based on whether or not the first learning target object is included and the movement amount of the first learning target object. In addition, the congestion variable of the area associated with the original road driving image includes at least one of demographic data or vehicle statistical data of the area associated with the original road driving image.

  Other specific details of the invention are included in the detailed description and drawings.

  According to the learning target image packaging device and method for artificial intelligence of video moving image of the present invention, the first sampling unit may perform the primary road driving image based on the congestion variable of the area related to the original road driving image. Since the sampling is performed, the cost and time required for processing the data in the subsequent secondary sampling step and the learning data set generating step can be significantly reduced, and the learning target image for autonomous traveling is The effect that the extraction efficiency can be improved is exhibited.

  In addition, the second sampling unit sets a reference of the movement amount of the first learning target object and selects a frame image corresponding to a predetermined reference value or more as a learning target frame image, thereby eliminating unnecessary frames. The image is reduced and only the required frame image can be selected.

  Further, by extracting only the core frame images as the learning target frame images through the primary sampling and the secondary sampling, the learning target frame images have a small number but are equivalent to the non-sampling frame images. It is possible to extract a learning target image for autonomous driving having a level of performance.

  The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned should be clearly understood by a person of ordinary skill from the following description.

6 is a schematic flowchart of a learning target image packaging method for artificial intelligence of a video animation according to an embodiment of the present invention. FIG. 3 is a view illustrating a frame image extracted by the primary sampling step of FIG. 1 according to an exemplary embodiment of the present invention. 2 is a schematic flowchart of a secondary sampling stage of FIG. 1. FIG. 4 is an exemplary diagram illustrating a method for measuring a movement amount of a first learning target object in the secondary sampling stage of FIG. 1. 2 is a schematic flowchart of a secondary sampling stage of FIG. 1 with a background removal stage added. FIG. 6 is an exemplary diagram illustrating a method for measuring a movement amount of a first learning target object using a frame image in which a background is removed in the secondary sampling stage of FIG. 5 is a schematic flowchart of a learning target image packaging method for artificial intelligence of a video moving image with a metadata extraction step according to another embodiment of the present invention. 6 is a schematic flowchart of a learning target image packaging method for artificial intelligence of a video image with a learning data set generation step according to another embodiment of the present invention. 9 is a schematic flowchart of a learning data set generation step of FIG. 8. 9 is a schematic flowchart of a learning data set generation step of FIG. 8 to which a validity verification step is added. 11 is a schematic flowchart of the validity verification stage of FIG. 10. FIG. 6 is a schematic configuration diagram of a learning target image packaging device for artificial intelligence of a video image according to another embodiment of the present invention.

  The advantages and features of the present invention, and the manner of achieving them, will be apparent with reference to the embodiments described in detail below in connection with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and can be implemented in various forms different from each other. However, the present embodiment is provided to complete the disclosure of the present invention and to allow a person skilled in the art to which the present invention belongs to a full understanding of the scope of the present invention. , Only as defined by the scope of the claims.

  The terms used in the present specification are for describing the embodiments, and are not intended to limit the present invention. In this specification, the singular forms also include the plural unless specifically stated otherwise. As used herein, "comprises" and/or "comprising" does not exclude the presence or addition of one or more other components than those mentioned. Like reference numerals refer to like elements throughout the specification, and "and/or" includes each and every combination of the referred elements. Of course, even though "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one element from another. Therefore, of course, the first component referred to below can also be referred to as the second component within the technical idea of the present invention.

  Unless defined otherwise, all terms (including technical and scientific terms) used herein have the meaning commonly understood by one of ordinary skill in the art to which this invention belongs. Also, commonly used dictionary-defined terms are not ideally or excessively construed, unless explicitly defined otherwise.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a schematic flowchart of a learning target image packaging method for artificial intelligence of a video animation according to an embodiment of the present invention.

  Referring to FIG. 1, a learning target image packaging method for artificial intelligence of a video moving image includes an original moving image receiving step (S100), a first sampling step (S200) and a second sampling step (S300).

  In operation S100, the original road running image receiving unit 610 receives the original road running image. The original road driving image is a moving image collected by a camera provided inside or outside the vehicle or on a roadway. For example, when receiving an original road driving video to generate artificial intelligence video learning data for autonomous driving of a car, the original road driving video is equipped with a camera for generating learning data for autonomous driving. It is possible to include all videos capable of generating learning data for autonomous driving, such as moving images acquired from a car, and the invention is not limited thereto.

  In operation S200, the first sampling unit 620 determines the number of extractions based on the congestion variable associated with the original road driving image and extracts the determined number of frame images from the original road driving image. The congestion variable indicates the degree to which the population or vehicles are densely associated with a specific area. For example, if the congestion variable is relatively high, it can be analyzed that a relatively large amount of data needs to be learned in order to autonomously drive the area.

  The congestion variable of the area associated with the original road driving image includes one or more of demographic data or vehicle statistical data of the area associated with the original road driving image. A congestion variable calculation unit (not shown) calculates a congestion variable for a specific area using one or more of demographic data or vehicle statistical data for the specific area. The demographic data may include, but is not limited to, at least one of a population density, a population distribution, and a floating population corresponding to a position and time when the road driving image was captured. In addition, the vehicle statistical data may include at least one of the number of registered vehicles corresponding to the position and time at which the road driving image was captured, statistics of traffic volume by time zone, and distribution of public transportation. Not limited to.

  The first sampling unit 620 extracts the frame images by the number equal to or larger than the predetermined reference when the congestion variable is equal to or larger than the predetermined reference. For example, assume that the congestion variable can have a value of 1-10. In this case, the first sampling unit 620 extracts 30 frame images per second in an area where the congestion variable is 10 and is relatively high, and 15 frame images per second in an area where the congestion variable is 5 and is relatively low. Can be extracted.

  The first sampling unit 620 may receive the congestion variable corresponding to the demographic data or the vehicle statistical data via the public data server, but is not limited thereto.

  In operation S300, the second sampling unit 630 determines whether or not each of the determined number of frame images includes the first learning target object and the determined number based on the moving amount of the first learning target object. One or more learning target frame images are extracted from the frame images of. The secondary sampling step (S300) will be described in more detail with reference to FIG.

  FIG. 2 is an exemplary diagram of a frame image extracted by the primary sampling stage of FIG. 1 according to an embodiment of the present invention.

  Referring to FIG. 2, the frame image can be extracted at a frame sampling rate of 60 fps (frame per second), 30 fps, or 1 fps according to the first sampling step (S200).

  In operation S200, the first sampling unit 620 determines the number of extractions based on the congestion variable associated with the original road driving image and extracts the determined number of frame images from the original road driving image. For example, when the frame rate of the original road driving image is 60 fps, the first sampling unit 620 extracts 60, 30, or 1 frame per second according to the metadata of the original road driving image. be able to.

  In operation S200, the first sampling unit 620 performs primary sampling of the original road driving image based on the congestion variables associated with the original road driving image (i.e., removing unnecessary data and transmitting to the secondary sampling step). (To reduce the amount of data to be processed), and to significantly reduce the cost and time required to process the data in the subsequent secondary sampling step (S300) and learning data set generation step (S500). be able to.

  FIG. 3 is a schematic flowchart of the secondary sampling stage of FIG.

  Referring to FIG. 3, the secondary sampling step (S300) includes a first learning target object detecting step (S310) and a first learning target object movement amount measuring step (S320).

  In operation S310, the second sampling unit 630 uses the object detection algorithm to detect one or more first learning target objects of one or more predetermined types on each of the determined number of frame images. To do.

  The first learning target object is an object included in the frame image. The frame image includes one or more first learning target objects, and includes one or more types of first learning target objects. For example, the first learning target object includes a person, an automobile, a bicycle, a building, a telephone pole, a motorcycle, a tree, a flower, a puppy, a cat, a road, a traffic display board, an overspeed prevention step, a road cone, a lane, and the like. However, the present invention is not limited to the above example, and includes all objects that can be distinguished as objects. Further, the type of the first learning target object includes the front surface, the rear surface, the right side surface, the left side surface, etc. of the object, but the type of each learning target object is not limited to the above example, and can be subdivided from the above example. It can be classified and can be classified as a completely different type from the above example.

  Detecting one or more objects of one or more types as the detection of the first learning target is detecting using an object detection algorithm, and the object detection algorithm may include an R-CNN model. Yes, but not limited to this.

  In operation S320, the second sampling unit 630 determines the positions of the one or more first learning target objects detected on the n-th (n is a natural number of 2 or more) frame image and the n-th frame immediately before the n-th frame image. -Comparing the position of one or more first learning target objects detected on one frame image, and measuring the amount of movement of each of the one or more first learning target objects detected on the nth frame image. To do.

  The method of extracting the learning target frame image is performed by determining whether or not the detected movement amount of the one or more first learning target objects is equal to or more than a predetermined reference and detecting the one or more detected first learning objects. When the amount of movement of a predetermined number or more of the first learning target objects among the target objects is greater than or equal to a predetermined reference, the corresponding frame image is selected as the learning target frame image. However, if the movement amount of a predetermined number or more of the first learning target objects among the detected one or more first learning target objects does not correspond to a predetermined criterion or more, a corresponding frame image is displayed. It is not selected as a learning target frame image.

  By setting a reference for the amount of movement of the first learning target object and selecting a frame image corresponding to a predetermined reference or more as a learning target frame image, unnecessary frame images are reduced and necessary frame images are reduced. Only images can be selected.

  Therefore, as for the learning target frame image, all the extracted frame images are not selected as the learning target frame images, the moving amount of the object is small, and the remaining frame images are only the learning target except the frame images that do not have a large influence in learning. By selecting the frame image, the amount of the learning data set can be reduced in the subsequent stages. Therefore, in the learning of the artificial intelligence module, the learning time can be shortened by using the learning data set having the reduced amount.

  In another embodiment, the second sampling unit 630 determines the presence/absence of a traffic control object from the frame image, and if the corresponding traffic control object meets a predetermined criterion, further determines the frame based on the traffic control object. Extract one or more learning target frame images from the image. The traffic control object may be a traffic light on a frame image, a traffic safety sign, a traffic safety line, a road ancillary facility, or the like, but is not limited thereto. For example, when the traffic light is detected from the frame image and the corresponding traffic light changes from the red signal to the blue light, the second sampling unit 630 extracts the entire corresponding frame image as a learning target frame and maintains the red light of the corresponding traffic light. In this case, the number of frame images per second can be halved to extract only a part of the relevant frame images as learning target frames.

  In another embodiment, the second sampling unit 630 extracts at least one of an internal sound and an external sound of the vehicle when the original road driving image is captured, and the frequencies of the internal sound and the external sound of the vehicle are predetermined. If it is within the reference range, one or more learning target frame images are extracted from the frame images based on at least one of the internal sound and the external sound of the vehicle. For example, the second sampling unit 630 extracts audio data from a black box road running image, and when a scream is detected from the corresponding audio data, extracts 30 learning target frames per second and detects no sound. If not, 15 learning target frames can be extracted per second.

  The first learning target object movement amount measuring step (S320) will be specifically described with reference to FIG.

  FIG. 4 is an exemplary diagram illustrating a method for measuring the movement amount of the first learning target object in the secondary sampling stage of FIG. 1.

  Referring to FIG. 4, a method of measuring the movement amount in the first learning target object movement amount measuring step (S320) is shown. FIG. 4A shows the (n-1)th frame image 11, and FIG. 4B shows the nth frame image 12.

  The measurement of the movement amount of the first learning target object is to compare the positions of the first learning target object 21 on the n-1th frame image 11 and the first learning target object 22 on the nth frame image 12. is there.

  The first learning target object 21 on the (n-1)th frame image 11 and the first learning target object 22 on the nth frame image 12 have the same shape, and are located at the same position of the first learning target object. First select the relevant part.

  When the computer selects the specific portion of the first learning target object 21 on the n-1st frame image 11 as A, the nth frame image 12 is selected as the portion corresponding to the same position of the first learning target object. A part corresponding to the same position as A on the first learning target object 22 is selected as A′.

  The computer selects a portion corresponding to the same position of the first learning target object, places the (n-1)th frame image 11 and the nth frame image 12 on the same plane, and then extracts coordinates for A and A'. To do.

  After extracting the coordinates for A and A', the computer uses the difference between the A and A'coordinates to measure the amount of movement.

  FIG. 5 is a schematic flowchart of the secondary sampling stage of FIG. 1 with the addition of the background removal stage.

  Referring to FIG. 5, the background removal step (S330) is performed after the first learning target object detection step (S310).

  In operation S330, the second sampling unit 630 removes the background excluding the first learning target object on each of the determined number of frame images.

  In the background removal step (S330), the object other than the first learning target object detected on the frame image is processed as a background, and the background portion is entirely removed. The removal of the background portion may include setting the pixel value of the area corresponding to the background to a null state or processing the pixel value to a predetermined special value. For example, the gray of the pixel in the area corresponding to the background may be processed to be 0 gray or 256 gray, but the gray is not limited thereto.

  The secondary sampling step (S300) to which the background removal step (S330) is added will be specifically described with reference to FIG.

  FIG. 6 is an exemplary diagram illustrating a method for measuring a movement amount of a first learning target object using a frame image in which a background is removed in the secondary sampling step of FIG. 1 in which a background removal step is added. is there.

  Referring to FIG. 6, FIG. 6A illustrates a frame image 10 extracted in the first sampling step S200, and the frame image 10 includes a first learning target object 20 and a background 30.

  Referring to FIG. 6, FIG. 6B illustrates that the background 30 is removed in the background removal step S330 in the frame image 10 of FIG. 6A. Only the first learning target object 20 is included.

  Referring to FIG. 6, FIG. 6C illustrates positions of one or more first learning target objects 21 detected on the n-th (n is a natural number of 2 or more) frame image and the n-th frame image. It is a figure which shows comparing with the position of one or more 1st learning object 22 detected on the n-1st frame image from which the background just before was removed.

  The computer can measure the amount of movement of the first learning target object by comparing the first learning target objects 21 and 22.

  After measuring the movement amount by comparing the positions of the first learning target objects 21 and 22, the computer determines a predetermined number or more of the first learning target objects 20 among the detected one or more first learning target objects 20. If the amount of movement of the object is equal to or greater than a predetermined reference, the nth frame image is selected as the learning target frame image, and if not, the nth frame image is not selected as the learning target frame image.

  For example, the computer compares the movement amount of the first learning target object detected on the first frame image with the movement amount of the first learning target object detected on the second frame image, and the movement amount is predetermined. If it is equal to or more than the standard, the second frame image is selected as the learning target frame image. When the amount of movement does not correspond to a predetermined reference or more, the second frame image is not selected as the learning target frame image.

  Further, the computer is not limited to selecting or not selecting the second frame image as the learning target frame image, and is detected again on the first learning target object detected on the second frame image and on the third frame image. The movement amount of the first learning target object is compared, and if the movement amount is equal to or more than a predetermined reference, the third frame image is selected as the learning target frame image.

  The step of measuring the movement amount of the first learning target object on the frame image and selecting or not selecting it as the learning target frame image is the n-th (n is a natural number of 2 or more) one or more detected ones on the frame image. Comparing one learning target object with the positions of one or more first learning target objects detected on the (n-1)th frame image, the detected ones on all extracted frame images It is repeated until one or more first learning target objects are compared, and selection or non-selection as a learning target frame is completed.

  If there are one or more first learning target objects 20, and the computer has a plurality of first learning target objects 20 on one frame image 10, the computer compares the respective first learning target objects 20, The amount of movement of each first learning target object 20 is measured.

  For example, when a plurality of first learning target objects 20 are included in one frame image 10, the computer measures the movement amount of all of the plurality of first learning target objects 20 and determines a predetermined number of first learning target objects 20. When the amount of movement of the object 20 is equal to or greater than a predetermined reference, the corresponding frame image 10 is selected as the learning target frame image.

  In one embodiment, the computer measures the movement amount of all of the plurality of first learning target objects 20, and when the movement amount of all of the plurality of first learning target objects 20 is equal to or greater than a predetermined reference, the corresponding frame image. 10 is selected as a learning target frame image.

  In another embodiment, the computer measures only the movement amount of a predetermined number of first learning target objects 20 among the plurality of first learning target objects 20, and measures the measured movement of the first learning target object 20. When the movement amount of the predetermined number of first learning target objects 20 among the amounts is equal to or larger than the predetermined reference, the corresponding frame image 10 is selected as the learning target frame image.

  In still another embodiment, the computer measures only the movement amount of a predetermined number of first learning target objects 20 among the plurality of first learning target objects 20, and measures all the measured first learning target objects 20. If the amount of movement is greater than or equal to a predetermined reference, the corresponding frame image 10 is selected as the learning target frame image.

  As the learning target frame image selected by the measurement of the movement amount in FIG. 6C, the learning target is selected from the extracted frame images 11, 12, 13, 14, and 15 as shown in FIG. 6D. The frame images 12 and 14 are selected.

  FIG. 7 is a schematic flowchart of a learning target image packaging method for artificial intelligence of a video image with a metadata extraction step according to another embodiment of the present invention.

  Referring to FIG. 7, the metadata extracting step (S400) is performed after the original moving image receiving step (S100).

  In operation S400, the metadata extraction unit 640 uses one or more of the GPS information of the vehicle, the moving speed information of the vehicle, and the shooting time of the original road traveling image to relate to the original road traveling image. Extract video metadata.

  The first sampling unit determines the number of extractions based on the metadata of the original road traveling video in addition to the congestion variable, and extracts the determined number of frame images from the original road traveling video. The metadata of the original road traveling image includes at least one of GPS information of the vehicle related to the original road traveling image, moving speed information of the vehicle, and shooting time of the original road traveling image.

  In an exemplary embodiment, the first sampling unit 620 may correspond to a corresponding area if the GPS information of the vehicle is included in a predetermined area based on the vehicle GPS information of the metadata of the original road driving image. Extract as many frame images as you want. For example, the first sampling unit 620 may extract 30 frames per second when the vehicle is in the lively area A, and may extract 15 frames per second when the vehicle is in the quiet area B. it can.

  In an exemplary embodiment, the first sampling unit 620 may determine that the moving speed of the vehicle is within a predetermined speed range based on the vehicle moving speed information included in the metadata of the original road driving image. Extract a corresponding number of frame images. For example, if the moving speed of the vehicle is 100 km/h, which is relatively high, the first sampling unit 620 extracts 30 frames per second, and the moving speed of the vehicle is 50 km/h, which is relatively low. In this case, 15 frames can be extracted per second.

  In one embodiment, the first sampling unit 620 corresponds to the time range if the traveling time of the vehicle is within a predetermined time range based on the shooting time of the metadata of the original road driving image. Extract a number of frame images. For example, the first sampling unit 620 extracts 30 frames per second when the time is from 7:00 to 9:00 or 18:00 to 20:00, which is a time when the vehicle is crowded and the movement of the vehicle is smooth. In the case of 1 am to 5 am at dawn, 15 frames can be extracted per second.

  In another embodiment, the first sampling unit 620 determines that the weather information is included in the predetermined weather based on the weather information corresponding to the shooting time of the metadata of the original road driving image, when the weather information is included in the predetermined weather. Extract a corresponding number of frame images. For example, if the weather information corresponding to the shooting time is cloudy, strong wind, heavy rain, heavy snow, etc., the first sampling unit 620 extracts 30 frames per second, and if the weather information is sunny, per second. Fifteen frames can be extracted.

  FIG. 8 is a schematic flowchart of a learning target image packaging method for artificial intelligence of a video image with a learning data set generation step according to another embodiment of the present invention.

  Referring to FIG. 8, the learning data set generating step (S500) is performed after the secondary sampling step (S300).

  In step S500, the learning data set generation unit 650 processes one or more learning target frame images to generate a learning data set. The step S500 will be described in detail with reference to FIG.

  FIG. 9 is a schematic flowchart of the learning data set generation step of FIG.

  Referring to FIG. 9, the learning data set generating step (S500) includes a second learning target object detecting step (S510) and a second learning target object annotating step (S520).

  In step S510, the learning data set generation unit 650 detects one or more second learning target objects of one or more predetermined types on the learning target frame image.

  The second learning target object is an object included in the learning target frame image. The learning target frame image includes one or more second learning target objects, and one or more types of first learning target objects. For example, the second learning target object includes a person, an automobile, a bicycle, a building, a telephone pole, a motorcycle, a tree, a flower, a puppy, a cat, a road, a traffic display board, an overspeed prevention step, a road cone, a lane, and the like. However, the present invention is not limited to the above example, and includes all objects that can be distinguished as objects. Further, the type of the second learning target object includes the front surface, the rear surface, the right side surface, the left side surface, etc. of the object, but the type of each learning target object is not limited to the above example, and can be subdivided from the above example. It can be classified and can be classified as a completely different type from the above example.

  If it is determined in step S300 that the detection of the first learning target object is to select an image for learning artificial intelligence, that is, to select a learning target frame image, in step S510, the second learning target object is selected. The detection of is for selection of objects for the annotation process.

  As the detection of the second learning target object, detecting one or more objects of one or more types is to detect using an object detection algorithm, and the object detection algorithm includes an R-CNN model. However, it is not limited to this.

  In step S520, the learning data set generation unit 650 performs annotation processing on the detected one or more second learning target objects on the learning target frame image.

  The annotation process means an operation of generating data for explaining the detected second learning target object on the learning target frame image. The annotation process includes performing labeling, coloring, or layering on the second learning target object. In addition, displaying what the second learning target object is can be included as an annotation process. For example, the one or more second learning target objects detected on the learning target frame image can be displayed by dividing the area in the form of a box. The annotation process can utilize tools that are automatically processed.

  For the region where the second learning target object is segmented and displayed, a description can be made about what each second learning target object is, and the labeling may be made in one word, and in one word, Instead, it may be created in detail in text.

  The annotation process creates a database of the information labeled, colored, or layered by one or more workers, generates a module learned by a computer, and a newly input learning target frame image by the learned module. By determining whether or not the second learning target object above is similar to the second learning target object stored in the database, labeling, coloring, or layering can be performed on the second learning target object.

  In another embodiment, the learning data set generation unit 650 determines the importance of one or more second learning target objects detected on the learning target frame image using video processing, and determines the importance to be predetermined. If it is equal to or more than the reference, the annotation information for the second learning target object can be generated. When there are one or more second learning target objects, the learning data set generation unit 650 can generate annotation information only for the second learning target objects having high importance. For example, if there are cars and traffic lights on the learning target frame image, it is possible to determine that the cars and traffic lights have a high degree of importance, and generate annotation information only for the learned cars and traffic lights.

  FIG. 10 is a schematic flowchart of the learning data set generation step of FIG. 8 to which the validity verification step is added.

  Referring to FIG. 10, the validity verification step (S530) is performed after the learning target object annotation step.

  In step S530, the inspector verifies the validity of the training data set. The inspector terminal receives the learning data set from the learning target image packaging device 600 for artificial intelligence of video animation. The step S530 will be described in detail with reference to FIG.

  FIG. 11 is a schematic flowchart of the validity verification stage of FIG.

  Referring to FIG. 11, the validity verification step (S530) includes a step of requesting the examination of the learning data set (S531) and an examination of the learning data set (S532).

  In operation S531, the learning target image packaging device 600 for artificial intelligence of a video motion picture requests an inspector terminal to perform an inspection.

  In step S532, the inspector receives the learning data set and performs the inspection via the inspector terminal.

  In one embodiment, when the examination result of the learning data set, the reliability of the learning data set is equal to or higher than a predetermined standard, the inspector terminal determines the learning data set and the corresponding reliability to be artificial intelligence of the video movie. To the learning target image packaging device 600 for. Therefore, the learning target image packaging device 600 for artificial intelligence of a video movie can confirm that the corresponding learning data set is a data set suitable for use in the artificial intelligence module.

  In one embodiment, when the inspection result of the learning data set and the reliability of the learning data set are less than a predetermined standard, the corresponding learning data set is returned. Then, the inspector terminal requests the learning target image packaging device 600 for the artificial intelligence of the video image to rework the learning data set. The learning data set re-worked by the learning target image packaging device 600 for artificial intelligence of video animation is re-examined according to steps S531 to S532.

  FIG. 12 is a schematic block diagram of a learning target image packaging device for artificial intelligence of a video image according to another embodiment of the present invention.

  Referring to FIG. 12, a learning target image packaging device 600 for artificial intelligence of a video image includes an original road driving image receiving unit 610, a first sampling unit 620, a second sampling unit 630, a metadata extraction unit 640, and The learning data set generation unit 650 is included.

  The components of the learning target image packaging device 600 for artificial intelligence of the video animation of FIG. 12 correspond to the components of the method described with reference to FIGS. 1 to 11. The original road running video receiving unit 610 receives the original moving image (S100), the first sampling unit 620 receives the primary sampling stage (S200), the second sampling unit 630 receives the secondary sampling stage (S300), and metadata extraction. The unit 640 can perform the metadata extraction step (S400), and the learning data set generation unit 650 can perform the learning data set generation step (S500). In the following description of the function or operation of each component of device 600, redundant description will be omitted.

  The original road running video receiving unit 610 receives the original road running video.

  The first sampling unit 620 determines the number of extractions based on the crowded variable of the area associated with the original road running image, and extracts the determined number of frame images from the original road running image.

  The congestion variable of the area associated with the original road driving image includes one or more of demographic data or vehicle statistical data of the area associated with the original road driving image.

  The second sampling unit 630 determines whether each of the determined number of frame images includes the first learning target object and the determined number of frame images based on the movement amount of the first learning target object. One or more learning target frame images are extracted.

  The metadata extraction unit 640 extracts metadata of the original road running video.

  The learning data set generation unit 650 processes one or more learning target frame images to generate a learning data set.

  The steps of the method or algorithm described in connection with the embodiments of the present invention may be implemented directly in hardware, a software module executed by hardware, or a combination thereof. .. The software module is a RAM (Random Access Memory), a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Programmable Erasable ROM), a hard disk, a flash memory (FRAM). The present invention can also be present in any form of computer-readable recording medium known in the technical field to which the present invention belongs.

  Although the embodiments of the present invention have been described above with reference to the accompanying drawings, a person skilled in the art to which the present invention pertains does not change the technical idea or essential features of the present invention, and It should be understood that it can be implemented in a specific form of. Therefore, the embodiments described above are to be understood as illustrative in all aspects and not restrictive.

600 Learning Target Image Packaging Device for Artificial Intelligence of Video Animation 610 Original Road Driving Video Receiving Unit 620 First Sampling Unit 630 Second Sampling Unit 640 Metadata Extracting Unit 650 Learning Data Set Generating Unit

Claims (10)

Original video receiving stage to receive original road driving video,
A primary sampling step of determining an extraction number based on a congestion variable of an area shown in the original road running image and extracting the determined number of frame images from the original road running image;
Based on whether or not each of the determined number of frame images includes a first learning target object, and based on the amount of movement of the first learning target object, one or more of the determined number of frame images A second sampling stage for extracting a frame image to be learned,
The area congestion variable shown in the original road driving image includes at least one of demographic data or vehicle statistical data of the area shown in the original road driving image. Target image packaging method. The primary sampling step is
Further comprising a metadata extraction step of extracting metadata of the original road driving video,
The learning target image packaging method for artificial intelligence of a video image according to claim 1, wherein the extraction number is determined further based on the metadata of the original road driving image. The metadata extraction step includes
Extracting metadata of the original road travel video using one or more of GPS information of the vehicle that captured the original road travel video , traveling speed information of the vehicle, and shooting time of the original road travel video. The learning target image packaging method for artificial intelligence of a video according to claim 2, characterized in that. The second sampling step is
A first learning target object detecting step of detecting one or more first learning target objects of one or more predetermined types on each of the determined number of frame images using an object detection algorithm;
The position of the one or more first learning target objects detected on the n-th (n is a natural number of 2 or more) frame image and the detected position on the (n-1)-th frame image immediately before the n-th frame image. First learning target object movement that compares the position of the one or more first learning target objects and measures the amount of movement of each of the detected one or more first learning target objects on the nth frame image Quantity measurement stage,
The learning target image packaging method for artificial intelligence of video motion according to claim 1, comprising: The secondary sampling step includes
The method of claim 4, further comprising a background removal step of removing a background excluding the first learning target object on each of the determined number of frame images. Image packaging method for learning.   The method of claim 1, further comprising: a learning data set generating step of processing the one or more learning target frame images to generate a learning data set. Learning target image packaging method. The learning data set generating step includes
A second learning target object detecting step of detecting one or more second learning target objects of one or more predetermined types on the learning target frame image;
A learning target object annotating step of performing an annotation processing on the one or more second learning target objects detected on the learning target frame image;
The learning target image packaging method for artificial intelligence of video motion according to claim 6, comprising: The learning data set generating step includes
The method of claim 7, further comprising a validity verification step of verifying the validity of the training data set. An original road running video receiving unit that receives the original road running video,
A first sampling unit that determines an extraction number based on a crowded variable of an area shown in the original road running video and extracts the determined number of frame images from the original road running video;
Based on whether or not each of the determined number of frame images includes a first learning target object, and based on the amount of movement of the first learning target object, one or more of the determined number of frame images A second sampling unit for extracting a learning target frame image,
The area congestion variable shown in the original road driving image includes at least one of demographic data or vehicle statistical data of the area shown in the original road driving image. Target image packaging equipment. A program that causes a computer to execute the learning target image packaging method according to any one of claims 1 to 8 .

Images