KR102375541B1 - Apparatus for Providing Artificial Intelligence Service with structured consistency loss and Driving Method Thereof - Google Patents

Abstract

The present invention relates to an artificial intelligence service device having a structural consistency loss and a driving method of the device thereof, wherein the artificial intelligence service device according to an embodiment of the present invention may comprise: a communication interface part that receives an image consisting of a series of video frames; and a control part that performs deep learning analysis after dropping each of the video frames at different positions in a first deep learning network and a second deep learning network of different paths for the received series of video frames, and outputs each of a first deep learning analysis result and a second deep learning analysis result by bringing the video data of the video frame corresponding to each dropped position from an adjacent path and performing a deep learning analysis. Therefore, the present invention is capable of increasing a generalization ability of a deep learning network.

Description

Apparatus for Providing Artificial Intelligence Service with structured consistency loss and Driving Method Thereof

The present invention relates to an artificial intelligence service apparatus having structural coherence loss and a method of driving the apparatus, and more particularly, for example, an important frame feature in video classification by forcing a relationship between video frames of an image to be consistent. It relates to an artificial intelligence service device having a structural coherence loss that improves the performance of learning through the effect of structural coherence loss added to semi-supervised learning in semantic segmentation by making the relationship between them consistent, and a method of driving the device.

When learning the existing deep learning network, it learns in the direction of minimizing an objective function called a loss function. The core of a neural network is that it learns from data, and the answer to how to learn with that data is a loss function. , and use the value to improve the neural network (gradient descent, etc.).

Consistency loss means that even if a plurality of images are generated from one image through data augmentation, since the plurality of images are all generated from the same original image, they contain the same semantic information. It seems that there is, so it is a loss that makes the deep learning network predict consistently for these plurality of images, thereby increasing its generalization ability. Data augmentation, or augmentation, is a technique used to increase the amount of data by adding slightly modified copies of existing data or newly created synthetic data from existing data. It plays a regular role and helps reduce overfitting when training machine learning models.

The existing coherence loss was used in the field of image classification, that is, one prediction vector was generated per one image, and the cosine similarity was measured and the loss was calculated.

However, when it comes to dense prediction (eg object detection, semantic segmentation), it is necessary to make multiple predictions per image, and especially in the case of segmentation, prediction vectors are generated from every pixel in the image. In this case, the concept of consistency loss in image classification can be extended to the concept of pixel-wise consistency loss.

However, there is a limit in that the relationship between pixels cannot be learned by a simple extension of the existing concept, and also there is a limit in that the relationship between frames cannot be learned even if the current concept is expanded to video (or image) classification.

Korean Patent Publication No. 10-2219561 (2021.02.18) Korean Patent Publication No. 10-2021-0029089 (2021.03.15)

An embodiment of the present invention provides consistency in the relationship between important frame features in video classification by consistently forcing the relationship between video frames of an image, for example, through the effect of structural coherence loss added to semi-supervised learning in semantic segmentation. An object of the present invention is to provide an artificial intelligence service device having a loss of structural consistency that improves performance and a method of driving the device.

An artificial intelligence service apparatus having structural coherence loss according to an embodiment of the present invention includes a communication interface unit for receiving an image composed of a series of video frames, and a first dip of different paths with respect to the received series of video frames. Deep learning analysis is performed after dropping video frames at different locations in the learning network and the second deep learning network, respectively, by taking video data of video frames corresponding to the dropped locations from adjacent paths and a control unit that performs a learning analysis and outputs a first deep learning analysis result and a second deep learning analysis result, respectively.

The controller maintains the Nth video frame in the second deep learning network when dropping an Nth video frame (where N is a positive integer) among the received series of video frames from the first deep learning network can do it

The controller may fetch and analyze the video data of the N-th video frame of the second deep learning network during the deep learning analysis operation of the first deep learning network.

The controller may perform parallel processing on the same video frame in the first deep learning network and the second deep learning network to output a deep learning analysis result, respectively.

The controller may process video data of a video frame in a form of processing a feature vector with respect to a specified object in the video frame.

The controller may drop one or more video frames when the same content in the video frame continues for a specified time.

In addition, the method of driving an artificial intelligence service apparatus having structural coherence loss according to an embodiment of the present invention includes the steps of: receiving, by a communication interface unit, an image composed of a series of video frames; and, by the control unit, the received series of videos After dropping video frames at different positions in the first deep learning network and the second deep learning network of different paths with respect to the frame, deep learning analysis is performed, and video data of video frames corresponding to the dropped positions respectively and performing deep learning analysis by bringing them from an adjacent path and outputting a first deep learning analysis result and a second deep learning analysis result, respectively.

According to an embodiment of the present invention, if there is a pixel-wise prediction in the case of video frame segmentation, a feature vector can be created for each frame in the video in video classification, By coherently forcing the relationship between these feature vectors, it will be possible to increase the generalization ability of deep learning networks.

In addition, according to the embodiment of the present invention, the effect of structural coherence loss added to semi-supervised learning in semantic segmentation may improve learning performance.

1 is a diagram showing an artificial intelligence service system having structural consistency loss according to an embodiment of the present invention;
Figure 2 is a view for explaining the operation method of the artificial intelligence service device having the structural consistency loss of Figure 1;
3 and 4 are diagrams for explaining a method of deep learning analysis of drunkenness in real images;
5 is a block diagram illustrating the detailed structure of an artificial intelligence service device having structural coherence loss of FIG. 1, and
6 is a flowchart illustrating a driving process of the artificial intelligence service device having the structural consistency loss of FIG. 1 .

Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

1 is a diagram illustrating an artificial intelligence service system having structural consistency loss according to an embodiment of the present invention, and FIG. 2 is a diagram for explaining an operation method of the artificial intelligence service apparatus having structural consistency loss of FIG. 1 .

As shown in FIG. 1 , the artificial intelligence service system 90 according to the embodiment of the present invention is a part of a photographing device 100 , an image providing device 110 , a communication network 120 , and an artificial intelligence service device 130 . or all inclusive.

Here, "including some or all" means that some components such as the photographing device 100 or the image providing device 110 are omitted to configure the artificial intelligence service system 90, or the artificial intelligence service device 130 It means that some or all of can be configured by being integrated into a network device (eg, a wireless switching device, etc.) constituting the communication network 120, and it will be described as including all in order to help a sufficient understanding of the invention.

The photographing apparatus 100 includes a camera installed in an arbitrary area or space such as a road or subway space to photograph a specific area in order to determine whether an event or an accident has occurred. The camera includes a CCTV or IP camera, and includes a fixed camera and a PTZ (Pan, Tilt, Zoom) camera. The photographing apparatus 100 may photograph an arbitrary area and transmit the photographed image in real time or periodically.

The image providing device 110 provides, for example, a photographed image captured through a black box, a camcorder, or a smart phone, even if it is not the case of the photographing device 100, or a desktop computer, a laptop computer, a tablet PC, It includes a device for providing a pre-recorded image through a wearable device worn on a smart phone, a wrist, or the like. For example, a digital image storage device (DVR) for storing images captured by a specific device may be included in the communication network 120 . For example, when the artificial intelligence service device 130 provides an artificial intelligence service to users in the form of a platform, for example, it is possible to provide a deep learning analysis result after receiving and analyzing a customer's image.

The communication network 120 includes both wired and wireless communication networks. For example, a wired/wireless Internet network may be used or interlocked as the communication network 120 . Here, the wired network includes an Internet network such as a cable network or a public telephone network (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, Evolved Packet Core (EPC), Long Term Evolution (LTE), Wibro network, etc. is meant to include Of course, the communication network 120 according to the embodiment of the present invention is not limited thereto, and may be used, for example, in a cloud computing network under a cloud computing environment, a 5G network, and the like. For example, when the communication network 120 is a wired communication network, the access point in the communication network 120 can connect to a switching center of a telephone company, etc., but in the case of a wireless communication network, it accesses the SGSN or GGSN (Gateway GPRS Support Node) operated by the communication company to transmit data. Data can be processed by accessing various repeaters such as a BTS (Base Transsive Station), NodeB, and e-NodeB.

The communication network 120 may include an access point. Here, the access point includes a small base station, such as a femto or pico base station, which is often installed in a building. Femto or pico base stations are classified according to the maximum number of access to the imaging device 100 or the image providing device 110, etc. in the classification of a small base station. Of course, the access point may include a short-distance communication module for performing short-distance communication such as Zigbee and Wi-Fi with the photographing device 100 or the image providing device 110 . The access point may use TCP/IP or Real-Time Streaming Protocol (RTSP) for wireless communication. Here, short-distance communication may be performed in various standards such as Bluetooth, Zigbee, infrared, radio frequency (RF) such as ultra high frequency (UHF) and very high frequency (VHF), and ultra-wideband communication (UWB) in addition to Wi-Fi. Accordingly, the access point can extract the location of the data packet, designate the best communication path for the extracted location, and deliver the data packet to the next device, such as the artificial intelligence service device 130 , along the designated communication path. The access point may share several lines in a general network environment, and includes, for example, a router, a repeater, and a repeater.

The artificial intelligence service device 130 may operate, for example, as a server, and is equipped with an artificial intelligence program to systematically classify and store the image analysis result in the DB 130a. The artificial intelligence service device 130 according to an embodiment of the present invention learns in a direction to minimize an objective function called a loss function when learning a deep learning network. That is, the artificial intelligence service device 130 performs an operation related to loss of structural consistency for video classification. To this end, the artificial intelligence service device 130 consistently enforces the relationship between frames so that the relationship between important frame features in video classification is consistent. As such, the effect of loss of structural consistency added to semi-supervised learning in semantic segmentation is to improve learning performance.

More specifically, as shown in Figure 2, the artificial intelligence service device 130 is a first deep learning network (or a first deep learning unit) for a series of the same video frame in order to perform a deep learning operation by an artificial intelligence program. and the second deep learning network (or the second deep learning unit) perform deep learning operations, respectively. Here, the first deep learning network and the second deep learning network may be the first deep learning module and the second deep learning module, and it is preferable to have the same deep learning program. Of course, it will not be particularly limited thereto. However, it can be seen that a plurality of deep learning networks, ie, modules, perform parallel processing operations for deep learning on the same video frame. For example, if the first to the seventh video frames are received for a specified time, the frames may be provided to the first deep learning network and the second deep learning network, respectively. In the embodiment of the present invention, two deep learning networks are exemplified, but in practice, it is preferable to configure a plurality of deep learning networks. The deep learning operation related to the loss function has already been sufficiently described, so we will replace it with the contents.

As shown in FIG. 2, the artificial intelligence service device 130 can create a feature vector for each frame in the video in video classification, and it can be seen that the generalization ability of the deep learning network is increased by consistently forcing the relationship between the feature vectors. there is. For example, when tracking an object in a video frame, it is representative to determine a movement position by processing a vector value for the tracking object. For example, since an object in a video frame at an arbitrary location has moved by a distance L in the X direction, an arbitrary object is that can be tracked. Of course, since the operation of processing an image for deep learning analysis, tracking the position of an object in the process, and determining an event or accident event occurring in the object can be performed in various ways, in an embodiment of the present invention, any It will not specifically limit to one form. 2 shows structural coherence loss (eg, L2 distance loss) for video classification using the cosine similarity between the feature vectors and the feature vectors for each frame in different deep learning networks. The cosine similarity is, for example, one of the methods of obtaining the similarity of text data, and is a method of obtaining the similarity between two vectors by obtaining the cosine angle between the two vectors.

More specifically, the artificial intelligence service device 130 according to an embodiment of the present invention drops video frames at different time positions with respect to a series of video frames of the same content input to different networks. For example, if the (n-1)-th video frame is dropped when 5 video frames input in the first deep learning network, that is, when n = 5, in the second deep learning network, (n-1) for the same 5 video frames )th video frame and drop at least one video frame from the remaining video frames. This process can be considered to be to reduce the computational throughput for data of the same video frame during deep learning operation so that rapid computation is performed. Of course, such a drop operation may be preset, but may vary depending on the content within the video frame, that is, the content. In other words, if the same scene lasted for 5 video frames, it is possible to drop 2-3 video frames in the middle.

Above all, in the artificial intelligence service device 130 according to the embodiment of the present invention, the video data of the dropped video frame, which may be data in the form of a feature vector, as mentioned above, may be located at different locations in a plurality of deep learning networks. After performing a drop operation on the video frame of After performing the running motion, the analysis result is output.

3 and 4 are diagrams for explaining a method of deep learning analysis of a drunken in an actual image.

Referring to FIGS. 3 and 4 together with FIG. 1 for convenience of explanation, the artificial intelligence service device 130 of FIG. 1 according to an embodiment of the present invention is a (photographed) image of a video frame as in FIG. may be received by the photographing apparatus 100 or the image providing apparatus 110 of FIG. 1 . Here, the image providing device 110 includes a device for providing a video shot through a black box or a user's smart phone.

The artificial intelligence service device 130 receives a series of video frames as in (a) of FIG. 3 as input to the first deep learning network and the second deep learning network as in FIGS. 3 (b) and (c), respectively. . Here, the first deep learning network and the second deep learning network may mean an IC chip on which a deep learning module, that is, a deep learning program is mounted, and for convenience of explanation, two deep learning networks are illustrated in FIG. 3 . However, since a plurality of deep learning networks can be further configured, the embodiment of the present invention will not be particularly limited to any one form.

The artificial intelligence service device 130 performs deep learning analysis in the deep learning network by dropping the second and third video frames among a series of video frames input to the first deep learning network, such as the first video frame and the fifth video frame. do. In addition, the artificial intelligence service device 130 performs deep learning analysis in the deep learning network by dropping the fifth video frame from a series of video frames as in (a) of FIG. 3 input to the second deep learning network. When the artificial intelligence service device 130 according to an embodiment of the present invention configures a plurality of deep learning networks, it is preferable not to drop video frames at positions overlapping each other. This is absolutely necessary to bring data of video frames dropped from one deep learning network to an adjacent deep learning network, which can be seen as essential for deep learning.

As described above, in the artificial intelligence service device 130 of FIG. 1 , as can be seen in FIG. When there is a relationship between features 1 to 4 of deep learning network No. 1 shown in (b) and a relationship between 1 to 4 of deep learning network No. 2 shown in FIG. It is difficult to understand the relationship between 1 and 4 because it is not possible to know what kind of video frame it is because of viewing, i.e., because it is dropped, and because network 2 sees both images 2 and 3, that is, because it can recognize what kind of video frame it is. Since the relationship is easy to understand, a structured consistency loss is given so that the relationship (1 to 4) between the feature vectors of the second network can be learned from the first network. In the end, it is difficult for Network 1 to match the main character, the label of the video viewed in the absence of video frames 2 and 3, but by learning the frame-to-frame relationship from the prediction of Network 2, it is possible to learn the same person between frames 1 to 4 By taking off your coat and confirming that you are about to lie down, you will be able to match the class or class of the drunkard.

As described above, the artificial intelligence service device 130 according to an embodiment of the present invention performs learning by bringing data for a video frame dropped from one deep learning network and related data from an adjacent deep learning network to learn each. It will be possible to perform computational processing in a deep learning network quickly and at the same time increase the learning accuracy. In the end, when learning the deep learning network, learning is performed in the direction of minimizing the objective function called the loss function.

FIG. 5 is a block diagram illustrating a detailed structure of an artificial intelligence service device having structural consistency loss of FIG. 1 .

As shown in FIG. 5 , the artificial intelligence service device 130 according to an embodiment of the present invention includes a communication interface unit 500 , a control unit 510 , a feature vector forcing unit 520 , and a part of the storage unit 530 . or all inclusive.

Here, "including some or all" means that some components such as the storage unit 530 are omitted to configure the artificial intelligence service device 130, or some components such as the feature vector forcing unit 520 are controlled by the control unit. As meaning that it can be configured by being integrated with other components such as 510, it will be described as including all to help a sufficient understanding of the invention.

Of course, although the artificial intelligence service device 130 according to the embodiment of the present invention has been described as being operated in the form of a server in FIG. 1, it may be an individual device configured in a stand-alone form such as a computer, and thus Since it is possible to perform the operation described above by connecting a storage medium (eg, USB, etc.) to a port, the embodiment of the present invention will not be limited to any one form.

The communication interface unit 500 receives a video frame of a moving picture from the photographing apparatus 100 or the image providing apparatus 110 via the communication network 120 of FIG. 1 . The received video frame is provided to the controller 510 . Of course, in this process, the communication interface unit 500 may perform operations such as modulation/demodulation, muxing/demuxing, encoding/decoding, etc., which are obvious to those skilled in the art, so further description will be omitted. .

The control unit 510 is in charge of overall control operations of the communication interface unit 500 of FIG. 5 , the feature vector forcing unit 520 , and the storage unit 530 constituting the artificial intelligence service device 130 of FIG. 1 . In other words, the control unit 510 sequentially stores a series of video frames received from the communication interface unit 500 in the storage unit 530 , recalls them in the stored order, and provides them to the feature vector forcing unit 520 . can do. In addition, the control unit 510 may control the feature vector forcing unit 520 to execute an artificial intelligence program for forcing the feature vector mounted therein, and the execution result is again systematically stored in the DB 130a of FIG. 1 . The communication interface unit 500 may be controlled to be classified and stored.

The feature vector forcing unit 520 may configure a plurality of deep learning network modules. Accordingly, a plurality of deep learning network modules can perform a parallel processing operation to perform a deep learning operation, and each deep learning module drops video frames at different time positions, and video for the dropped video frames Data, for example, data in the form of a feature vector may be imported from an adjacent deep learning module to perform a deep learning operation. Of course, adjacent here means not only the second deep learning module or the fourth deep learning module in the case of the third deep learning module, but also the video dropped in the form of skipping like the first deep learning module or the fifth deep learning module. You can also get the data of the frame, more precisely, it is preferable to get the video data, that is, the feature vector from the deep learning module that maintains the video data for the dropped video frame.

In this way, the feature vector forcing unit 520 performs an operation of forcing the result of analyzing the dropped video frame, that is, the feature vector. That is, it operates with generalized capabilities. Accordingly, it can be seen that the feature vector coercion unit 520 drops video frames in a preset manner in order to have a generalized capability and performs learning in a direction to reduce the loss function. For example, the number of video frames to be dropped may vary according to content content within the received video frame. For example, it is desirable to reduce the number of dropped video frames when the object within a video frame moves quickly, and when the object moves slowly, increasing the number of dropped video frames reduces the computational burden and speeds up processing. can increase As described above, the dropped video frame may vary according to the characteristics of the received video frame. Of course, in the embodiment of the present invention, it is preferable to operate in a fixed form rather than a variable, but it will not be particularly limited to any one form.

The storage unit 530 may temporarily store data of a video frame received under the control of the control unit 510 and then output it. In general, video data received through a data packet may be provided with additional information on the corresponding video data, for example, information on the format of encoding, and subtitle information in the case of an image, in the header part, and the payload part Real data for pixels constituting the video frame, that is, pixel values for the pixel data is received. Accordingly, as mentioned above, the compressed, that is, encoded video data is decoded to restore the original video frame, and when the video data is restored, it is restored by referring to encoding information as additional information. As such, a series of restored video frames are stored in the storage 530 .

In addition to the above, the communication interface unit 500, the control unit 510, the feature vector forcing unit 520, and the storage unit 530 of FIG. 5 may perform various operations, and other details have been sufficiently described above. instead of content.

The communication interface unit 500, the control unit 510, the feature vector forcing unit 520, and the storage unit 530 of FIG. 5 according to an embodiment of the present invention are composed of hardware modules physically separated from each other, but each module is It may be possible to store software for performing the above operation therein and execute it. However, since the software is a set of software modules, and each module can be formed of hardware, it will not be particularly limited to the configuration of software or hardware. For example, the storage unit 530 may be a hardware storage (storage) or a memory (memory). However, since it is possible to store information in software (repository), it will not be particularly limited to the above.

Meanwhile, as another embodiment of the present invention, the control unit 510 may include a CPU and a memory, and may be formed as a single chip. The CPU includes a control circuit, an arithmetic unit (ALU), a command interpreter and a registry, and the memory may include a RAM. The control circuit performs a control operation, the operation unit performs an operation operation of binary bit information, and the instruction interpretation unit converts a high-level language into a machine language and a machine language into a high-level language, including an interpreter or compiler. , the registry may be involved in software data storage. According to the above configuration, for example, at the beginning of the operation of the artificial intelligence service device 130, the program stored in the feature vector coercion unit 520 is copied, loaded into a memory, that is, RAM, and then executed, thereby speeding up data operation processing. can be increased quickly.

6 is a flowchart illustrating a driving process of the artificial intelligence service device having the structural consistency loss of FIG. 1 .

Referring to FIG. 6 together with FIG. 1 for convenience of explanation, the artificial intelligence service device 130 according to an embodiment of the present invention receives a (moving) image composed of a series of video frames (S600). Since the artificial intelligence service device 130 of FIG. 1 may include a server or a computer operating in a standalone form, it is possible to receive the received video through a storage medium or the like.

In addition, the artificial intelligence service device 130 performs deep learning analysis after dropping video frames at different locations in the first deep learning network and the second deep learning network of different paths with respect to the received series of video frames, respectively. However, the video data of the video frame corresponding to each dropped position is taken from an adjacent path, a deep learning analysis is performed, and a first deep learning analysis result and a second deep learning analysis result are respectively output (S610).

For example, when the artificial intelligence service device 130 drops the second and third video frames from the first to fifth video frames received by the first deep learning network (pre), the dropped second and third videos After finding or recognizing the data of a frame, such as an adjacent deep learning network that has feature vector information, for example, if the second deep learning network has the corresponding feature vector information, the data is retrieved and the deep learning operation is performed to output the analysis result. there is. Of course, if there is corresponding feature vector information in the third deep learning network with respect to the dropped video frame, it is also possible to perform the deep learning operation by bringing it from the third deep learning network. I won't.

For example, if the location of a video frame dropped by each deep learning network is fixed, other deep learning networks can share it, but as mentioned above, if it is variable, inquire with an adjacent deep learning network. Alternatively, it may operate in the form of retrieving feature vector information when there is an answer by searching. Here, the feature vector may mean tracking the movement of an object based on vector information with respect to a feature of an arbitrary object, for example. Therefore, when the movement of an object occurs with respect to time change, it is possible to know what kind of object it is based on the characteristic information. Therefore, by judging only the position using the vector information, data analysis can be done with only vector information with little information. Of course, although the feature vector information has been exemplified in the embodiment of the present invention, it will not be particularly limited thereto.

On the other hand, even though it has been described that all components constituting the embodiment of the present invention are combined or operated in combination, the present invention is not necessarily limited to this embodiment. That is, within the scope of the object of the present invention, all the components may operate by selectively combining one or more. In addition, all of the components may be implemented as one independent hardware, but a part or all of each component is selectively combined to perform some or all of the combined functions in one or a plurality of hardware program modules It may be implemented as a computer program having Codes and code segments constituting the computer program can be easily deduced by those skilled in the art of the present invention. Such a computer program is stored in a computer-readable non-transitory computer readable media, read and executed by the computer, thereby implementing an embodiment of the present invention.

Here, the non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, not a medium that stores data for a short moment, such as a register, cache, memory, etc. . Specifically, the above-described programs may be provided by being stored in a non-transitory readable recording medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

100: photographing device 110: image providing device
120: communication network 130: artificial intelligence service device
500: communication interface unit 510: control unit
520: feature vector forcing unit 530: storage unit

Claims (7)

a communication interface unit for receiving an image composed of a series of video frames; and
Deep learning analysis is performed after dropping video frames at different positions in the first deep learning network and the second deep learning network of different paths with respect to the received series of video frames, respectively, A control unit for taking video data of a video frame corresponding to a location from an adjacent path and performing deep learning analysis to output a first deep learning analysis result and a second deep learning analysis result, respectively;
The video data of the video frame corresponding to each dropped position is,
It is data that has been subjected to predetermined processing in a deep learning network that is not dropped on the video data of the dropped video frame,
The adjacent path is
In the case of the first deep learning network, the second deep learning network is an adjacent path, and in the case of the second deep learning network, the first deep learning network is an artificial intelligence service device having a structural coherence loss. . The method of claim 1,
The control unit maintains the N-th video frame in the second deep learning network when dropping an N-th (here, N is a positive integer) video frame among the received series of video frames in the first deep learning network An artificial intelligence service device with a loss of structural consistency. 3. The method of claim 2,
The control unit, an artificial intelligence service device having a structural coherence loss for fetching and analyzing the video data of the N-th video frame of the second deep learning network during the deep learning analysis operation of the first deep learning network. The method of claim 1,
The control unit performs parallel processing in the first deep learning network and the second deep learning network for the same video frame to output a deep learning analysis result, respectively, an artificial intelligence service device having a structural consistency loss. The method of claim 1,
The control unit, an artificial intelligence service device having a structural coherence loss for processing the video data of the video frame in the form of processing a feature vector (feature vector) with respect to a specified object in the video frame. The method of claim 1,
The control unit, an artificial intelligence service device having a structural coherence loss to drop one or more video frames when the same content in the video frame continues for a specified time. receiving, by the communication interface unit, an image composed of a series of video frames; and
The control unit performs deep learning analysis after dropping video frames at different positions in the first deep learning network and the second deep learning network of different paths with respect to the received series of video frames, respectively, Taking video data of a video frame corresponding to a location from an adjacent path, performing deep learning analysis, and outputting a first deep learning analysis result and a second deep learning analysis result, respectively; including;
The video data of the video frame corresponding to each dropped position is,
It is data that has been subjected to predetermined processing in a deep learning network that is not dropped on the video data of the dropped video frame,
The adjacent path is
In the case of the first deep learning network, the second deep learning network is an adjacent path, and in the case of the second deep learning network, the first deep learning network is an artificial intelligence service device having a structural coherence loss. driving method.

Images