KR102462934B1 - Video analysis system for digital twin technology - Google Patents

Abstract

In order to implement a virtual human object that can be applied to a digital twin technology that performs simulation by composing a real object in a virtual space, the present invention tracks human movement based on image analysis and provides information on its posture. It relates to an image analysis system for digital twin technology that collects and generates movements and postures of virtual human objects that can be applied to digital twin technology based on this, and analyzes real images to determine movement paths and skeletons of objects in images After storing information-level posture information together, it is possible to create virtual objects with movement paths and posture information based on the situational distribution of the stored object information. This has the effect of improving the quality of digital twin-based simulations.

Description

Video analysis system for digital twin technology

The present invention relates to an image analysis system for digital twin technology, and more particularly, image analysis to implement a virtual human object that can be applied to digital twin technology that performs simulation by configuring a real object in a virtual space. About an image analysis system for digital twin technology that tracks human movements based on will be.

Digital twin technology is a technology that predicts the result by forming a physical object into a virtual model identical to the real one in the virtual space of a computer. .

In this way, by constructing virtual facilities identical to the real ones in the virtual space and simulating various situations using the facilities in the virtual environment, effects such as understanding the operation method for optimal utilization and improvement contents can be obtained. is regarded as the core of the 4th industrial revolution.

In particular, various 3D modeling technologies and technologies such as virtual reality and physics engines for various virtual environments are developing rapidly, and even urban-scale virtual reality environments are being built and utilized for various services. Much attention is focused on

However, in order for the simulation through this digital twin technology to proceed effectively, information on various target environments must be accurately applied. can

Currently, only information about the amount of statistical movement or movement trajectory of virtual human objects has been utilized, but more precise information on virtual objects is required for the advancement of digital twin technology.

Korean Patent Registration No. 10-1989982, [Modeling system and modeling method through digital twin-based indoor space analysis] Korean Patent Registration No. 10-1325401, [Device and method for generating a movement path of a virtual object]

An object of the present invention to solve the above problems is to analyze the actual image to track the movement path of the objects in the image, and to extract the posture of the object as skeleton information according to the joint classification to determine the movement path and posture of the real object. By collecting specific information including, but learning it, the performance of classifying the posture information of an object in the image is improved, and based on the posture of the object, the same object can be maintained even when objects in the image are clustered, lost, or occluded. , to effectively collect information about human objects in the real world.

Furthermore, another object of the present invention is to create a virtual human object to be applied to a digital twin based on the stored object information for each classification after storing classification information about a situation in which information on the movement and posture of the real human object is collected. The performance of the digital twin technology can be improved by reflecting the realistic environment that includes the movement of statistical objects and even the sudden movement of objects according to the specific movement of real human objects.

The image analysis system for digital twin technology according to an embodiment of the present invention selects an object from images collected through a camera or an image database, learns the posture of the selected object through a neural network model, and uses the neural network model obtained after learning. After classifying the posture of the object using the method to calculate the posture information including the skeleton information, the object information including the identifier of the object, the movement path and the posture information is stored in the image analysis database, and the object Storing the stored object information classification information in the image analysis database according to at least one classification criterion among the date, time, weather, and environmental conditions previously selected as being capable of affecting the appearance of the object, and the object stored in the image analysis database An image analysis-based object creation device that creates virtual objects for each classification criterion based on information and stores them in a virtual object database, configures virtual reality model data for a simulation target in a virtual reality environment, and performs simulation in an image analysis-based object creation device It includes a digital twin-based simulation device that requests virtual objects according to conditions, receives virtual object information for each classification having variability accordingly, and applies it to a virtual reality environment.

As an example, the apparatus for generating an object based on image analysis includes an image collection unit that collects images through a camera or an image database, and an object selection unit that selects a region of interest from the images collected by the image collection unit and selects objects existing in the region of interest A posture learning unit that learns the posture of the object selected by the object selector through a neural network model, and a neural network model to which the weights learned through the posture learning unit are applied to classify the posture of the object selected by the object selector By assigning an identifier to each of the plurality of objects selected by the posture classification unit and the object selection unit, the plurality of objects are continuously tracked at the same time, and the posture of each object classified in the posture classification unit is linked to the tracking objects to cluster and disappear. Alternatively, when an occluded object reappears, the previous object identifier is maintained or newly created based on the classified posture, and object information including the object identifier and movement path and posture information for the tracked objects is stored in the image analysis database. It may include an object identification and tracking unit.

On the other hand, the object identification and tracking unit further includes classification information in the stored object information according to at least one classification criterion among the date, time, weather, and environmental conditions previously selected as being capable of affecting the appearance of the object to be tracked. It can be stored in the image analysis database.

In addition, a virtual object for each classification criterion is created based on the object information stored in the image analysis database, but a path and a similar path and and a virtual object generator that generates a virtual object having a posture and stores the virtual object in a virtual object database.

Furthermore, when virtual objects according to simulation conditions are requested from the digital twin-based simulation device, the virtual object generator may select and provide virtual objects for each classification according to the simulation condition from among the stored virtual objects according to the distribution by classification.

The image analysis system for digital twin technology according to an embodiment of the present invention analyzes an actual image, stores movement paths of objects in the image and posture information of the skeleton information level together, and then based on the situational distribution of the stored object information By making it possible to create virtual objects with movement path and posture information, the motion of virtual human objects close to reality can be provided to the digital twin-based simulation, thereby increasing the quality of the digital twin-based simulation.

In particular, after classifying the path, movement, and posture information of real human objects into sub-categories, information on virtual objects is generated based on this information. Even information on the specific behavior of objects can be reflected, which has the effect of greatly improving the completeness of the digital twin-based simulation.

1 is a conceptual diagram showing the configuration of an image analysis system for digital twin technology according to an embodiment of the present invention.
2 is a block diagram showing the configuration of an apparatus for generating an object based on image analysis according to an embodiment of the present invention.
3 is a conceptual diagram of an image analysis neural network according to an embodiment of the present invention.
4 is a conceptual diagram of a neural network for selecting an object of interest in an image according to an embodiment of the present invention.
5 is a conceptual diagram of a neural network for determining the posture of an object in an image according to an embodiment of the present invention.
6 is an exemplary diagram illustrating a posture determination result of an object in an image according to an embodiment of the present invention.
7 is a flowchart illustrating an operation process according to an embodiment of the present invention.

It should be noted that the technical terms used in the present invention are only used to describe specific embodiments, and are not intended to limit the present invention. In addition, the technical terms used in the present invention should be interpreted as meanings generally understood by those of ordinary skill in the art to which the present invention belongs, unless otherwise specifically defined in the present invention. It should not be construed as meaning or in an excessively reduced meaning. In addition, when the technical term used in the present invention is an incorrect technical term that does not accurately express the spirit of the present invention, it should be understood by being replaced with a technical term that can be correctly understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted as defined in the dictionary or according to the context before and after, and should not be interpreted in an excessively reduced meaning.

Also, the singular expression used in the present invention includes the plural expression unless the context clearly dictates otherwise. In the present invention, terms such as "consisting of" or "comprising" should not be construed as necessarily including all of the various components or various steps described in the invention, and some components or some steps may not be included. It should be construed that it may further include additional components or steps.

Also, terms including ordinal numbers such as first, second, etc. used in the present invention may be used to describe the components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

Hereinafter, a preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of the reference numerals, and the redundant description thereof will be omitted.

In addition, in the description of the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings.

The image analysis-based object generating apparatus and digital twin-based simulation apparatus according to an embodiment of the present invention may be in the form of various terminal devices capable of calculation or a server having network functions. When these devices are configured as a server, the server is generally connected to an unspecified number of clients and/or other servers through an open computer network such as the Internet, receives a request to perform a task from a client or other server, and transmits the work result. It refers to a computer system derived and provided and computer software (web server program) installed therefor. However, in addition to the above-described server program, it should be understood as a broad concept including a series of application programs operating on the server and, in some cases, various databases built therein. These servers can be implemented using server programs that are provided in various ways depending on operating systems such as DOS, Windows, Linux, UNIX, and Macintosh on general server hardware. , it may be implemented with one or more servers operating in a cloud manner.

Furthermore, with the development of technology, such an image analysis-based object creation device or digital twin-based simulation device may be configured as a terminal rather than a server. , virtual terminal device, display-integrated computer, POS terminal-based device, PDA (Personal Digital Assistants), MID (Mobile Internet Device), PMP (Portable Multimedia Player), etc. It may display information to the user, and may be various terminals having calculation and storage functions and communication functions.

Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings.

1 is a conceptual diagram showing the configuration of an image analysis system for digital twin technology according to an embodiment of the present invention.

As shown, the image analysis system 10 for digital twin technology includes an image analysis-based object generating device 100 and a digital twin-based simulation device 200 .

The image analysis-based object generating apparatus 100 may basically perform object selection and learning of object posture classification through an image (ie, tagged learning data) in which an object and posture information about each object are secured. A neural network model may be utilized.

On the other hand, when a neural network model is established through such learning, an object can be classified in an image by using the neural network model as a classifier, and posture information about the object can be classified. The posture information is a skeleton including information about joints. It may be a classification result for information and a corresponding posture.

Such an image analysis-based object generating apparatus 100 selects an object from an image collected from an image, learns a posture for the selected object through a neural network model, and classifies a posture for an object using the neural network model obtained after learning Thus, the posture of the object is calculated as posture information including the skeleton information, and then the object information including the identifier of the object, the movement path, and the posture information is stored in the image analysis database 101 .

Meanwhile, the information about the object stored in the image analysis database 101 in this way becomes basic information for generating a virtual human object to be provided to the digital twin-based simulation device 200 in the future.

Therefore, since it is necessary to classify the collected object information according to each situation, the image analysis-based object generating apparatus 100 determines in advance that the date, time, weather, and object appearance of the tracking object may be affected. Classification information is further included in the stored object information according to at least one classification criterion among selected environmental conditions (national holiday, sale period, vacation, school opening, anniversary, event, etc.) and stored as object information in the image analysis database 101 do.

Furthermore, the image analysis-based object generating apparatus 100 generates a virtual object for each classification criterion based on the object information stored in the image analysis database 101 and stores it in the virtual object database 102 , and the digital twin-based simulation apparatus 200 ) can select and provide appropriate impersonation objects upon request. The provided virtual object may be transmitted as simulation object-related format information that can be used as an object in the digital twin-based simulation apparatus 200 .

The digital twin-based simulation device 200 configures the virtual reality model data in the virtual reality environment in conjunction with the simulation target database 201 having virtual reality model data for the simulation target, and the image analysis-based object generating device 100 ) in accordance with the simulation conditions (the date, time, weather, and environmental conditions previously selected as those that can affect the appearance of objects as object classification criteria), and receive virtual object information for each classification with variability accordingly. applied to the virtual reality environment. Here, the virtual object information for each classification having variability means that the virtual objects received from the image analysis-based object generating apparatus 100 are objects having slight variability similar to real objects, rather than objects having the exact same posture and path. As such, the image analysis-based object generating apparatus 100 generates virtual objects according to a distribution according to statistical analysis or similarly simulates real object information and generates not only average object information but also unusual object information according to the distribution. It is also created and provided appropriately. For example, in the case of a virtual object for an airport, there are more objects roaming the waiting area, various nearby shops, information desks, toilets, etc. based on a known movement rather than an object that moves along a known movement. It will also be possible to reflect the unusual movement of an object, such as an employee, or wandering around.

This configuration will be described in more detail with reference to FIG. 2 .

2 is a block diagram showing the configuration of an image analysis-based object generating apparatus 100 according to an embodiment of the present invention.

The image analysis-based object generating apparatus 100 includes an image collecting unit 110 that collects images through an image source 105 such as a camera or an image database, and a region of interest from the images collected by the image collecting unit 110 . An object selector 120 that selects and selects an object existing in a corresponding region of interest, a posture learner 130 that learns a posture of the object selected by the object selector 120 through a neural network model, and posture learning The posture classification unit 140 classifies the posture of the object selected by the object selector 120 through the neural network model to which the weight learned through the unit 130 is applied, and a plurality of objects selected by the object selector 120 . A plurality of objects are continuously tracked at the same time by assigning an identifier to each, and the posture of each object classified by the posture classification unit 140 is linked to the tracking objects, and the classified posture when a cluster, disappearance or occluded object reappears Maintains or creates new object identifiers based on the object identification and tracking unit 150 that stores object information including object identifiers for tracked objects and movement paths and posture information in the image analysis database. .

Here, various neural network models may be applied to the object selection unit 120 and the posture learning unit 130 , and the object selection unit 120 may use a known and weighted classification model, and if necessary, a separate neural network model may be used. You can go through the learning process. The posture learning unit 130 learns the posture using real-time or non-real-time image information (which may be tagged learning information) through the image source 105 , and when the learning is completed, the posture learned by the posture classification unit 140 is A classification model may be applied. In this case, a real-time or non-real-time image is analyzed through the image source 105 and the result is provided to the object identification and tracking unit 150 . Details of such a specific neural network model will be described later with reference to FIGS. 3 to 5 .

On the other hand, the object identification and tracking unit 150 of the image analysis-based object generating apparatus 100 shown is at least one of the date, time, weather, and environmental conditions pre-selected as being capable of affecting the appearance of the object to be tracked. According to the above classification criteria, classification information is further added to the object information stored in the image analysis database 101 and stored.

In addition, the image analysis-based object generating apparatus 100 generates a virtual object for each classification criterion based on the object information stored in the image analysis database 101, but within the classification range of the path and posture of the virtual object for each classification criterion, and a virtual object generator 160 that generates a virtual object having a similar path and posture according to a distribution ratio of real objects with respect to the path and stores the virtual object in the virtual object database 102 .

When virtual objects according to simulation conditions are requested from the digital twin-based simulation device 200, the virtual object generator 160 selects and provides virtual objects for each classification according to the simulation condition from among the stored virtual objects according to the distribution by classification. .

3 to 5 are neural networks applied to the object selection unit 120 , the posture learning unit 130 , and the posture classifying unit 140 according to an embodiment of the present invention.

3 is a conceptual diagram of an image analysis neural network according to an embodiment of the present invention.

The illustrated configuration is for a convolutional neural network (CNN) specialized in image analysis among neural network models, and shows the configuration of a known VGG16 CNN.

As shown in the figure, an image representing the calculation result of each layer of CNN is called a feature map. In the case of the input layer, the feature map becomes a matrix of three RGB channels, which are input images. The feature map of the hidden layer is images calculated by convolution and activation functions, and the pixels in each image may be regarded as features of the feature map. Each feature map is calculated through the following process. Neurons (neurons) or pixels in the feature map undergo a convolution operation with parts of neighboring neurons (receptive field) and a weight matrix obtained by the error backpropagation method. A new feature map is created after various transformations such as /average/L2 pooling and other nonlinear functions (eg, sigmoid, ReLU) calculation processes. For reference, the pooling process can generate more robust features (pixels in the feature map) by summarizing the responses of the receptive field.

Through the continuous convolution and pooling process, the initial feature hierarchy is constructed, which is fine-tuned through Fully Connected Layers (FCN) by various supervised learning methods to derive results according to various computer vision purposes. will do Therefore, the final layer produces conditional probability values for each output neuron through different activation functions. The overall neural network is trained through the process of optimizing objective functions such as mean squared error (MSE) and cross-entropy, and stochastic gradient descent (SGD) in the error backpropagation method. . For example, in the case of VGG16 in Figure 2 above, it consists of a total of 13 convolutional layers, 3 fully connected layers, and finally a softmax classification layer. Specifically, the convolutional feature map is calculated through a 3x3 convolution filter, and the resolution of the feature map is lowered through a 2-stride max pooling layer. Of course, if the size of the input image is different, preprocessing such as size conversion or image cropping is required.

This CNN is effective for image analysis compared to other types of artificial intelligence techniques, so it is mainly used for object identification or scene context analysis.

The object selection unit 120 of the present invention is based on such a CNN, but utilizes an object detection algorithm for a region of interest as shown in FIG. 4 .

4 is a conceptual diagram of a neural network for selecting an object of interest in an image according to an embodiment of the present invention.

In general object detection, the goal is to find the location of objects existing in an image, classify them, and label them with a rectangular bounding box together with the classification reliability (probability). This object detection framework considers (1) the traditional method of classifying object categories for each proposed area after proposing an object area, and (2) considering object detection as a regression analysis and classification problem, and final results such as category classification and location can be classified by a method of deriving them from a single unified framework point of view.

Representative methods based on region proposals include R-CNN, SPP-net, high-speed R-CNN, high-speed R-CNN, R-FCN, FPN, and mask R-CNN. Regression/classification-based methods include MultiBox, AttentionNet, G-CNN, YOLO, SSD, YOLOv2, DSSD, and DSOD. The object selection unit 120 of the present invention uses the R-CNN (Regions with CNN features) method, which is one of the region proposal-based approaches that made a big leap in deep learning-based object detection.

It is necessary to further deepen the structure of deep learning to improve the quality of bounding box candidates and extract high-level features. R-CNN to reach this was proposed by Ross Girshick in 2014, and obtained an average precision (mAP) of 53.3%, which is more than 30% improvement over the previous best result (DPM HSC) in PASCAL VOC 2012. 4 shows a flowchart of R-CNN.

First, R-CNN adopts a selective search to generate about 2,000 region proposals for each image. Selective search methods rely on simple bottom-up grouping and saliency signals to provide more accurate candidates. Through selective search, boxes of any size are quickly detected and the search space is reduced in object detection.

After that, we proceed with CNN-based feature extraction. In this step, each proposed region is converted to a fixed size through warping or truncation, and 4096-dimensional features are extracted as a final representation through the CNN module. Because of CNN's high-level learning ability, expressive power, and hierarchical structure, abstract, semantic, and robust feature-based representations for each proposed area are possible.

Then, classification and localization proceed. Scores are determined for several proposed domains (+: object, -: background) using a linear support vector machine (SVM) for a specific category, pre-trained for several classes. These scored regions are adjusted by bounding box regression analysis to calculate the final bounding box after greedy non-maximum suppression (NMS) is applied.

Through this, it is possible to effectively select desired objects in the image.

5 is a conceptual diagram of a neural network for determining the posture of an object in an image according to an embodiment of the present invention.

An embodiment of the present invention uses a posture tracking framework that tracks the poses of several human objects in an image and assigns a unique instance ID to each keypoint across an image frame. Most of the existing posture tracking research methods are offline methods and focus on the accuracy of object tracking rather than real-time image processing. Therefore, existing methods sequentially perform human object detection, candidate posture estimation, and identification association processes. According to the posture estimation result, the posture tracking result is calculated by solving the optimization problem. In this case, the postures in the next frame must be pre-calculated.

However, in the case of the present invention, the LightTrack method is applied, which is a general and top-down approach, and has the advantage that the posture estimation is performed after candidates for posture estimation are detected, and image processing is possible in real time.

The present invention effectively selects a human object from an image, tracks it, and recognizes the posture, utilizing the fact that the human posture can be used to better infer the human position.

Therefore, in a top-down approach for posture estimation after detecting a posture estimation candidate, the posture of a human object can be easily estimated through an accurate human object position. Therefore, (1) the approximate human object position can be grafted to the postural estimation of a single human object, (2) the human joint position can be roughly used to identify the position of the human object, and (3) iteratively, the estimation process This enables single-person pose tracking (SPT).

Multi-target Pose Tracking (MPT) tracks multiple objects simultaneously and identifies the objects through additional re-identification (Re-ID), rather than simply repeating SPT for several human objects. it is advantageous to do

Accordingly, the object identification and tracking unit 150 according to an embodiment of the present invention uses a method of simultaneously tracking a plurality of objects.

In the case of object tracking, attention should be paid to the following situations: (1) when a human object disappears or is occluded from the camera view, (2) new candidates appear in the camera view, or when candidates that previously disappeared reappear in the view case, (3) when human objects move toward each other and look like one object, (4) when tracking fails due to fast movement or zooming of the camera.

Therefore, if the object is lost in such a situation, if a new object is traced again, it is difficult to secure a continuous movement path for the human object. to be maintained as much as possible.

That is, the detection module for re-introducing object candidates is executed only when object candidates are missed due to occlusion or camera movement, and the object candidates are associated with the tracked targets of the previous frame through the posture matching method.

And in the identification and association process, spatial information consistency and posture consistency information are used. In the re-identification process, matching between a newly observed object and a tracked object candidate is required, which uses a visual feature classifier. In an embodiment of the present invention, graph representation of human object joints is made possible by using a graph convolution network (GCN). The graph expressiveness of this human object skeleton is used in the candidate matching process for posture matching.

Such a posture classification method using the GCN may be applied to the posture classification unit 140 .

The configuration shown in FIG. 4 shows the network structure for the Siamese GCN.

Given the 2D coordinates of the joints of a human object, a spatial graph can be created with the joints as nodes and the connectivity of human body structures as edges. In an embodiment of the present invention, a joint coordinate vector of a graph node is used as an input of the GCN, and a feature expression as a conceptual summary of a human posture is generated through a multi-graph convolution operation.

Through the distance between the network output features, it is determined how similar two postures are, that is, whether the two postures match each other. As the Chamise GCN structure, two feature vectors are extracted from a pair of input graphs of the network with the same weight vectors, so the feature vectors imply the spatial relationship information of human object joints.

This skeleton-based representation not only effectively expresses the similarity of the human object posture, but also has the characteristics of being efficient in the amount of computation and robust to camera movement.

6 is an exemplary diagram illustrating a posture determination result of an object in an image according to an embodiment of the present invention.

As shown, a human being an object of interest in the image is selected and an identifier for each human object is assigned and tracked while simultaneously calculating the skeleton-based posture information of the corresponding object, and based on this, the object's consistency is continuously maintained.

7 is a flowchart illustrating an operation process according to an embodiment of the present invention.

As shown, the present invention selects an object based on R-CNN, estimates a posture for the position of the selected object, and performs a learning process.

After that, when the learning is completed, the object is selected from the image source through R-CNN and the posture of the tracking object is classified while simultaneously tracking the objects in the image. When clustering, disappearance, or occlusion of each object occurs and new object candidates are found, the tracking object is managed while maintaining consistency based on the posture.

The tracking path and posture information for each object are stored together with classification information such as the date, time, weather, and environmental conditions previously selected as those that may affect the appearance of the tracking object.

Thereafter, the collected and accumulated object information is classified to generate virtual objects for each classification. These virtual objects include virtual objects having similar paths and postures according to a distribution ratio of postures and real objects to postures within a classification range of virtual object paths and postures for each classification criterion.

Although not shown, when virtual objects according to the simulation conditions are requested from the digital twin-based simulation device, virtual objects for each classification according to the simulation conditions among the stored virtual objects can be selected and provided according to the distribution for each classification.

Through this embodiment of the present invention, it is possible to provide the motion of virtual human objects close to reality to the digital twin-based simulation, thereby increasing the quality of the digital twin-based simulation.

The various devices and components described herein may be implemented by hardware circuitry (eg, CMOS-based logic circuitry), firmware, software, or a combination thereof. For example, it may be implemented using transistors, logic gates, and electronic circuits in the form of various electrical structures.

Those of ordinary skill in the art to which the present invention pertains may modify and modify the above-described contents without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical spirit of the present invention, but to explain, and the scope of the technical spirit of the present invention is not limited by these embodiments. The protection scope of the present invention should be construed by the following claims, and all technical ideas within the equivalent range should be construed as being included in the scope of the present invention.

100: image analysis-based object generating device 110: image collecting unit
120: object selection unit 130: posture learning unit
140: posture classification unit 150: object identification and tracking unit
160: virtual object generator
200: digital twin-based simulation device

Claims (5)

Objects are selected from images collected through a camera or image database, and the posture of the selected object is learned through a neural network model. Calculating with posture information including: tracking while assigning an identifier to the selected object to obtain an identifier and a movement path of the corresponding object, and object information including the identifier and movement path and posture information for the selected object The data is stored in the image analysis database, but classification information is added to the stored object information according to at least one classification criterion among the date, time, weather, and environmental conditions previously selected as being capable of affecting the appearance of the selected object. In a manner of including and storing, object information including the classification information for each of a plurality of objects selected from images collected through the camera or image database is stored in the image analysis database, and based on the object information stored in the image analysis database an image analysis-based object generating device for generating virtual objects according to classification criteria and storing them in a virtual object database;
A digital application that configures virtual reality model data for a simulation target in a virtual reality environment, requests virtual objects according to simulation conditions from the image analysis-based object generating device, and receives virtual object information by classification accordingly and applies it to the virtual reality environment Includes a twin-based simulation unit;
The image analysis-based object generating apparatus generates a virtual object for each classification criterion based on the object information accumulated and stored in the image analysis database, but the virtual object path for each classification criterion and the actual position for the path within a classification range of the virtual object are generated. After creating virtual objects with similar paths and postures according to the distribution ratio of the objects, they are stored in the virtual object database, and the date, time, weather, and objects that are selected in advance as those that can affect the appearance of objects are selected from the digital twin-based simulation device. Image for digital twin technology, characterized in that when virtual objects according to a simulation condition including at least one of environmental conditions are requested, virtual objects by classification according to the simulation condition from among the stored virtual objects are selected and provided according to distribution by classification analysis system.
The method according to claim 1, The image analysis-based object generating device
an image collecting unit for collecting images through a camera or an image database;
an object selection unit that selects a region of interest from the images collected by the image collection unit and selects objects existing in the region of interest;
a posture learning unit for learning the posture of the object selected by the object selecting unit through a neural network model;
a posture classification unit for classifying the posture of the object selected by the object selector through the neural network model to which the weight learned through the posture learning unit is applied;
By assigning an identifier to each of the plurality of objects selected by the object sorting unit, the plurality of objects are continuously tracked simultaneously, the posture of each object classified in the posture classification unit is linked to the tracking objects, and When re-appearing, an object identification and tracking unit that maintains or creates new object identifiers based on the classified postures and stores object information including object identifiers for tracked objects and movement paths and posture information in the image analysis database Image analysis system for digital twin technology, characterized in that it comprises.
The method according to claim 2, wherein the object identification and tracking unit classification information in the stored object information according to at least one classification criterion among the date, time, weather, and environmental conditions pre-selected as being capable of affecting the appearance of the tracking object An image analysis system for digital twin technology, characterized in that it is further included and stored in the image analysis database.
The method according to claim 3, wherein the virtual object for each classification criterion is created based on the object information stored in the image analysis database, and the path of the virtual object for each classification criterion and the distribution ratio of the real objects to the posture and the path within the classification range of the classification criterion. An image analysis system for digital twin technology, comprising a virtual object generating unit that generates a virtual object having a similar path and posture according to the user and stores the virtual object in a virtual object database.
The method according to claim 4, wherein when virtual objects according to the simulation conditions are requested from the digital twin-based simulation device, the virtual object creation unit selects and provides virtual objects according to the classification according to the simulation conditions from among the stored virtual objects according to the distribution by classification. An image analysis system for digital twin technology.

Images