KR101964282B1

KR101964282B1 - 2d image data generation system using of 3d model, and thereof method

Info

Publication number: KR101964282B1
Application number: KR1020150183663A
Authority: KR
Inventors: 김형관; 김홍조
Original assignee: 연세대학교 산학협력단
Priority date: 2015-12-22
Filing date: 2015-12-22
Publication date: 2019-04-01
Also published as: KR20170074413A

Abstract

The present invention relates to a 2D image learning data generation system using a 3D model and a method of generating the 2D image learning data, and to a data acquisition and processing technology capable of diffusing innovation in the artificial intelligence field into other industries will be. That is, the present invention creates a 3D model of a cognition object and objects needed in a specific industrial field, generates 2D images by rendering the 3D model in various conditions, and learns the generated 2D image by machine learning or big data technology Analysis, and processing, thereby providing a marketable and powerful system and method in the field of image processing and information technology.

Description

TECHNICAL FIELD [0001] The present invention relates to a 2D image data generation system and a method of generating 2D image data using the 3D model,

The present invention relates to a 2D image data generation system and a method thereof, and more particularly, to a system and method for generating 2D image data using a 3D model for learning an image pattern recognition system.

At this time, securing data is an indispensable condition in the age of big data analysis and artificial intelligence technology (especially, study of deep running in machine running especially). Especially, for example, in the field of image information, we are researching and using the latest artificial intelligence technology such as deep running in companies that have secured big data (eg Google, Facebook, etc.).

Visual Big Data is used as training data of artificial intelligence technologies to learn computer system's perception and reasoning ability. At present, these artificial intelligence technologies show performance close to or rather than human ability in areas of speech recognition, image recognition, and character interpretation. Due to its high cognitive performance, the technology is now recognized as having an innovative impact that will result in the replacement of many service workers.

However, there is a problem that it is difficult to apply the latest technology because there is not enough amount of learning data in industries other than the companies or fields that have been able to secure big data in social networks and the like.

Korean Registered Patent No. 10-1556601 (registered on September 23, 2015) Korean Registered Patent No. 10-0511210 (Registered Date: Aug. 23, 2005)

The 2D image learning data generation system and method for generating 2D image data using the 3D model according to the present invention have the following problems.

First, the present invention aims to provide a system and method for generating a large number of 2D image data through a variety of rendering conditions and combinations thereof.

Second, the present invention provides a system and method for efficiently and quickly acquiring 2D image data for learning of an image pattern recognition system required in various industrial fields or educational fields.

The present invention has been made in view of the above problems, and it is an object of the present invention to provide an apparatus and method for controlling the same.

According to a first aspect of the present invention, there is provided a 2D image learning data generation system utilizing a 3D model, comprising: a 3D model generation unit for constructing a 3D image model for a target object to be learned; A rendering condition setting unit that sets at least any one of a virtual photographing condition, a target object condition, and an environmental condition to the generated 3D image model; A 2D image data generation unit that generates a 2D image data by rendering a 3D model according to a set rendering condition; And a data learning unit that learns 2D image data through an image pattern recognition learning process. The imaginary photographing condition of the rendering condition setting unit includes the photographing distance of the camera, the photographing distance is a distance away from the object, and the photographing distance can be changed by changing the distance r from the object to the camera to generate an image of the object have. The object condition of the rendering condition setting unit may include an overlapping condition with a predetermined other object and the overlapping condition may generate an image by overlapping the other object with the object to be recognized.

Here, it is preferable that the 3D model generation unit constructs a 3D image model for a target object by using three-dimensional modeling software, and the shooting conditions include at least one of a shooting distance of the camera, a shooting position, And it is preferable that the photographing distance of the camera is a distance away from the object when photographing.

Preferably, the photographing position of the camera is a three-dimensional camera position about the object at the time of photographing, and the photographing angle of the camera is a throwing angle of the camera with respect to the object located at a specific distance from the object at the time of photographing.

In addition, it is preferable that the camera photographing profile includes at least one of a lens of a camera, a focal length, a color filter, and a photographing effect, and the object condition may be at least one of a color, a texture, It is preferable that the environmental condition is at least one of color, texture, image, climatic environment and noise for the background of the object.

Preferably, the rendering condition setting unit sets the conditions by randomly combining the setting conditions so as to generate an image of a predetermined desired number of acquisitions.

A second aspect of the present invention relates to a method of generating 2D image learning data using a 3D model. The 2D image learning data generation method includes the steps of (a) generating a 3D image model ; (b) setting a condition of at least one of a virtual photographing condition, a target object condition, and an environmental condition of the 3D image model in which the rendering condition setting unit is created; (c) rendering 2D image data by rendering the 3D model according to a rendering condition set by the 2D image data generating unit; And (d) learning the generated 2D image data through an image pattern recognition learning process. The imaginary photographing condition of the rendering condition setting unit includes the photographing distance of the camera, the photographing distance is a distance away from the object, and the photographing distance can be changed by changing the distance r from the object to the camera to generate an image of the object have. The object condition of the rendering condition setting unit may include an overlapping condition with a predetermined other object and the overlapping condition may generate an image by overlapping the other object with the object to be recognized.

The step (c) includes the steps of: (c1) generating 2D image data by rendering the 3D model according to a rendering condition set by the 2D image data generation unit; And (c2) converting the 2D image data generated by the 2D image data generator into the big data by using the file distribution system, the parallel data processing technique, and the big table technique.

In the step (c), the rendering condition setting unit randomly combines the rendering conditions set to generate the image of the desired number of acquisitions, and generates the 2D image big data by rendering the 3D model according to the set conditions Preferably, the step (d) is a step of learning the generated 2D image data using a machine learning technique.

A third aspect of the present invention is a computer program stored in a medium for executing a 2D image learning data generation method using the 3D model in combination with hardware.

The 2D image learning data generation system and method for generating 2D image data using the 3D model according to the present invention have the following effects.

First, the present invention creates a 3D model of a cognitive object and objects needed in a specific industrial field, generates 2D image data by adding various effects realistically to the 3D model, and produces an effect like real big data Thereby providing a system and method that can be secured.

Second, the present invention generates numerous data through various rendering conditions and combinations of 3D models, and uses the generated data to learn a machine learning technique. Based on the learned contents of artificial intelligence, The present invention provides a system and method for efficiently and quickly processing and acquiring necessary data in various industrial fields or educational fields.

Third, the present invention provides a technology that has the potential to spread innovation in the artificial intelligence field to other industrial fields that have not secured video big data.

Fourth, the present invention provides a video processing and data securing technique having a great deal of marketability and power in the field of image information processing.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

FIG. 1 is a block diagram of a 2D image learning data generation system utilizing a 3D model according to an embodiment of the present invention.
2 is a flowchart illustrating a method of generating a 2D image learning data using the system of FIG.
3 is a schematic diagram illustrating various examples of rendering conditions in a 2D image learning data generation system and method according to an embodiment of the present invention.
4 is a schematic diagram illustrating an example of generating 2D image learning data of a 3D model according to an embodiment of the present invention.

Further objects, features and advantages of the present invention will become more apparent from the following detailed description and the accompanying drawings.

Before describing the present invention in detail, it is to be understood that the present invention is capable of various modifications and various embodiments, and the examples described below and illustrated in the drawings are intended to limit the invention to specific embodiments It is to be understood that the invention includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, the terms "comprises" or "having" and the like refer to the presence of stated features, integers, steps, operations, elements, components, or combinations thereof, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Further, terms such as " part, "" unit," " module, "and the like described in the specification may mean a unit for processing at least one function or operation.

In the following description of the present invention with reference to the accompanying drawings, the same components are denoted by the same reference numerals regardless of the reference numerals, and redundant explanations thereof will be omitted. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram of a 2D image learning data generation system utilizing a 3D model according to an embodiment of the present invention. FIG. 2 is a flowchart illustrating a 2D image learning data generation method using the system of FIG. 1 .

As shown in FIG. 1, a 2D image learning data generation system according to an embodiment of the present invention includes a 3D model generation unit 100 for constructing a 3D image model for a target object to be learned; A rendering condition setting unit (200) for setting a condition of at least one of a virtual photographing condition, a target object condition, and an environmental condition; A 2D image data generation unit 300 for rendering a 3D model according to a set rendering condition to generate 2D image data; And a data learning unit 400 for learning the generated 2D image data through an image pattern recognition learning process.

2, the 2D image learning data generating method according to the embodiment of the present invention includes the steps of (a) constructing a 3D image model of a target object to be learned, (b) (C) generating the 2D image data by rendering the 3D model according to the set rendering conditions, and (d) setting the condition of at least one of the virtual shooting condition, the object condition, ), And the artificial intelligence learns 2D image data using a machine learning technique such as deep learning of the generated 2D image data.

As described above, the 2D image learning data generation system and method using the 3D model according to the embodiment of the present invention can be applied to data acquisition and processing capable of diffusing innovation in the artificial intelligence field into other industrial fields, As a technology, it is possible to create objects and objects that are needed in specific industrial fields as 3D models, render them in various conditions to generate 2D images, and learn the generated 2D images by machine learning or big data technology And provides a marketable and powerful system and method in the fields of image processing and information technology.

Here, the 3D model generation unit 100 is a device for generating a specific object or object in a 3D model, and is a 3D modeling apparatus or system for generating a 3D model of an object using 3D modeling software.

The 3D modeling device is used mainly in the aspect of the automatic construction of the computer in the early architectural field, which reduces the time and effort required for the drawing in the design work by using the CAD system, Because it was perceived to be more appropriate for the automation of drawing creation than as a tool. However, the development of computers and programs has led to the introduction of high - end CAD systems and the recognition of design using CAD systems has become a useful tool for analyzing and evaluating alternatives in design process as well as automatic drawings. A 3D modeling system for representing and manipulating a 3D shape has been developed and used.

Therefore, in the embodiment of the present invention, since the 3D model of the object is converted into a large amount of 2D image data and the data is used as 2D image learning data of the machine learning technique, The user can easily and quickly perform a 3D modeling operation and acquire a desired object using the system.

The 3D modeling operation can be performed by forming a design work directly by a user through the 3D modeling system to generate a 3D model or using an existing 3D model, or a method of acquiring the 3D model automatically by a 3D scanning device. Further, the 3D model is a model of a three-dimensional shape similar to a prototype but morphologically the same or similar to the same object having the same texture or color similar to that of an actual object.

The rendering condition setting unit 200 is a device for setting a condition for converting the 3D model generated before rendering by the rendering apparatus into 2D image data. Rendering refers to the process of creating realistic 3-D images, taking into account shadows, colors, and densities, which are generally different from 2D information, such as shape, position, It means a computer graphic process of adding a sense of realism by giving a shadow or a change in density to a visible object by adding a sense of dimensionality. However, in the embodiment of the present invention, as a reverse process, a process of rendering a 2D image through a 3D model do. That is, the rendering condition setting unit 200 sets various external conditions such as a shooting condition, a target condition, and an environmental condition before converting or generating the 3D model to 2D image data.

The 2D image data generation unit 300 is a rendering device that generates 2D image data according to conditions set in the rendering condition setting unit 200. [ That is, the apparatus acquires 2D image data, which is a result of rendering various conditions on a camera, setting various conditions on the 3D model, rendering various conditions on the background of the 3D model, and rendering the created 3D model.

The data learning unit 400 performs a process of learning data generated by the 2D image data generation unit 300 by using a big data technique or a machine learning technique, It is a device that learns artificial intelligence by using it, and it is processed by big data technology to secure the recognition and analysis system that user needs.

Here, the "big data" refers to a large-scale data that includes a large size, a short generation cycle, and a shape as well as numerical data, as well as text and image data, compared with data generated in the past analog environment. A device for analyzing and processing such big data is called a big data server or a big data device. A big data server is a computing device capable of network communication or a group thereof, and a Hadoop cluster group capable of distributing and parallelizing a large amount of data . In other words, it is composed of a large number of NoSQL databases, a cluster of Hadoop format composed of a master node and a plurality of slave nodes, and a big data server is equipped with an Elastics search search engine, Can be strengthened.

The 2D image data generation unit 300 applied to the embodiment of the present invention is different from the general big data server in that it analyzes and processes a large number of data using network communication, As an apparatus for analyzing and processing a large number of big data in a distributed and parallel processing format, a configuration of a conventional big data system can be used.

As described above, in the embodiment of the present invention, rather than collecting and analyzing and processing a large number of data on a network as in the conventional big data analysis system, it is possible to set various rendering conditions through at least one 3D model, We propose a system and method that can take advantage of big data technology by analyzing the generated data and the big data system.

The data learning unit 400 is a learning process for recognizing a large number of data generated by the 2D image data generation unit 300 and a technique for learning artificial intelligence using a machine learning technique To provide optimized data and information to the user.

Here, machine learning is a set of algorithms that try to combine high-level abstractions (a task that summarizes core content or functions in large amounts of data or complex data) through a combination of several nonlinear transformation techniques. And is a field of machine learning that teaches computers how people think in a large frame.

In other words, deep learning or machine learning is an artificial intelligence (AI) technology that allows a computer to think and learn like a human being. The human brain finds patterns in a lot of data, To imitate the information processing method of the computer to learn the machine to learn things. In addition, machine learning technology enables a computer to recognize, deduce, and judge itself without having to set all judgment criteria, so that it is widely used for voice and image recognition and photo analysis.

Therefore, in the embodiment of the present invention, the 2D image data generation unit 300 generates a large number of data through a variety of rendering conditions and a combination of the 3D models, and based on the generated data, Thus, it is possible to provide a system and method for efficiently and quickly acquiring necessary data in various industrial fields or educational fields.

The 3D model generating unit 100, the rendering condition setting unit 200, the 2D image data generating unit 300, and the data generating unit 300 may be configured as a 2D image learning data generating system utilizing the 3D model according to the above- The learning unit 400 may be configured not only as a single computing device or system, but also may be configured such that each component is connected to an independent computing device or system.

FIG. 3 is a schematic diagram illustrating various examples of rendering conditions in a 2D image learning data generation system and method according to an embodiment of the present invention. FIG. 4 is a diagram illustrating an example of generating 2D image learning data using a 3D model according to an embodiment of the present invention. Fig.

3 and 4, the rendering condition setting unit 200, which is applied to the embodiment of the present invention, sets various rendering conditions to select an appropriate image generation method according to the characteristics in which the user recognition object exists, To generate 2D images.

Hereinafter, a case where the object to be recognized is a cube is exemplified.

1. Shooting conditions (methods of different shooting point and shooting position) (see FIG. 3)

The photographing condition is one of the rendering conditions, and includes the photographing position, the photographing angle and the camera photographing profile of the camera, as illustrated below. In addition to this, it is possible to use any shooting condition in which a virtual camera captures an object to generate a 2D image.

1) A method of generating an image while fixing the distance r (center of the sphere) to the camera from the object and rotating the camera with respect to the horizontal plane (specifying the rotation angle when the object is moved once between 0 and 180 degrees)

2) A method of generating an image while holding the camera and distance r (center of the sphere) from the object and rotating the camera with respect to the vertical plane (specifying the rotation angle when the object is moved once between 0 and 180 degrees)

3) Fixing the distance r (the center of the sphere) to the camera and limiting the range of movement of the horizontal or vertical plane from the object

4) A method of generating an image of an object by changing the distance r from the object to the camera

5) When the distance r from the object to the camera is fixed, a method of generating the image of the object by tilting the camera left and right and up and down

6) Artificial creation of image periphery or center distortion effect according to camera lens type

7) Artificial creation of perimeter or center distortion effect of object in image according to camera lens type

8) Artificially giving the focusing effect that appears when changing the focal length of the camera to give image clarity and blur effect

9) How to change the color of image by applying various profiles of camera

2. Object Condition (object condition)

An object is a condition for changing the state of an object or object to be studied, and it can set various colors such as a color, a texture, a shape of the object and a superposition condition with a specific object. As illustrated below.

1) To superimpose objects that have the shape of a specific object (smoke, circular band, barbed wire, stick, square, circle, person, car, tree, branch, How to create

2) A method of artificially generating an image by transforming the object shape in the image to a certain level. When a straight line is a curved line, as in the case of an ellipse of a circular shape, a method of irregularly distorting the boundary line

3) How to create an image by applying a shadow effect to the object

3. Environmental conditions

The environmental condition is a condition that changes the surrounding environment such as the background of the object to be studied, and the condition can be set by various factors such as background image, color, and climatic condition. As illustrated below.

1) A method of generating an image by converting a specific color region (RGV, HSV, HIS, Lab, Luv, black and white, grayscale) Or a method of generating an image using only a few color spaces in a specific color region

2) A method of artificially synthesizing the background behind an object to be recognized: a single color background, a natural background (sea, river, meadow, wilderness, sky, city, house, road,

3) A method of generating images by applying artificial noise (dust, Gaussian random noise).

4) A method of artificially setting weather conditions such as rain, snow, cloudy, and clear, and generating images with different light source effects.

FIG. 4 is a schematic diagram for generating a 3D model as 2D image learning data through the above-described rendering conditions. A 3D model of a hanok is constructed and various rendering conditions (photographing conditions, object conditions, environmental conditions, And the generated 2D image data is learned by a machine learning technique to illustrate that 2D image learning data desired by the user is generated.

As described above, the present invention creates a large number of 2D image data by setting various rendering conditions described above in the 3D model or by combining these conditions. If the number of images desired by the user is set in advance in the rendering condition setting unit 200, 2D image data in which the conditions are randomly combined can be generated. In addition, the rendering condition can be set to a large number of conditions desired by the user, so that various effects can be selected to generate big data of a 2D image.

As described above, the present invention can be applied to a real-time image processing apparatus or a real-time image processing apparatus using a machine learning technique such as using 2D images acquired from a 3D model as training data in a learning process of an image pattern recognition system, The present invention provides a system and method that can be used to detect (identify, classify, approximate, clustering) an object or environment in an image.

The embodiments and the accompanying drawings described in the present specification are merely illustrative of some of the technical ideas included in the present invention. Accordingly, the embodiments disclosed herein are for the purpose of describing rather than limiting the technical spirit of the present invention, and it is apparent that the scope of the technical idea of the present invention is not limited by these embodiments. It will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

100: 3D model generation unit 200: rendering condition setting unit
300: 2D image data generation unit 400:

Claims

A 3D model generation unit for constructing a 3D image model for a target object to be learned;
A rendering condition setting unit that sets at least any one of a virtual photographing condition, a target object condition, and an environmental condition to the generated 3D image model;
A 2D image data generation unit that generates a 2D image data by rendering a 3D model according to a set rendering condition; And
And a data learning unit that learns 2D image data through an image pattern recognition learning process,
Wherein the imaginary photographing condition of the rendering condition setting unit includes a photographing distance of the camera and the photographing distance is a distance apart from the object when photographing from the object and the photographing distance is a distance from the object to the camera,
Wherein the object condition of the rendering condition setting unit includes an overlapping condition with a predetermined other object and the overlapping condition generates an image by overlapping the other object with the object to be recognized, Learning data generation system.

The method according to claim 1,
Wherein the 3D model generating unit comprises:
And a 3D image model for the object is constructed using the 3D modeling software.

The method according to claim 1,
The photographing condition is,
And a condition of at least one of a photographing position of the camera, a photographing angle, and a photographing profile of the camera.

delete

The method of claim 3,
The photographing position of the camera,
Wherein the camera position is a three-dimensional camera position around the object at the time of photographing.

The method of claim 3,
The photographing angle of the camera,
Wherein the projection angle is a projection angle for a target object of the camera located at a specific distance from the object at the time of photographing.

The method of claim 3,
The camera-
A camera lens, a focal length, a color filter, and a photographing effect.

The method according to claim 1,
Wherein the object condition further includes at least one of a color, a texture, and a shape of a target object.

The method of claim 3,
The environmental conditions include,
Wherein the 3D model is at least one of a color, a texture, an image, a climatic environment, and noise of a background of a target object.

The method according to claim 1,
The rendering condition setting unit,
And a condition is set by randomly combining setting conditions to generate an image of a desired number of acquisitions in advance.

Claims 1. A 2D image learning data generation system comprising:
(a) constructing a 3D image model for a target object to be learned by the D model generating unit;
(b) setting a condition of at least one of a virtual photographing condition, a target object condition, and an environmental condition of the 3D image model in which the rendering condition setting unit is created;
(c) rendering 2D image data by rendering the 3D model according to a rendering condition set by the 2D image data generating unit; And
(d) learning the generated 2D image data through an image pattern recognition learning process,
Wherein the imaginary photographing condition of the rendering condition setting unit includes a photographing distance of the camera and the photographing distance is a distance apart from the object when photographing from the object and the photographing distance is a distance from the object to the camera,
Wherein the object condition of the rendering condition setting unit includes an overlapping condition with a predetermined other object and the overlapping condition generates an image by overlapping the other object with the object to be recognized, Learning data generation method.

The method of claim 11,
The step (c)
(c1) generating 2D image data by rendering the 3D model according to a rendering condition set by the 2D image data generation unit; And
(c2) converting the 2D image data generated by the 2D image data generator into the big data using the file distribution system, the parallel data processing technique, and the big table technique. Data generation method.

The method of claim 11,
The step (c)
Wherein the rendering condition setting unit randomly combines rendering conditions set to generate an image of a desired number of acquisitions in advance and generates 2D image big data by rendering the 3D model according to the set conditions. A method for generating image learning data.

The method of claim 11,
The step (d)
And learning the generated 2D image data using a machine learning technique.

11. A computer program stored in a medium for executing a method of generating a 2D image learning data utilizing the 3D model of claim 11 in combination with hardware.