KR102252796B1 - Apparatus for providing three dimensional model of plant - Google Patents

Abstract

An apparatus for providing a three-dimensional model of a plant capable of interacting with a user includes a receiving unit for receiving plant data for a plurality of plants from a server, a plant selection unit for selecting one plant from among plant data for the plurality of plants, the A three-dimensional model display unit for displaying a three-dimensional model of the selected plant and a parameter input unit for receiving at least one parameter related to the selected plant, wherein the three-dimensional model display unit displays a three-dimensional model of the plant based on the received parameter. .

Description

A device that provides a three-dimensional model of plants {APPARATUS FOR PROVIDING THREE DIMENSIONAL MODEL OF PLANT}

The present invention relates to an apparatus for providing a three-dimensional model of a plant.

With the development of communication and imaging technology, the field of immersive media is striving to utilize free-view images in a variety of ways. For example, in the field of sports broadcasting, a multi-view image is acquired by installing a plurality of cameras in a stadium, and a three-dimensional model of a player created by using this is provided. Through this, viewers can watch the game from various angles, and can view more realistic and immersive images.

However, there is insufficient effort to apply and utilize free-view video technology to educational contents. In producing educational content, it is important whether or not interaction with the user is made smoothly.

Korean Patent Publication No. 2019-0074911 discloses a configuration of acquiring image data taken from a plurality of cameras radially installed around a dynamic subject, and generating 3D modeling data according to each frame of the subject.

We want to provide a three-dimensional model that can interact with users.

It is intended to provide a device and a server that implements a three-dimensional model of a plant as a free viewpoint.

An object is to provide an apparatus capable of observing a three-dimensional model of a plant according to the control of the growth process and parameters of the plant.

However, the technical problem to be achieved by the present embodiment is not limited to the technical problems as described above, and other technical problems may exist.

As a means for achieving the above-described technical problem, an embodiment of the present invention is an apparatus for providing a three-dimensional model of a plant capable of interacting with a user, the receiving unit receiving plant data for a plurality of plants from a server, A plant selection unit for selecting one plant from among plant data for the plurality of plants, a three-dimensional model display unit for displaying a three-dimensional model of the selected plant, and a parameter input unit for receiving at least one parameter related to the selected plant, The three-dimensional model display unit may provide a three-dimensional model providing apparatus that displays a three-dimensional model of the plant based on the received parameter.

In one embodiment, the apparatus for providing a three-dimensional model includes a multi-view image acquisition unit that acquires a multi-view image of a plant photographed using a plurality of cameras, and each of the plurality of cameras It may further include a camera posture estimating unit for estimating a posture of and a three-dimensional model generating unit for generating a three-dimensional model of the plant from the multi-view image based on the estimated postures of the plurality of cameras.

In one embodiment, the camera posture estimating unit, a distortion correction unit that corrects distortion of the multi-view image based on the positions and rotation angles of each of the plurality of cameras, and feature point matching that extracts and matches a feature point from the multi-view image It may include a camera posture derivation unit for deriving the camera posture of each of the plurality of cameras estimated based on the unit and the matched feature points.

In an embodiment, the three-dimensional model generator may generate a three-dimensional model according to a growth process of the plurality of plants and adjustment of the parameters.

In an embodiment, the parameter may include at least one of time, water, temperature, humidity, illuminance, soil, pests, climate, and whether or not to treat drugs.

The above-described problem solving means are merely exemplary and should not be construed as limiting the present invention. In addition to the above-described exemplary embodiments, there may be additional embodiments described in the drawings and detailed description of the invention.

According to any one of the above-described problem solving means of the present invention, a three-dimensional model of a plant capable of interacting with a user can be provided as a free viewpoint.

In addition, it is possible to provide a three-dimensional model of plants having various shapes according to the control of the plant growth process and parameters.

In addition, it is possible to reduce the cost and time required to generate and provide a three-dimensional model of a plant.

In addition, a three-dimensional model of a plant can be used to produce contents related to education, gardening, hobbies, and games.

1 is a block diagram of a system for providing a three-dimensional model according to an embodiment of the present invention.
2 is a block diagram of an apparatus for providing a three-dimensional model according to an embodiment of the present invention.
3 exemplarily shows a three-dimensional model according to the control of the growth process and parameters for plants.
4 is a block diagram of a server providing a three-dimensional model according to another embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" with another part, this includes not only "directly connected" but also "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, it means that other components may be further included, and one or more other features, not excluding other components, unless specifically stated to the contrary. It is to be understood that it does not preclude the presence or addition of any number, step, action, component, part, or combination thereof.

In the present specification, the term "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized by using two or more hardware, or two or more units may be realized by one piece of hardware. Meanwhile,'~ unit' is not meant to be limited to software or hardware, and'~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example,'~ unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , Subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays and variables. Components and functions provided in the'~ units' may be combined into a smaller number of elements and'~ units', or may be further separated into additional elements and'~ units'. In addition, components and'~ units' may be implemented to play one or more CPUs in a device or a security multimedia card.

The "user terminal" mentioned below may be implemented as a computer or portable terminal that can access a server or other terminal through a network. Here, the computer includes, for example, a notebook equipped with a web browser, a desktop, a laptop, and the like, and the portable terminal is, for example, a wireless communication device that guarantees portability and mobility. , Smart phones, tablet PCs, wearable devices, as well as communication modules such as Bluetooth (BLE, Bluetooth Low Energy), NFC, RFID, Ultrasonic, infrared, WiFi, and LiFi. It may include various mounted devices. In addition, “network” refers to a connection structure that enables information exchange between nodes such as terminals and servers, and includes a local area network (LAN), a wide area network (WAN), and the Internet. (WWW: World Wide Web), wired and wireless data networks, telephone networks, wired and wireless television networks, etc. Examples of wireless data communication networks include 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Bluetooth communication, infrared communication, and ultrasound. Communication, Visible Light Communication (VLC), LiFi, and the like are included, but are not limited thereto.

Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a system for providing a three-dimensional model according to an embodiment of the present invention. Referring to FIG. 1, a system for providing a three-dimensional model may include a server 110 and a device 120.

The server 110 and the device 120 may transmit or receive data through a network. The server 110 may collect and store plant data 111 for a plurality of plants. The device 120 may provide a three-dimensional model of a plant that can interact with a user. Specific operations performed by the server 110 and the device 120 will be described below.

2 is a block diagram of an apparatus for providing a three-dimensional model according to an embodiment of the present invention. Referring to FIG. 2, the apparatus 120 may include a receiving unit 121, a plant selection unit 122, a three-dimensional model display unit 123, and a parameter input unit 124.

The receiving unit 121 may receive plant data 111 for a plurality of plants from the server 110. The plant data 111 for a plurality of plants may include, for example, an image for each of the plurality of plants, as shown in FIG. 1. The plant data 111 for a plurality of plants may include, for example, one or more information on a name, classification, habitat, use, growth process, etc. for each of the plurality of plants.

The plant selection unit 122 may receive a selection of one plant from among plant data for a plurality of plants. The plant selection unit 122 may display plant data for a plurality of plants using the display of the device 120 and may select one plant from among them. The plant selection unit 122 may display an image for each of a plurality of plants using, for example, a display, and may select one plant from among them.

The three-dimensional model display unit 123 may display a three-dimensional model of a selected plant. The three-dimensional model display unit 123 may display a three-dimensional model of a plant selected by the plant selection unit 122 as a free viewpoint. That is, when the three-dimensional model display unit 123 displays a three-dimensional model of a plant, a reference point of view is not limited.

The three-dimensional model display unit 123 may display a three-dimensional model that is freely rotated according to a direction operated by a user. In addition, the three-dimensional model display unit 123 may display a three-dimensional model whose size is enlarged or reduced according to a user's manipulation. In addition, the three-dimensional model display unit 123 may display a three-dimensional model whose shape is changed according to the passage of time or by input of a parameter to be described later.

The parameter input unit 124 may receive at least one parameter related to the selected plant. The parameter may be a factor affecting the growth of the plant. The parameter may include, for example, at least one of time, water, temperature, humidity, illuminance, soil, pests, climate, and whether or not to treat drugs, but is not limited thereto.

For example, with respect to the three-dimensional model of a plant, a parameter for adjusting time may be input so that the speed at which time flows is faster or slower than the actual speed at which time flows. For example, with respect to the three-dimensional model of a plant, the time may be adjusted so that time flows in the forward or reverse direction, or parameters may be input to display the three-dimensional model of the plant at a specific point in time.

For example, parameters for controlling a period of virtually supplying water to a three-dimensional model of a plant, an amount supplied, and the like may be input.

For example, parameters that control the conditions related to the environment of the virtual space where the three-dimensional model of plants exists, such as temperature, humidity, illumination, soil condition and composition, whether or not diseases and pests occur, climate change, and whether or not to treat drugs can be received. have. A parameter for adjusting the conditions of the environment of the virtual space in which the three-dimensional model of the plant exists to be constant, periodically changes, or to generate a specific event may be input.

Referring back to FIG. 2, the apparatus 120 may further include a multi-view image acquisition unit 125, a camera posture estimation unit 126, and a three-dimensional model generation unit 127.

The multi-view image acquisition unit 125 may acquire a multi-view image of a plant photographed using a plurality of cameras. The multi-view image may be obtained by photographing after installing a plurality of cameras around a plant to be photographed. The plurality of cameras may be installed at intervals of 25 degrees or less centering on the plant to be photographed. The plurality of cameras may be installed to form a circumference around a plant to be photographed. Alternatively, a plurality of cameras may be installed while forming a certain pattern. The multi-view image may be obtained using a total of 14 to 18 cameras, and the total number of cameras used may be adjusted in consideration of the growth stage and size of the plant to be photographed.

The multi-view image acquisition unit 125 may acquire a multi-view image photographed at a certain period according to the growth process of the plant. For example, the multi-view image may be a photograph of a plant at a daily (24 hours) cycle.

Alternatively, the multi-view image acquisition unit 125 may acquire a multi-view image captured at short time intervals while moving the position of the camera around the plant using one camera provided in the device 120. The multi-view image may be obtained by photographing at a constant cycle according to a plant growth process using a camera provided in the device 120.

The camera posture estimating unit 126 may estimate the posture of each of the plurality of cameras based on the positions and rotation angles of each of the plurality of cameras. Referring to FIG. 2, the camera posture estimating unit 126 may include a distortion correcting unit 126a, a feature point matching unit 126b, and a camera posture deriving unit 126c.

The distortion correction unit 126a may correct distortion of a multiview image based on the positions and rotation angles of each of the plurality of cameras. The distortion correction unit 126a may estimate parameters representing positions and rotation angles of each of the plurality of cameras using the check board. The distortion correction unit 126a may correct distortion of the multiview image based on the estimated parameter. The distortion correction unit 126a may correct distortion of a multiview image using camera calibration.

The feature point matching unit 126b may extract and match feature points from a multi-view image. The feature point matching unit 126b may determine a reference image as the first image from the multiview image for which distortion correction has been performed. For example, when a multi-view image is taken using N cameras, the multi-view image is a second image, a third image, ... , And the Nth image.

The feature point matching unit 126b may extract feature points from all images of the multi-view image and match the feature points extracted from each image. The feature point matching unit 126b may match feature points of all images based on the coordinate system of the first image.

For example, the feature point matching unit 126b may compare the first image and the second image to match the feature points between the two images. Thereafter, the feature point matching unit 126b may compare the second image and the third image to match the feature points between the two images. In the same way, the feature point matching unit 126b may match feature points of all images of a multi-view image.

The camera posture derivation unit 126c may derive the camera posture of each of the plurality of cameras estimated based on the matched feature points. For example, the camera posture derivation unit 126c may derive the posture of the camera that has captured the second image based on the matched feature points between the first image and the second image. After that, the camera posture derivation unit 126c may derive the posture of the camera that has captured the third image based on the matched feature points between the second image and the third image. In the same way, the camera posture derivation unit 126c may derive the posture of the camera that has captured all the images of the multiview image.

The camera posture estimating unit 126 may correct the derived camera posture of each of the plurality of cameras using 3D coordinates and a bundle adjustment algorithm. The camera posture estimating unit 126 may accurately estimate the camera posture of each of the plurality of cameras through matching and correction of feature points.

The three-dimensional model generator 127 may generate a three-dimensional model of a plant from a multi-view image based on the estimated postures of each of the plurality of cameras. For example, the 3D model generator 127 may reconstruct 3D coordinates based on the estimated postures of each of the plurality of cameras. The three-dimensional model generation unit 127 may reconstruct the surface of the three-dimensional model based on the reconstructed three-dimensional coordinates. The three-dimensional model generator 127 may map a texture of a three-dimensional model based on color information of a multi-view image.

The three-dimensional model generation unit 127 may generate a three-dimensional model according to a growth process for a plurality of plants and adjustment of parameters. For example, the three-dimensional model generation unit 127 may generate a three-dimensional model for each of a plurality of plants whose shape is changed according to a growth process. For example, the three-dimensional model generation unit 127 may generate a three-dimensional model having a different shape by adjusting parameters for each of a plurality of plants. The parameter may be a factor affecting the growth of the plant. The parameter may include, for example, at least one of time, water, temperature, humidity, illuminance, soil, pests, climate, and whether or not to treat drugs, but is not limited thereto.

In addition, the three-dimensional model generation unit 127 may receive parameters related to the environment around the plant using a sensor installed around the plant. The sensor may include, for example, one or more of a temperature sensor, a humidity sensor, and an illuminance sensor, but is not limited thereto. The three-dimensional model generator 127 may generate a three-dimensional model of a plant based on the received parameter values and a multi-view image of the plant.

3 exemplarily shows a three-dimensional model according to the control of the growth process and parameters for plants. Referring to FIG. 3, the device 120 may provide a three-dimensional model 303 of a plant according to a growth process 301 of the plant as a free viewpoint. In addition, the device 120 may provide a three-dimensional model 303 of a plant according to the adjustment 302 of one or more parameters as a free viewpoint.

Banana trees, for example, are plants in tropical climates that require direct sunlight, high temperatures and humidity. When a parameter for adjusting the temperature to be lowered is input to the three-dimensional model of the banana tree, the three-dimensional model of the banana tree may be displayed in a withered form. Alternatively, when parameters for maintaining suitable temperature and humidity are input for the three-dimensional model of a banana tree, a three-dimensional model in which the banana tree grows well may be displayed.

For another example, cactus is a plant that is not affected by humidity and requires a lot of light. When a parameter for adjusting the humidity to be lowered or increased for the three-dimensional model of the cactus is input, it may be displayed that there is no change in the three-dimensional model of the cactus. When a parameter for maintaining an appropriate illuminance for a three-dimensional model of a cactus is input, a three-dimensional model in which the cactus grows well may be displayed.

The device 120 may transmit a three-dimensional model according to a growth process and parameter adjustment for a plurality of plants generated by the three-dimensional model generator 127 to the server 110.

4 is a block diagram of a server providing a three-dimensional model according to another embodiment of the present invention. 4, the server 110 may include a receiving unit 401, a multi-view image acquisition unit 402, a camera posture estimation unit 403, a three-dimensional model generation unit 404, and a transmission unit 405. have.

The server 110 may provide a three-dimensional model of a plant that can interact with a user. The server 110 may transmit and receive data to and from the device 120 or another server through a network. The server 110 may collect and store plant data for a plurality of plants.

The receiving unit 401 may receive plant data for a plurality of plants from the device 120 or another server. Plant data for a plurality of plants may include, for example, an image for each of the plurality of plants. Plant data for a plurality of plants may include, for example, one or more information on a name, classification, habitat, use, growth process, etc. for each of the plurality of plants.

The receiving unit 401 may receive a three-dimensional model of a plant from the device 120 or another server. The receiving unit 401 may receive a three-dimensional model according to a growth process for a plurality of plants and adjustment of parameters.

The multi-view image acquisition unit 402 may acquire a multi-view image of a plant photographed using a plurality of cameras. The multi-view image may be obtained by photographing after installing a plurality of cameras around a plant to be photographed. The plurality of cameras may be installed at intervals of 25 degrees or less centering on the plant to be photographed. The plurality of cameras may be installed to form a circumference around a plant to be photographed. Alternatively, a plurality of cameras may be installed while forming a certain pattern. The multi-view image may be obtained using a total of 14 to 18 cameras, and the total number of cameras used may be adjusted in consideration of the growth stage and size of the plant to be photographed.

The multi-view image acquisition unit 402 may acquire a multi-view image photographed at a constant cycle according to a plant growth process. For example, the multi-view image may be a photograph of a plant at a daily (24 hours) cycle.

Alternatively, the multi-view image acquisition unit 402 may acquire a multi-view image captured at short time intervals while moving the position of the camera around the plant using one camera provided in the device 120. The multi-view image may be obtained by photographing at a constant cycle according to a plant growth process using a camera provided in the device 120.

The camera posture estimating unit 403 may estimate the posture of each of the plurality of cameras based on the positions and rotation angles of each of the plurality of cameras. Referring to FIG. 4, the camera posture estimating unit 403 may include a distortion correcting unit 403a, a feature point matching unit 403b, and a camera posture deriving unit 403c.

The distortion correction unit 403a may correct distortion of the multiview image based on the positions and rotation angles of each of the plurality of cameras. The distortion correction unit 403a may estimate parameters representing positions and rotation angles of each of the plurality of cameras using the check board. The distortion correction unit 403a may correct distortion of the multiview image based on the estimated parameter. The distortion correction unit 403a may correct distortion of a multiview image using camera calibration.

The feature point matching unit 403b may extract and match feature points from a multi-view image. The feature point matching unit 403b may determine a reference image as the first image from the multi-view image for which distortion correction has been performed. For example, when a multi-view image is taken using N cameras, the multi-view image is a second image, a third image, ... , And the Nth image.

The feature point matching unit 403b may extract feature points from all images of the multi-view image and match the feature points extracted from each image. The feature point matching unit 403b may match feature points of all images based on the coordinate system of the first image.

For example, the feature point matching unit 403b may compare the first image and the second image to match the feature points between the two images. Thereafter, the feature point matching unit 403b may compare the second image and the third image to match the feature points between the two images. In the same way, the feature point matching unit 403b may match feature points of all images of a multi-view image.

The camera posture derivation unit 403c may derive the camera posture of each of the plurality of cameras estimated based on the matched feature points. For example, the camera posture derivation unit 403c may derive the posture of the camera that has captured the second image based on the matched feature points between the first image and the second image. After that, the camera posture derivation unit 403c may derive the posture of the camera that has captured the third image based on the matched feature points between the second image and the third image. In the same way, the camera posture derivation unit 403c may derive the posture of the camera that has captured all the images of the multiview image.

The camera posture estimating unit 403 may correct the derived camera posture of each of the plurality of cameras using 3D coordinates and a bundle adjustment algorithm. The camera posture estimating unit 403 may accurately estimate the camera posture of each of the plurality of cameras through matching and correction of feature points.

The three-dimensional model generator 404 may generate a three-dimensional model of a plant from a multi-view image based on the estimated postures of each of the plurality of cameras. For example, the 3D model generator 404 may reconstruct 3D coordinates based on the estimated postures of each of the plurality of cameras. The three-dimensional model generation unit 404 may reconstruct the surface of the three-dimensional model based on the reconstructed three-dimensional coordinates. The three-dimensional model generator 404 may map a texture of a three-dimensional model based on color information of a multi-view image.

The three-dimensional model generation unit 404 may generate a three-dimensional model according to a growth process for a plurality of plants and adjustment of parameters. For example, the three-dimensional model generation unit 404 may generate a three-dimensional model for each of a plurality of plants whose shape is changed according to a growth process. For example, the three-dimensional model generation unit 404 may generate a three-dimensional model having a different shape by adjusting parameters for each of a plurality of plants. The parameter may be a factor affecting the growth of the plant. The parameter may include, for example, at least one of time, water, temperature, humidity, illuminance, soil, pests, climate, and whether or not to treat drugs, but is not limited thereto.

In addition, the three-dimensional model generation unit 404 may receive parameters related to the environment around the plant using a sensor installed around the plant. The sensor may include, for example, one or more of a temperature sensor, a humidity sensor, and an illuminance sensor, but is not limited thereto. The three-dimensional model generator 404 may generate a three-dimensional model of a plant based on the received parameter value and a multi-view image of the plant.

The transmission unit 405 may transmit plant data for a plurality of plants to the device 120, a device other than the device 120, or a server other than the server 110. The transmission unit 405 may transmit the generated three-dimensional model of the plant to the device 120, a device other than the device 120, or a server other than the server 110. The transmission unit 405 may transmit a three-dimensional model according to the adjustment of a growth process and parameters for a plurality of plants to the device 120, another device other than the device 120, or a server other than the server 110. That is, in the present invention, a three-dimensional model of a plant generated in a user terminal and related data can be transmitted and uploaded to a server, and by receiving a three-dimensional model of a plant generated by another user from the server, data with other users Share can be done.

The above description of the present invention is for illustrative purposes only, and those of ordinary skill in the art to which the present invention pertains will be able to understand that other specific forms can be easily modified without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative and non-limiting in all respects. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as being distributed may also be implemented in a combined form.

The scope of the present invention is indicated by the claims to be described later rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. do.

121: receiver
122: plant selection unit
123: three-dimensional model display unit
124: parameter input unit
125: multi-view image acquisition unit
126: camera posture estimation unit
126a: distortion correction unit
126b: feature point matching unit
126c: camera posture derivation unit
127: three-dimensional model generation unit

Claims (5)

In the device for providing a three-dimensional model of a plant that can interact with a user,
A multi-view image acquisition unit for obtaining a multi-view image of a plant photographed using a plurality of cameras;
A three-dimensional model generation unit that generates a three-dimensional model of the plant that can be displayed as a free viewpoint from the multi-view image by matching corresponding feature points of the multi-view image;
A receiving unit for receiving plant data for a plurality of plants from a server;
A plant selection unit for selecting one plant from among plant data for the plurality of plants;
A three-dimensional model display unit that displays a three-dimensional model of the selected plant; And
Parameter input unit for receiving at least one parameter related to the selected plant
Including,
The three-dimensional model display unit displays the three-dimensional model of the selected plant based on the input parameter, and displays the three-dimensional model of the selected plant as a free viewpoint so that it is freely rotated and its size is enlarged or reduced according to the user's manipulation. That, a three-dimensional model providing device.
The method of claim 1,
Camera posture estimating unit for estimating the posture of each of the plurality of cameras based on the position and rotation angle of each of the plurality of cameras
Including more,
The three-dimensional model generating unit generates a three-dimensional model of the plant from the multi-view image based on the estimated postures of each of the plurality of cameras.
The method of claim 2,
The camera posture estimation unit,
A distortion correction unit correcting distortion of the multi-view image based on the position and rotation angle of each of the plurality of cameras;
A feature point matching unit extracting and matching feature points from the multi-view image; And
Camera posture derivation unit for deriving a camera posture of each of the plurality of cameras estimated based on the matched feature points
Containing, a three-dimensional model providing apparatus.
The method of claim 2,
The three-dimensional model generation unit generates a three-dimensional model according to a growth process of the plurality of plants and adjustment of the parameters.
The method of claim 1,
The parameter includes at least one of time, water, temperature, humidity, illuminance, soil, pests, climate, and whether or not to treat drugs.

Images