KR20220017605A - Method for detecting and tracking object using multi cameras and satellites including the same - Google Patents

Abstract

The present invention relates to a method for detecting and tracking an object in a satellite. An operating method of an artificial satellite having a plurality of cameras having different resolutions and viewing angles from each other comprises: an operation of photographing the ground using a first camera having the largest viewing angle among the plurality of cameras; an operation of detecting an object from the image taken of the ground; an operation of selecting the object to be tracked from among the detected objects; and an operation of tracking the selected object using at least one of the plurality of cameras. Accordingly, it is possible to provide the method for detecting an object in a wide range on the ground and simultaneously observing the detected object and the surroundings thereof with high resolution. By using multiple cameras with different resolutions, objects in a wide range on the ground can be detected and the detected objects and their surroundings can be observed at the same time with high resolution.

Description

Object detection and tracking method using multiple cameras and satellite including the same

Various embodiments relate to a method for detecting and tracking an object and a satellite including the same, and more particularly, to a method for detecting and tracking an object on the ground using multiple cameras in a satellite.

The advent of satellite technology has made it possible to collect data using satellites in a variety of technological applications. In particular, it has become possible to recognize objects on the ground using a camera.

However, there is a problem that the desire to detect an object in a wide range and the desire to view a specific area to be observed in high resolution at the same time cannot be satisfied because the resolution of the existing camera used in the satellite is fixed.

In order to solve the above problem, various embodiments of the present invention provide a method for detecting an object in a wide range on the ground and simultaneously observing an object detected with high resolution and its surroundings using a plurality of cameras having different resolutions, and It is to provide artificial satellites to be used.

The technical problems to be achieved in this document are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those of ordinary skill in the art to which the present invention belongs from the description below. There will be.

According to various embodiments of the present disclosure, the artificial satellite includes a communication unit that transmits and receives data with a mobile device or a ground station on the ground, a power supply unit that provides power to drive the artificial satellite, an attitude control unit that controls the posture of the artificial satellite, and the A thrust unit that provides thrust for movement and posture change of the artificial satellite, a structure and thermal control unit that detects an overheat or low temperature state of an electronic device, board, or module in the satellite, and performs an operation to restore it to a normal state; A multi-camera unit including a plurality of cameras having different resolutions and different viewing angles to take images of the ground, an object detection and tracking unit controlling to detect an object in the image captured by the multi-camera unit, and track the selected object; It may include a processor unit for performing overall control of the satellite.

According to various embodiments of the present disclosure, the multi-camera unit includes a wide-angle camera having a viewing angle of 120 degrees and a resolution of 900 m, a narrow-angle camera having a viewing angle of 80 degrees and a resolution of 5 m, and a zoom camera having a viewing angle of 24 degrees and a resolution of 2 m. can do.

According to various embodiments of the present disclosure, the object detection and tracking unit detects an object based on the image captured by the wide-angle camera, selects an object to be tracked from among the detected objects, determines the location of the selected object, The position of the artificial satellite may be controlled so that the selected object is included within a field of view of a camera to be photographed based on the location of the selected object and the respective viewing angles of the plurality of cameras.

According to various embodiments of the present disclosure, the object detection and tracking unit transmits the image captured by the wide-angle camera and the detected object information to a mobile device, and among the detected objects based on a control command received from the mobile device You can select objects to track.

According to various embodiments of the present disclosure, the object detection and tracking unit receives and stores object information to be tracked in advance from a mobile device, compares the detected object with the previously stored object information, and, as a result of the comparison, the detection Among the objects, the object with the highest matching rate can be selected.

According to various embodiments of the present disclosure, the object detection and tracking unit may detect and track an object based on an artificial intelligence algorithm learned to detect an object in an image.

According to various embodiments of the present disclosure, the artificial intelligence algorithm may be learned using an image obtained by blurring an image captured by an aircraft.

According to various embodiments of the present disclosure, a method of operating a satellite having a plurality of cameras having different resolutions and viewing angles includes: photographing the ground using a first camera having the largest viewing angle among the plurality of cameras; The method may include an operation of detecting an object in an image captured on the ground, an operation of selecting an object to be tracked from among the detected objects, and an operation of tracking the selected object using at least one of the plurality of cameras.

According to various embodiments of the present disclosure, the operation of selecting an object to be tracked from among the detected objects is based on an operation of transmitting an image captured by a mobile device and the detected object and a control command received from the mobile device. It may include an operation of selecting an object to be tracked from among the detected objects.

According to various embodiments of the present disclosure, the operation of selecting an object to be tracked from among the detected objects includes an operation of receiving and storing information on an object to be tracked in advance from a mobile device, and an operation of storing the detected object and the previously stored object information. Comparing may include, as a result of the comparison, selecting an object having the highest matching rate among the detected objects as an object to be tracked.

According to various embodiments of the present disclosure, the operation of detecting the object in the image photographed of the ground may include the operation of detecting the object in the image photographed of the ground based on a learned artificial intelligence algorithm.

According to various embodiments of the present disclosure, the artificial intelligence algorithm may be learned using an image obtained by blurring an image captured by an aircraft.

According to various embodiments of the present disclosure, the operation of tracking the selected object using at least one of the plurality of cameras is based on the operation of recognizing the position of the selected object, the identified position, and the respective viewing angles of the plurality of cameras. and controlling the posture of the artificial satellite so that the selected object is included within the viewing angle of the camera to be photographed.

According to various embodiments of the present disclosure, a method of operating a satellite having a plurality of cameras having different resolutions and viewing angles includes: photographing the ground using a first camera having the largest viewing angle among the plurality of cameras; An operation of detecting an object in an image taken on the ground, an operation of recognizing at least three objects in which ground coordinates are stored among the detected objects, an operation of obtaining the ground coordinates of the at least three recognized objects, and the obtained and determining the posture of the artificial satellite based on at least three ground coordinates.

The method and apparatus proposed by the present disclosure may provide a function of detecting a moving object on the ground in a wide range, and observing and tracking the motion of the detected object as a high-resolution image.

In addition, the method and apparatus proposed by the present disclosure may provide a function of determining the posture of the artificial satellite using a camera that captures the ground.

Effects obtainable in the present disclosure are not limited to the above-mentioned effects, and other effects not mentioned may be clearly understood by those of ordinary skill in the art to which the present disclosure belongs from the description below. will be.

1 is a diagram illustrating a schematic diagram of a system for communication between one or more satellites and a mobile device) according to various embodiments of the present disclosure;
2 is a diagram illustrating an example of detecting and tracking an object using a plurality of cameras mounted on an artificial satellite according to various embodiments of the present disclosure;
3 is a diagram illustrating a schematic configuration of an artificial satellite according to various embodiments of the present disclosure;
4 is a diagram illustrating a structure of a convolutional neural network as an example of an artificial intelligence algorithm.
5 is a flowchart illustrating an operation of detecting and tracking an object in a satellite according to various embodiments of the present disclosure.
6 is a flowchart illustrating an operation of selecting an object to be tracked by a satellite according to various embodiments of the present disclosure;
7 is a flowchart illustrating another example of an operation of selecting an object to be tracked by a satellite according to various embodiments of the present disclosure;
8 is a flowchart illustrating an operation of tracking an object selected by an artificial satellite according to various embodiments of the present disclosure;
9 is a flowchart illustrating an operation of determining a posture of an artificial satellite by detecting an object from an artificial satellite according to various embodiments of the present disclosure;
In connection with the description of the drawings, the same or similar reference numerals may be used for the same or similar components.

Hereinafter, the embodiments disclosed in the present specification will be described in detail with reference to the accompanying drawings, but the same or similar components are assigned the same reference numbers regardless of reference numerals, and redundant description thereof will be omitted.

In the following, specific details may be set forth in order to provide an understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these details. In addition, those skilled in the art will recognize that the various embodiments of the present invention described below may be implemented in various ways, such as as a process, an apparatus, a system, or a method on a computer-readable medium.

The components shown in the drawings show only exemplary embodiments of the present invention, and are intended to avoid obscuring the present invention. In addition, the connection between the components in the drawing is not limited to a direct connection. Rather, data between these components may be modified, reformatted, or otherwise altered by intermediate components or devices. Also, more or fewer connections may be used. The terms “coupled” or “communicatively coupled” should be understood to include direct connections, indirect connections through one or more intermediary devices, and wireless connections.

In addition, by applying the related art, a person skilled in the art will recognize that (1) certain operations may be selectively performed, (2) operations may not be limited to a specific order presented herein, and (3) specific operations may be performed differently. may be performed in sequence, and (4) certain operations may be performed concurrently.

Hereinafter, the term “mobile device” may refer to various types of electronic devices that allow a user to communicate directly with a satellite and exchange data with a satellite in real time. In addition, electronic devices include mobile phones, palm computers, tablet PCs, notebook computers, desktop computers, augmented reality (AR) devices, virtual reality (VR) devices, smart wearables such as smart goggles, smart glasses and smart watches, remote control devices, It may include a computing device embedded in a vehicle, and the like.

1 is a schematic diagram of a system 100 for communication between one or more satellites 104a-104n and a mobile device 102a-102m in accordance with various embodiments. As shown in FIG. 1 , a plurality of satellites 104a - 104n may form and fly in a particular formation, and may be directly communicatively connected to mobile devices 102a - 102m or communicated through a ground station 106 . can be connected

In various embodiments, the satellites 104a - 104n may include low earth orbiting (LEO) satellites orbiting the Earth to appear to move when viewed from the Earth. In various embodiments, the satellites 104a - 104n are also geosynchronous satellites that orbit the Earth at the same rate as the Earth's rotation (i.e., an orbital duration equal to the Earth's rotation period so that when viewed from Earth, they appear to remain in the same position) ) may be included. In various embodiments, the satellites 104a - 104n may communicate with each other to form the satellite network 104 , ie, perform inter-satellite communication and share data.

In various embodiments, each satellite 104a - 104n may be a satellite of various sizes, such as microsat, nanosat, and cubesat. Further, the number and location of the satellites 104a - 104n included in the satellite network 104 may be determined to cover the entire surface area of the Earth. Accordingly, one mobile device (eg, 102a) can communicate with another mobile device (eg, 102m) at any location on the earth. For example, a signal transmitted from the mobile device 102a is transmitted to another mobile device ( 102m).

In various embodiments, the satellites 104a - 104n may perform missions individually or in association with missions. In various embodiments, missions may include direct communication with mobile devices 102a-102m, surface observations, weather tracking, sea measurements, and scientific studies of celestial bodies, and the like. Missions may also include various scientific studies of polar landscapes and icebergs, Earth's surface and atmosphere, ocean circulation, water/energy cycles, and monitoring of concentrations of phytoplankton, suspended particulate matter and/or dissolved organic matter in a given area. . In addition, missions may include the use of cameras (optics, synthetic aperture radar (SAR), IR (Infrared), etc.) to analyze surface topography and directly recognize and check in real time specific phenomena occurring in specific areas, The task may also include detecting and tracking specific objects on the surface of the earth.

In various embodiments, the mobile device 102a or 102m may be in direct real-time data communication with the satellites 104a-104n. Here, the data may include one or more of text messages, voice messages, images/pictures in JPEG format, video clips in MPEG format, telemetry data, and the like. As such, the satellites 104a-104n may allow two users of the mobile devices 102a and 102m to make a real-time phone call with each other. For direct communication, no ground station or base station may be required for communication between the two mobile devices 102a and 102m in system 100 . (Hereinafter, the term ground station refers to a station having a fixed or moving position on the earth). Thus, in various embodiments, the mobile device (eg, 102m) is located in an area such as a desert or remote/remote area so that the mobile device 102m is capable of providing wireless communication services between mobile devices, any ground station or any Even if the base station cannot be reached, a phone call between the two mobile devices may be possible.

In various embodiments, if the mobile device is unable to transmit a signal directly to the satellites 104a-104n due to limited output power, it is connected to a mobile relay antenna capable of communicating directly with the satellites 104a-104n and the satellites ( 104a-104n).

In various embodiments, the mobile device may communicate with the satellites 104a - 104n via the ground station 106 . The ground station 106 may have a server capable of sharing (and storing) information with the satellites 104a - 104n and performing various tasks on behalf of the satellites 604 .

In various embodiments, the ground station 106 may communicatively connect with one or more servers (or computing devices) 110 wirelessly, by wire, or via the Internet 112 . In various embodiments, one of the servers 110 may be a computing device of an advertising company, and may transmit advertising information to the satellites 104a - 104n via the ground station 106 . In addition, one of the servers 110 may be capable of learning an artificial intelligence algorithm for detecting an object in an image.

In various embodiments, a user of a mobile device (eg, 102a ) may have direct access to data stored on satellites, and/or a user of a mobile device (eg, 102a ) may control one or more satellites to control one or more of the satellites. Can perform space missions. In various embodiments, mobile device 102a may transmit data to a satellite (eg, 104a) to control the satellite (eg, 104b), wherein the data includes a user ID, password, target satellite, satellite identification information. , one or more control parameters such as, but not limited to, an angular range for determining a coverage area, a control purpose, an observation time range, and the like.

2 is a diagram illustrating an example of detecting and tracking an object using a plurality of cameras mounted on an artificial satellite according to various embodiments of the present disclosure;

Referring to FIG. 2 , a satellite (eg, 104a) may be equipped with a plurality of cameras 210 , 220 , and 230 having different field of view (FOV) and resolution. Here, the viewing angle may refer to an angle expressed by the size of an area that the camera can observe at a time. For example, the camera 210 may be a wide-angle camera having a viewing angle of 120 degrees and a resolution of 900 m, the camera 220 may be a narrow-angle camera having a viewing angle of 80 degrees and a resolution of 5 m, and the camera 230 may be 24 It may be a zoom camera having a viewing angle of degrees and a resolution of 2 m.

According to various embodiments, the satellite 104a may detect an object in a wide area by first using the camera 210 having a wide viewing angle and resolution. In this case, the learned artificial intelligence algorithm may be used to detect the object 250 in the image taken by the camera 210 .

When the object 250 is detected from the image taken by the camera 210, the detected object 250 can be observed at high resolution using a camera having a smaller viewing angle and resolution (eg, the narrow-angle camera 220), and also , it is possible to observe and track the detected object 250 as a high-resolution image by using a camera having a smaller viewing angle and resolution (eg, the zoom camera 230 ).

3 is a diagram illustrating a schematic configuration of an artificial satellite according to various embodiments of the present disclosure;

Referring to FIG. 3 , the satellite 104a according to various embodiments includes a processor unit 310 , a communication unit 320 , a power supply unit 330 , an attitude control unit 340 , a thrust unit 350 , a structure and a heat control unit 360 . ) and a multi-camera unit 370 and an object detection and tracking unit 380 .

The processor unit 310 may execute a basic program for operating the artificial satellite 104a and various application programs to perform tasks assigned to the artificial satellite 104a. For program execution, the processor unit 310 may include one or more processing units such as FPGAs, EPLDs, CPUs, and ASICs, and a memory capable of storing programs and related data. As the memory, SSD, DRAM, SRAM, or the like may be used. The processor unit 310 may generate a control signal necessary to perform a task while executing an application program and transmit it to each unit of the satellite 104a, and process the data received from each unit of the satellite 104a to provide an additional control signal can create In addition, data may be transmitted to the mobile devices 102a-102m or the ground station 106 to provide the user with relevant information on what they are doing. In addition, it receives a mission-related control signal from the mobile device 102a-102m or the ground station 106, and generates and transmits other control signals for each part of the satellite 104a in order to execute control according to the received control signal. . In one embodiment, the processor unit 310 receives an object tracking command from the mobile devices 102a-102m or the ground station 106, and transmits a control signal for at least one camera of the multi-camera unit 370 to photograph the ground. It may be transmitted to the camera unit 370 , and a control signal for detecting and tracking an object may be transmitted to the object detection and tracking unit 380 .

In another embodiment, the processor unit 310 obtains an artificial intelligence algorithm learned for object detection from the mobile devices 102a-102m or the ground station 106, and the object detection and tracking unit 380 uses the corresponding artificial intelligence algorithm. can be set to be used. According to an embodiment, an artificial intelligence algorithm for object detection may be learned by the servers 110 and 108 .

The processor unit 310 may generate a control signal for controlling each unit of the artificial satellite 104a based on a control request coming from each unit of the artificial satellite 104a. For example, when it is necessary to change the posture of the satellite 104a in order to track the detected object, the object detection and tracking unit 380 may transmit a posture control request to the processor unit 310 . The processor unit 310 may transmit a control signal to change the posture of the satellite 104a to the posture control unit 340 based on the received posture control request.

The processor unit 310 may communicate with the mobile devices 102a-102m or the ground station 106 through the communication unit 320 .

The communication unit 320 may provide a data transmission/reception function with the mobile devices 102a-102m or the ground station 106 . This not only provides communication between mobile devices 102a-102m or between mobile devices 102a-102m and the ground station 106, but also provides content generated by satellite 104a to mobile devices 102a-102m or may be transmitted to the ground station 106 .

The communication unit 320 may provide a function for transmitting and receiving control-related data with the mobile devices 102a-102m or the ground station 106, and the communication unit 320 is an application program or camera developed by an actual user. An application containing an artificial intelligence algorithm used to detect and track an object in an image taken by the camera is received during operation, and the application in use can be updated or applied by changing to a new application. Data received by the communication unit 320 may be transmitted to the processing unit 310 .

In addition, when the plurality of artificial satellites 104a - 104n form the satellite network 104 , the communication unit 320 may provide data transmission and reception functions with other satellites.

The power supply unit 330 provides power for driving the satellite 104a and may include a battery. The power source 330 may include a solar panel and a photovoltaic device to generate power using sunlight.

The attitude control unit 340 may control the attitude of the artificial satellite 104a. By receiving a control command from the processor unit 310, the user can control the posture of the satellite 104a in a desired form. As an embodiment, the posture control unit 340 may rotate left or right according to a control command from the processor unit 310 . In addition, the posture control unit 340 may control the posture of the satellite 104a so that the detected object 250 may be included within the viewing angle of the camera of the multi-camera unit 370 according to a control command from the processor unit 310 . have.

The thrust unit 350 provides thrust to move the artificial satellite 104a to a desired position or to create a specific posture. Liquid thrusters can be used for large satellites, and electric thrusters can be used for small satellites. The thrust unit 350 may provide thrust according to a control command from the posture control unit 340 or the processor unit 310 .

The structure and heat control unit 360 may turn on a heater or operate a cooler to avoid overheating or low temperature of a specific board or module. If necessary, a control command may be transmitted to the attitude control unit 340 directly or through the processor unit 310 to control the attitude of the artificial satellite 104a for temperature control. In one embodiment, the module or board shown as a low temperature is positioned toward the sun so that the temperature may rise, it may be requested to rotate the artificial satellite (104a) to the attitude control unit 340. As another embodiment, in order to lower the temperature of the overheated module, the orientation control unit 340 may request rotation of the artificial satellite 104a so that the overheated module is positioned in the opposite direction to the sun. The structure and heat control unit 360 may include a temperature measuring sensor for measuring the temperature of a specific board or module.

The multi-camera unit 370 may include a plurality of cameras 210 , 220 , and 230 having different viewing angles and resolutions. For example, the multi-camera unit 370 includes a wide-angle camera 210 having a viewing angle of 120 degrees and a resolution of 900 m, a narrow-angle camera 220 having a viewing angle of 80 degrees and a resolution of 5 m, and a viewing angle of 24 degrees and a resolution of 2 m. A zoom camera 230 may be included. In addition, the multi-camera unit 370 may include a plurality of cameras of various combinations.

The plurality of cameras 210 , 220 , 230 in the multi-camera unit 370 may photograph the ground by at least one camera based on the control of the processor unit 310 . According to an embodiment, the ground may be photographed using only the wide-angle camera 210 , or the ground may be photographed simultaneously using the wide-angle camera 210 and the zoom camera 230 . The multi-camera unit 370 may transmit an image or image obtained through photographing to the object detection and tracking unit 380 .

The object detection and tracking unit 380 may detect an object from an image or an image from the multi-camera unit 370 and track the movement of the detected object in the image.

The object detection and tracking unit 380 may detect an object in an image from the wide-angle camera 210 having a wide viewing angle of the multi-camera unit 370 . According to an embodiment, the object detection and tracking unit 380 may detect an object based on a control signal from a user transmitted by the processor unit 310 . In this case, the user may designate an object to be tracked and notify the object detection and tracking unit 380 through a control signal.

According to an embodiment, the object detection and tracking unit 380 may detect an object from an image using a learned artificial satellite algorithm.

The object detection and tracking unit 380 may determine the location of the detected object. According to an embodiment, the object detection and tracking unit 380 may acquire GPS coordinates of the detected object.

The object detection and tracking unit 380 uses a narrow-angle camera 220 or a zoom camera 230 with a narrow viewing angle and high resolution in the multi-camera unit 370 to observe the detected object in high resolution. can be controlled to shoot. According to an embodiment, the object detection and tracking unit 380 may transmit a corresponding control through the processor unit 310 .

Also, the object detecting and tracking unit 380 may transmit a command for controlling the posture of the artificial satellite 104a to the posture controlling unit 340 in order to track the detected object. According to an embodiment, the object detection and tracking unit 380 may photograph the detected object using the narrow-angle camera 220 or the zoom camera 230 to track the detected object with high resolution, in this case, The posture of the satellite 104a may be controlled based on the position of the object so that the detected object comes within the viewing angle of the narrow-angle camera 220 or the zoom camera 230 .

The object detection and tracking unit 380 may detect and track an object in an image by using the learned artificial intelligence algorithm.

The artificial intelligence algorithm may be learned on the server 110, 108 or the mobile device 102a-102m.

An example of an artificial intelligence algorithm is shown in FIG. 3 .

4 is a diagram illustrating a structure of a convolutional neural network (CNN) as an example of an artificial intelligence algorithm.

An artificial intelligence algorithm having a convolutional neural network structure as shown in FIG. 4 may be effective in identifying structured spatial data such as images, moving pictures, and character strings. The convolutional neural network can effectively recognize features with adjacent images while maintaining spatial information of the image.

Referring to FIG. 4 , the convolutional neural network may include a feature extraction layer 60 and a classification layer 70 . The feature extraction layer 60 may extract features of the image by synthesizing spatially nearby ones in the image using convolution.

The feature extraction layer 60 may be configured in a form in which a plurality of convolution layers 61 and 65 and pooling layers 63 and 67 are stacked. The convolution layers 61 and 65 may be formed by applying a filter to input data and then applying an activation function. The convolution layers 61 and 65 may include a plurality of channels, and each channel may have different filters and/or different activation functions applied to each other. The result of the convolutional layers 61 and 65 may be a feature map. The feature map may be data in the form of a two-dimensional matrix. The pooling layers 63 and 67 may receive the output data of the convolution layers 61 and 65, that is, the feature map as an input, and may be used to reduce the size of the output data or to emphasize specific data. The pooling layers 63 and 67 include max pooling that selects the largest value among some data of the output data of the convolution layers 61 and 65, average pooling that selects the average value, and the minimum value. The output data can be generated by applying a function of min pooling of choice.

The size of the feature map generated by going through a series of convolutional layers and pooling layers can be gradually reduced. The final feature map generated through the last convolutional layer and the pooling layer may be converted into a one-dimensional form and input to the classification layer 70 . The classification layer 70 may be a fully connected artificial neural network structure. The number of input nodes of the classification layer 70 may be equal to the number of elements in the matrix of the final feature map multiplied by the number of channels.

Training an artificial intelligence algorithm can be viewed as determining the model parameters that minimize the loss function. The loss function can be used as an index to determine the optimal model parameter in the learning process of the artificial intelligence algorithm. In the case of a fully connected artificial neural network, the weight of each synapse may be determined by learning, and in the case of a convolutional neural network, a filter of the convolutional layer for extracting a feature map may be determined by learning.

Learning of artificial intelligence algorithms may be performed by supervised learning. Supervised learning refers to a method of learning an artificial intelligence algorithm in a state where a label for the learning data is given, and the label is the correct answer (or result) that the artificial intelligence algorithm must infer when the learning data is input to the artificial intelligence algorithm. value) can be

The purpose of the artificial intelligence algorithm used in the present invention may be to find an object in an image captured by a satellite. Therefore, for learning the artificial intelligence algorithm used in the present invention, an image captured by an artificial satellite and an object captured from the image can be used. In addition, in order to obtain more learning data, it may be possible to blur an image taken from an aircraft in a lower orbit than the satellite and then use it as an image for learning.

5 is a flowchart illustrating an operation of detecting and tracking an object in a satellite according to various embodiments of the present disclosure.

Referring to FIG. 5 , in operation S100 , the satellite may photograph a wide area of the ground using a wide-angle camera.

In operation S200, the satellite may detect objects in the captured image. According to an embodiment, the object detection in the image may be performed by a learned artificial intelligence algorithm.

In operation S300, the satellite may select an object to be tracked from among the detected objects.

In operation S400 , the satellite may track the selected object by using at least one of a wide-angle camera, a narrow-angle camera, and a zoom camera with respect to the selected object.

6 is a flowchart illustrating an operation of selecting an object to be tracked by a satellite according to various embodiments of the present disclosure;

Referring to FIG. 6 , in operation S310 , the artificial satellite may transmit an image captured by the wide-angle camera to the mobile devices 102a-102m. The user of the mobile devices 102a-102m may select at least one of an object included in an image displayed on the mobile device 102a-102m or an object detected by an artificial satellite. According to an embodiment, although not shown, the mobile devices 102a-102m may transmit a control command requesting a higher resolution image to the satellite to the satellite, and in response, the satellite may use a narrow-angle camera and/or a zoom camera. The captured image may be transmitted to the mobile devices 102a-102m.

In operation S320, the satellite may receive an object selection related control command. And based on this, you can select an object to track.

7 is a flowchart illustrating another example of an operation of selecting an object to be tracked by a satellite according to various embodiments of the present disclosure;

Referring to FIG. 7 , in operation S330 , the satellite may obtain and store object information to be tracked in advance from the mobile devices 102a-102m. According to an embodiment, the object information may have an output format of a learned artificial intelligence algorithm.

In operation S340, the satellite may compare the object detected in operation S200 with object information stored in advance.

In operation S350, the satellite may select an object having the highest matching rate as a result of the comparison.

Additionally, in operation S360, the satellite may transmit the selected object to the mobile devices 102a-102m, and receive a confirmation request from the user. In response, the mobile devices 102a-102m may provide information on whether the object to be tracked is correct.

8 is a flowchart illustrating an operation of tracking an object selected by a satellite according to various embodiments of the present disclosure;

Referring to FIG. 8 , in operation S410 , the satellite may determine the location of the selected object.

In operation S420 , the satellite may adjust the position of the satellite to include the location of the selected object within the viewing angle of the camera and the movement path based on the camera that captured the image.

In operation S430, the satellite may track the selected object using at least one of a wide-angle camera, a narrow-angle camera, and a zoom camera.

9 is a flowchart illustrating an operation of determining a posture of an artificial satellite by detecting an object from an artificial satellite according to various embodiments of the present disclosure;

Referring to FIG. 9 , in operation S100 , the artificial satellite may photograph a wide area of the ground using a wide-angle camera.

In operation S200, the satellite may detect objects in the captured image. According to an embodiment, the object detection in the image may be performed by a learned artificial intelligence algorithm.

In operation S500, the artificial satellite may detect an object having known coordinates on the ground from among the detected objects. To this end, the satellite may use at least one of a narrow-angle camera or a zoom camera to photograph the ground at a higher resolution. For example, the satellite stores longitude and latitude coordinates for landmark buildings, lakes, rivers, and mountains on the ground, and coordinates among objects detected in the image captured in operation S200 or additional captured image in operation S500 The object in which information is stored can be recognized. In this case, the satellite may recognize three or more objects and obtain coordinates corresponding to the objects.

In operation S600, the artificial satellite may determine the posture of the artificial satellite based on the obtained three or more coordinates. That is, the position determination of the artificial satellite, which has been performed using the conventional sun or star tracker, can be performed using landmark objects in the image captured by the camera.

Based on the artificial satellite and the operation of the artificial satellite proposed in the present invention, it is possible to use a plurality of cameras having different resolutions to detect objects in a wide range on the ground and to provide a method for tracking while observing and surrounding objects detected with high resolution You can do it. Also, it is possible to provide a method of determining the posture of the artificial satellite by using an object capable of recognizing coordinates among the detected objects.

Programs implementing various aspects of the present invention may be accessed from a remote location (eg, a server) via a network. Such data and/or programs may be delivered via any of a variety of machine-readable media. Machine-readable media include hard disks, magnetic media such as floppy disks and magnetic tapes, optical media such as CD-ROMs and holographic devices, magneto-optical media and application specific integrated circuits (ASICs), programmable logic devices (PLDs), flash may include, but are not limited to, hardware devices specifically configured to store or store and execute program code such as memory devices, ROM and RAM devices.

Embodiments of the present invention may be encoded in one or more non-transitory computer-readable media as instructions for one or more processors or processing units to cause operations to be performed. The one or more non-transitory computer readable media may include volatile and non-volatile memory. Alternative implementations may be possible, including hardware implementations or software/hardware implementations. Hardware-implemented functions may be realized using ASICs, programmable arrays, digital signal processing circuits, and the like. Accordingly, the term “means” in any claim may include both software and hardware implementations. Similarly, the term "computer readable medium" as used herein includes software and/or hardware having a program of instructions embodied thereon, or a combination thereof. With these implementation alternatives in mind, the drawings and the accompanying description describe the functions necessary for a person skilled in the art to write program code (ie, software) and/or to manufacture circuits (ie, hardware) to perform the required processing. It should be understood that information is provided.

Claims (16)

In artificial satellites,
a communication unit for transmitting and receiving data with a ground mobile device or a ground station;
a power supply unit providing power for driving the satellite;
an attitude control unit for controlling the attitude of the artificial satellite;
a thrust unit providing thrust for movement and posture change of the artificial satellite;
a structure and thermal control unit that detects an overheat or low temperature state of the electronic device, board, and module in the satellite, and performs an operation to restore it to a normal state;
a multi-camera unit including a plurality of cameras having different resolutions and different viewing angles from each other to take images of the ground;
an object detection and tracking unit that detects an object in the image taken by the multi-camera unit and controls to track the selected object; and
A satellite comprising a processor unit for performing overall control of the satellite.
According to claim 1,
The multi-camera unit,
A satellite comprising a wide-angle camera having an angle of view of 120 degrees and a resolution of 900 m, a narrow-angle camera having an angle of view of 80 degrees and a resolution of 5 m and a zoom camera having an angle of view of 24 degrees and a resolution of 2 m.
3. The method of claim 2,
The object detection and tracking unit,
Detecting an object based on the image taken by the wide-angle camera,
Select an object to be tracked from among the detected objects,
Determine the location of the selected object,
and controlling a posture of the satellite based on a position of the selected object and a viewing angle of each of the plurality of cameras so that the selected object is included within a viewing angle of a camera to be photographed.
4. The method of claim 3,
The object detection and tracking unit,
Transmitting the image captured by the wide-angle camera and the detected object information to the mobile device,
and selecting an object to be tracked from among the detected objects based on a control command received from the mobile device.
4. The method of claim 3,
The object detection and tracking unit,
Receive and store object information to be tracked in advance from the mobile device,
comparing the detected object with the previously stored object information;
As a result of comparison, the satellite that selects the object with the highest matching rate among the detected objects.
According to claim 1,
The object detection and tracking unit,
A satellite that detects and tracks an object based on an artificial intelligence algorithm learned to detect the object in an image.
7. The method of claim 6,
The artificial intelligence algorithm is learned using the image obtained by blurring the image taken from the aircraft, artificial satellite.
A method of operating an artificial satellite having a plurality of cameras having different resolutions and viewing angles from each other, the method comprising:
photographing the ground using a first camera having the largest viewing angle among the plurality of cameras;
detecting an object in the image of the ground;
selecting an object to be tracked from among the detected objects;
and tracking the selected object using at least one of the plurality of cameras.
9. The method of claim 8,
The operation of selecting an object to be tracked from among the detected objects includes:
transmitting the captured image and the detected object to a mobile device;
and selecting an object to be tracked from among the detected objects based on a control command received from the mobile device.
9. The method of claim 8,
The operation of selecting an object to be tracked from among the detected objects includes:
receiving and storing object information to be tracked in advance from a mobile device;
comparing the detected object with the previously stored object information;
and selecting, as an object to be tracked, an object having the highest matching rate among the detected objects as a result of the comparison.
9. The method of claim 8,
The operation of detecting an object in the image taken of the ground is,
An operation method of an artificial satellite, comprising the operation of detecting an object in the image taken on the ground based on the learned artificial intelligence algorithm.
12. The method of claim 11,
The artificial intelligence algorithm is a method of operation of an artificial satellite, which is learned using an image obtained by blurring an image taken from an aircraft.
9. The method of claim 8,
The operation of tracking the selected object using at least one of the plurality of cameras includes:
detecting the location of the selected object;
and controlling the posture of the artificial satellite so that the selected object is included within a field of view of a camera to be photographed based on the identified position and the respective viewing angles of the plurality of cameras.
A method of operating an artificial satellite having a plurality of cameras having different resolutions and viewing angles from each other, the method comprising:
photographing the ground using a first camera having the largest viewing angle among the plurality of cameras;
detecting an object in the image of the ground;
recognizing at least three objects in which ground coordinates are stored among the detected objects;
obtaining ground coordinates of the at least three recognized objects;
and determining the posture of the artificial satellite based on the obtained at least three ground coordinates.
15. The method of claim 14,
The operation of detecting an object in the image taken of the ground is,
An operation method of an artificial satellite, comprising the operation of detecting an object in the image taken on the ground based on the learned artificial intelligence algorithm.
16. The method of claim 15,
The artificial intelligence algorithm is a method of operation of an artificial satellite, which is learned using an image obtained by blurring an image taken from an aircraft.

Images