WO2020144936A1

WO2020144936A1 - Information processing device, information processing method, and program

Info

Publication number: WO2020144936A1
Application number: PCT/JP2019/044630
Authority: WO
Inventors: 駿李
Original assignee: ソニー株式会社
Priority date: 2019-01-09
Filing date: 2019-11-14
Publication date: 2020-07-16

Abstract

Provided are an information processing device, an information processing method, and a program so that it is possible to minimize power consumed by a plurality of sensors mounted on a mobile body.　The information processing device (10) comprises: a sensor control unit (16) that controls the respective detection conditions of a plurality of sensors mounted on a mobile body (100) on the basis of control information for controlling the mobile body (100) and environmental information indicating the surroundings of the mobile body (100); and an acquisition unit (11) that acquires the sensor information detected by the plurality of sensors under the respective detection conditions.

Description

Information processing apparatus, information processing method, and program

The present disclosure relates to an information processing device, an information processing method, and a program.

[Patent Document 1] discloses a control device and the like that obtains information about the general condition of a structure and flies the periphery of the structure based on the information to generate flight information of a flying body that images the structure.

International Publication No. 2015/163106

In the above conventional technology, it is desirable to cover all directions of a moving body such as a flying body when the traveling direction is all directions such as front-rear, left-right, up-down, etc. However, when the weight of the battery that can be mounted on the mobile body is limited, there is a demand for suppressing the power consumed by the plurality of sensors that image in all directions.

Therefore, the present disclosure proposes an information processing device, an information processing method, and a program capable of suppressing the power consumed by a plurality of sensors mounted on a mobile body.

In order to solve the above problems, an information processing device according to an aspect of the present disclosure is mounted on the moving body based on control information for controlling the moving body and environment information indicating the surroundings of the moving body. A sensor control unit that controls the detection condition of each of the plurality of sensors, and an acquisition unit that acquires the sensor information detected by each of the plurality of sensors under the detection condition are provided.

Further, in an information processing method according to an aspect of the present disclosure, a computer uses a plurality of sensors mounted on the mobile body based on control information for controlling the mobile body and environment information indicating the surroundings of the mobile body. Each detection condition is controlled, and each of the plurality of sensors acquires sensor information detected under the detection condition.

In addition, a program according to an aspect of the present disclosure causes a computer to control each of a plurality of sensors mounted on the moving body based on control information for controlling the moving body and environment information indicating the surroundings of the moving body. The sensor control unit that controls the detection condition and each of the plurality of sensors function as an acquisition unit that acquires the sensor information detected under the detection condition.

It is a figure for explaining an example of a flying object provided with an information treatment device concerning a 1st embodiment. It is a figure which shows the structural example of the flying body provided with the information processing apparatus which concerns on 1st Embodiment. It is a figure which shows the recognition example of the dynamic object of the information processing apparatus which concerns on 1st Embodiment. It is a figure which shows the example of recognition of the external environment of the information processing apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the map information of the information processing apparatus which concerns on 1st Embodiment. It is a figure for explaining an example of processing of sensor information of an information processor concerning a 1st embodiment. 6 is a flowchart showing an example of a processing procedure executed by the information processing apparatus according to the first embodiment. It is a figure which shows the example of determination of the frame rate of the information processing apparatus which concerns on 1st Embodiment. It is a figure which shows an example of a structure of the information processing apparatus which concerns on 2nd Embodiment. 9 is a flowchart showing an example of a processing procedure executed by the information processing apparatus according to the second embodiment. It is a figure for explaining an example of determination of a detection condition of a sensor of an information processor concerning a 2nd embodiment. It is a hardware block diagram which shows an example of the computer which implement|achieves the function of an information processing apparatus.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In addition, in each of the following embodiments, the same portions are denoted by the same reference numerals, and a duplicate description will be omitted.

(First embodiment)
[Outline of system according to first embodiment]
FIG. 1 is a diagram for explaining an example of an air vehicle provided with the information processing device according to the first embodiment.

As shown in FIG. 1, the air vehicle 100 includes, for example, a drone capable of autonomous movement, an unmanned aerial vehicle (UAV), and the like. The flying body 100 is an example of a moving body that can move autonomously. In the case where the flying object 100 can fly in all directions, for example, front and rear, left and right, and up and down, in order to fly without colliding with the external environment, a plurality of sensors are mounted so as to cover all directions. Is desirable. However, when the air vehicle 100 is provided with the sensors in all directions, the power consumption increases according to the number of the sensors, and the power consumption can be suppressed in the air vehicle 100 in which the weight of the battery to be mounted is limited. Is desired. In addition, when the flying object 100 processes the detection results of a plurality of sensors with a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc., the heat of the CPU, GPU, etc. may rise due to an increase in load. .. However, it is desired that the aircraft 100 suppresses the rise of heat without using a cooling device or the like in order to suppress the mounted weight. In the present disclosure, it is an object of the present disclosure to eliminate the situation of the air vehicle 100 equipped with a plurality of sensors. In addition, in the following description, the flying body 100 may be referred to as “own aircraft”.

The air vehicle 100 includes a plurality of imaging devices 111 and an information processing device 10. The plurality of imaging devices 111 and the information processing device 10 are electrically connected. The flying vehicle 100 includes a plurality of imaging devices 111 so that different areas around the aircraft can be imaged. The plurality of imaging devices 111 are examples of a plurality of sensors that detect the external environment of the own device. The imaging device 111 includes, for example, a ToF (Time Of Flight) camera, a stereo camera, a monocular camera, an infrared camera, a Depth camera, and other cameras. Although the present disclosure describes a case where the flying object 100 has four imaging devices 111, the number is not limited. Further, in the following description, in the description for distinguishing each of the four image pickup devices 111, the four image pickup devices 111 are appropriately referred to as an image pickup device 111a, an image pickup device 111b, an image pickup device 111c, and an image pickup device 111d.

In the example shown in FIG. 1, it is assumed that the flying object 100 is flying (moving) in the traveling direction A. The traveling direction A changes according to the flight direction of the flying object 100. The imaging device 111a captures an image of the imaging region D1 of the flying object 100. The imaging area D1 is an area in which the traveling direction A of the flying object 100 is imaged. The imaging device 111b captures an image of the imaging region D2 of the flying object 100. The imaging region D2 is a region in which the right direction of the aircraft 100 moving in the traveling direction A is imaged. The imaging device 111c captures an image of the imaging region D3 of the flying object 100. The imaging region D3 is a region in which a direction opposite to the traveling direction A of the flying object 100 is imaged. The imaging device 111d captures an image of the imaging region D4 of the flying object 100. The imaging region D4 is a region in which the left direction of the aircraft 100 moving in the traveling direction A is imaged.

When the information processing device 10 acquires sensor information (for example, an image) from a plurality of image pickup devices 111, the information processing device 10 executes an algorithm process on the sensor information. The algorithm process includes, for example, a process of recognizing the state of the flying object 100, a process of recognizing the external environment of the flying object 100, a process of creating a map around the flying object 100, and the like. The information processing device 10 has a function of correcting the action plan of the flying object 100 based on the sensor information and the processing result of the algorithm processing. The action plan includes information such as a flight route, a flight speed, and a posture, for example. The information processing device 10 dynamically controls the imaging condition (detection condition) of each of the plurality of imaging devices 111 of the flying object 100 that is flying based on the action plan. The information processing device 10 dynamically changes the respective detection conditions of the plurality of sensors mounted on the air vehicle 100 according to, for example, the operating state of the own device, the state of the external environment of the own device, and the like.

[Configuration Example of Aircraft According to First Embodiment]
Next, an example of the configuration of the information processing device 10 mounted on the aircraft 100 according to the first embodiment will be described. FIG. 2 is a diagram illustrating a configuration example of an air vehicle 100 including the information processing device 10 according to the first embodiment.

As shown in FIG. 2, the air vehicle 100 includes a sensor unit 110, a communication unit 120, a drive unit 130, and an information processing device 10. The sensor unit 110, the communication unit 120, the drive unit 130, and the information processing device 10 are connected to each other so that data and signals can be exchanged.

The sensor unit 110 includes various sensors that detect sensor information used for the processing of the flying object 100, and supplies the detected data to each unit of the flying object 100. In the present embodiment, the sensor unit 110 includes, for example, a plurality of imaging devices 111 and a state sensor 112. The plurality of image pickup devices 111 include, for example, the above-described

image pickup devices

111a, 111b, 111c, and 111d. The state sensor 112 includes, for example, a gyro sensor, an acceleration sensor, a surrounding information detection sensor, a sensor for detecting the motor rotation speed, and the like. The ambient information detection sensor detects, for example, an object around the flying object 100. The ambient information detection sensor includes, for example, an ultrasonic sensor, radar, LiDAR (Light Detection and Ranging, Laser Imaging Detection and Ranging), sonar, and the like. A plurality of state sensors 112 may be provided, like the image pickup device 111, for example.

For example, the sensor unit 110 may include various sensors for detecting the current position of the flying object 100. Specifically, for example, the sensor unit 110 may include a GPS (Global Positioning System) receiver, a GNSS receiver that receives a GNSS signal from a GNSS (Global Navigation Satellite System) satellite, and the like. For example, the sensor unit 110 may include a microphone that collects sounds around the air vehicle 100.

The communication unit 120 communicates with other flying objects different from its own, as well as various external electronic devices, information processing servers, base stations, and the like. The communication unit 120 outputs the data received from the information processing server or the like to the information processing apparatus 10 or transmits the data from the information processing apparatus 10 to the information processing server or the like. The communication protocol supported by the communication unit 120 is not particularly limited, and the communication unit 120 can support a plurality of types of communication protocols.

The drive unit 130 includes various devices related to the drive system of the aircraft 100. The drive unit 130 includes, for example, a drive force generation device for generating drive force of a plurality of drive motors and the like. The drive motor rotates, for example, the rotary wings of the flying object 100. The drive unit 130 rotates the drive motor based on control information including a command from the information processing device 10, for example, so that the flying object 100 floats and flies.

Next, an example of the functional configuration of the information processing device 10 according to the present embodiment will be described. As illustrated in FIG. 2, the information processing device 10 includes an acquisition unit 11, a state recognition unit 12, an environment recognition unit 13, a map creation unit 14, a storage unit 15, a sensor control unit 16, and an operation control unit. 17 is provided. Each functional unit of the acquisition unit 11, the state recognition unit 12, the environment recognition unit 13, the map creation unit 14, the sensor control unit 16, and the operation control unit 17 is provided inside the information processing device 10 by, for example, a CPU or MPU. It is realized by executing the stored program using a RAM or the like as a work area. Further, each processing unit may be realized by an integrated circuit such as an ASIC or FPGA, for example.

The acquisition unit 11 acquires sensor information detected by the sensor unit 110 mounted on the aircraft 100. For example, the acquisition unit 11 acquires sensor information output from each of the four

imaging devices

111a, 111b, 111c and 111d, and the state sensor 112. The acquisition unit 11 outputs the acquired sensor information to the state recognition unit 12, the environment recognition unit 13, the map creation unit 14, the operation control unit 17, and the like. The acquisition unit 11 has a function of changing the detection condition for each sensor of the sensor unit 110, for example. The detection condition includes, for example, the number of times of detection per unit time. For example, the detection condition includes the frame rate of each of the imaging device 111a, the imaging device 111b, the imaging device 111c, and the imaging device 111d. The frame rate of the imaging device 111 means the number of frames processed per unit time in a moving image. The detection condition may include a condition indicating whether to drive each of the plurality of sensors. For example, the acquisition unit 11 changes the detection condition in the sensor unit 110 by outputting information indicating the detection condition to the sensor unit 110. As a result, the acquisition unit 11 can acquire the sensor information detected by each of the plurality of sensors of the sensor unit 110 based on the detection condition. The sensor information may include the detection results of the plurality of sensors or may include only the detection results of each of the plurality of sensors.

The state recognition unit 12 recognizes the state of the flying vehicle 100. The state of the flying object 100 includes, for example, whether or not the vehicle is flying, the speed, the traveling direction A, and the altitude. The state recognition unit 12 recognizes the speed and the traveling direction of the aircraft 100 based on the control information that controls the flight of the aircraft 100. For example, in the case of the air vehicle 100, the information processing apparatus 10 acquires control information such as the direction in which the air vehicle 100 moves, the attitude of the airframe of the aircraft, and the like immediately before regardless of autonomous movement or human operation. be able to. In this case, the state recognition unit 12 can previously recognize the future operation of the aircraft 100 based on the control information. For example, the state recognition unit 12 can acquire the control information from the operation control unit 17 and recognize the state of the aircraft 100 according to the future operation based on the control information. Further, the state recognition unit 12 can recognize the speed and the traveling direction of the flying body 100 that actually flew, based on the sensor information of the state sensor 112. Then, the state recognition unit 12 outputs the recognized result and the like to the sensor control unit 16.

The environment recognition unit 13 recognizes environment information indicating the surroundings of the air vehicle 100 based on the sensor information acquired by the acquisition unit 11. The environment information includes, for example, information indicating the external environment such as the surroundings of the flying vehicle 100 and a specific direction. The environment information includes, for example, the presence or absence of obstacles such as a stationary object and a moving object, and the distance to the object. The environment recognition unit 13 analyzes, for example, sensor information such as an image and a distance image including a depth value, and recognizes a distance to an object in the traveling direction A of the flying object 100 based on the analysis result. Then, the environment recognition unit 13 outputs the recognized result and the like to the sensor control unit 16.

FIG. 3 is a diagram showing an example of recognition of a dynamic object by the information processing device 10 according to the first embodiment. As illustrated in FIG. 3, the environment recognition unit 13 has a function of analyzing an image captured by the imaging device 111, a distance image, and the like to recognize the presence or absence of a dynamic object. The dynamic object includes, for example, a person, an animal, another moving body, and the like. The environment recognition unit 13 recognizes at least the object 200 in the forward direction A of the flying object 100 in the imaging region D1. In the example shown in FIG. 3, the object 200 is a dynamic object, but is not limited thereto.

FIG. 4 is a diagram showing an example of recognition of the external environment of the information processing device 10 according to the first embodiment. In the example illustrated in FIG. 4, the imaging device 111 outputs sensor information including an image G obtained by imaging the imaging region D1 in the traveling direction A of the flying object 100 to the information processing device 10. The environment recognition unit 13 creates the attribute information C such as the semantic map of the image G by executing the semantic segmentation, for example. The attribute information C includes a map displayed so that the user can recognize the attributes of the object shown in the image G. In the example illustrated in FIG. 4, the environment recognition unit 13 determines that the area such as the sky (the sky portion in the image G) in the imaging area D1 indicated by the image G has an object based on the attribute information C more than other areas. Recognize that you are not likely to. Further, the environment recognition unit 13 recognizes that there is no object in the sky region in front of the flying object 100.

Returning to FIG. 2, the map creation unit 14 creates map information around the air vehicle 100 based on the sensor information acquired by the acquisition unit 11. The map information is, for example, information indicating a map of the surrounding environment. The map creating unit 14 creates map information such as a semantic map by executing semantic segmentation, for example. The semantic map is a bird's-eye view map in which the attributes of objects appearing in an image are displayed so that the user can recognize them.

FIG. 5 is a diagram showing an example of map information of the information processing device 10 according to the first embodiment. As shown in FIG. 5, the map creation unit 14 creates map information M indicating a bird's-eye view looking down obliquely from the sky. In this embodiment, the map information M is, for example, a bird's-eye view looking down obliquely from the sky. The map information M includes information indicating a bird's-eye view map to which information (for example, color) indicating the attribute of the object is added.

For example, the map creation unit 14 acquires an image captured by the imaging device 111 of the sensor unit 110. The map creation unit 14 generates a texture map by converting the image into a bird's eye view. Various methods can be used for bird's-eye view conversion. The map creation unit 14 acquires the depth data (sensor information) detected by the imaging device 111. The depth data is data including the distance (depth value) to the object. The map creation unit 14 creates an Occupancy Grid Map based on the depth data and adds semantics to the Occupancy Grid Map to create a semantic map. Semantics are, for example, attributes of objects (meaning image areas) shown in an image. The map creation unit 14 recognizes the attribute of an object (image area) shown in an image by performing semantic segmentation on the image captured by the image capturing device 111. The map creation unit 14 creates map information M indicating the integrated map by integrating (for example, overlaying) the texture map and the semantic map. In the present embodiment, the map information M is a type of a bird's-eye view map that is a three-dimensional Voxel grid expression (an expression that represents a space by a cube). The information processing device 10 generates the map information M so that the information (for example, color or pattern) indicating the attribute of the image area and the three-dimensional shape of the object are included in the map. Then, the map creation unit 14 outputs the created map information M to the sensor control unit 16.

Returning to FIG. 2, the storage unit 15 stores various data and programs. For example, the storage unit 15 can store various kinds of information such as sensor information detected by the sensor unit 110, an action plan, control information of the flying object 100, and map information M. The storage unit 15 can store information such as an action plan received from an external electronic device or the like via the communication unit 120 of the aircraft 100. The storage unit 15 is electrically connected to, for example, the sensor control unit 16, the operation control unit 17, and the like. The storage unit 15 is, for example, a RAM, a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like. The storage unit 15 may be provided in a cloud server connected to the information processing device 10 via the communication unit 120.

The sensor control unit 16 controls the detection conditions of each of the plurality of sensors in the sensor unit 110 based on the control information for controlling the flying object 100 and the environmental information indicating the surroundings of the flying object 100. The sensor control unit 16 has a configuration capable of exchanging information with the acquisition unit 11, the state recognition unit 12, the environment recognition unit 13, the map creation unit 14, the storage unit 15, the operation control unit 17, and the like. The sensor control unit 16 can acquire the recognition result from the state recognition unit 12 and the environment recognition unit 13. The sensor control unit 16 can acquire map information from the map creation unit 14. The sensor control unit 16 can acquire control information from the operation control unit 17.

In the present embodiment, the case where the sensor control unit 16 controls the detection condition via the acquisition unit 11 by instructing the detection condition of each of the plurality of sensors to the acquisition unit 11, will be described. However, the present invention is not limited to this. Not done. For example, the sensor control unit 16 may be configured to directly instruct the sensor unit 110 about the detection conditions of each of the plurality of sensors and control the detection conditions.

For example, the sensor control unit 16 controls the imaging conditions of each of the

imaging devices

111a, 111b, 111c, and 111d based on the control information and the environment information. The sensor control unit 16 controls the detection conditions of each of the plurality of sensors based on the control information, the environment information, and the map information. In the present embodiment, the sensor control unit 16 controls the respective detection conditions of the

imaging devices

111a, 111b, 111c, 111d and the state sensor 112.

The sensor control unit 16 detects the plurality of imaging devices 111, the state sensor 112, and the like based on at least one of the speed and the traveling direction of the aircraft 100 recognized by the state recognition unit 12 and the environment information recognized by the environment recognition unit 13. Control each detection condition. For example, when the flying object 100 is in the moving state, the sensor control unit 16 sets the frame rate of the imaging device 111a that images the traveling direction A of the flying object 100 to the frame rate of the

imaging devices

111b, 111c, and 111d that images the other directions. Increase above the frame rate. The moving state includes, for example, a state in which the aircraft 100 is moving in the air.

The sensor control unit 16 controls to extract an area necessary for detecting an obstacle in the imaging area D1 indicated by the sensor information of the imaging device 111a that images the traveling direction A of the flying object 100. When the sensor information has information indicating the attitude of the flying object 100, the sensor control unit 16 performs control to extract a region necessary for detecting an obstacle based on the attitude.

FIG. 6 is a diagram for explaining a processing example of sensor information of the information processing device 10 according to the first embodiment. For example, the attitude of the flying object 100 may change during flight. In the example illustrated in FIG. 6, when the flying object 100 is flying in the traveling direction A, the flying object 100 changes to the posture P1 and the posture P2. The posture P1 is a posture in which the flying body 100 flies horizontally with respect to a horizontal plane such as the ground. The posture P2 is a posture in which the flying body 100 inclines with respect to the horizontal plane and flies. The posture P2 is a posture in which the image capturing apparatus 111 captures a downward image. In this case, the image captured by the image capturing apparatus 111a of the aircraft 100 differs between the posture P1 and the posture P2. For example, in the case of the posture P1, in the image captured by the image capturing apparatus 111a, the attention area R focused by the information processing apparatus 10 is near the center of the image capturing area D1. Further, in the case of the posture P2, in the image captured by the image capturing apparatus 111a, the attention area R focused by the information processing apparatus 10 is near the upper portion of the image capturing area D1. That is, the information processing apparatus 10 may not need to pay attention to all the images captured by the imaging device 111 when the flying object 100 is flying. Therefore, the information processing device 10 changes the region of interest in the image capturing region D1 captured by the image capturing device 111a, based on the posture P1 and the posture P2 in which the flying object 100 is flying.

In addition, the sensor control unit 16 sets the frame rate of the imaging device 111a that captures the traveling direction A when the object does not exist in the traveling direction A of the flying object 100 so that the object exists in the traveling direction A of the flying object 100. Lower than if. The sensor control unit 16 controls the detection conditions in the sensor unit 110 based on the degree of construction of map information. For example, the detection condition of the sensor of the sensor unit 110 that detects a portion having a high degree of construction of map information is relaxed. When the flying object 100 creates map information, the sensor control unit 16 controls the detection condition such that the frame rate of the imaging device 111 that images the direction required for creating the map information is prioritized. The sensor control unit 16 controls to increase the frame rate of the imaging device 111 when the speed of the flying object 100 increases, and decreases the frame rate of the imaging device 111 when the speed of the flying object 100 decreases.

Returning to FIG. 2, an example in which the sensor control unit 16 determines the frame rates (imaging conditions) of the plurality of imaging devices 111 will be described. For example, the sensor control unit 16 can change the frame rate from the minimum to the maximum according to the speed of the air vehicle 100. The sensor control unit 16 calculates the frame rate FR using the following equation (1).
FR=FR _min +(FR _max- FR _min )*(1-1/(V+1))...Equation (1)

In Expression (1), FR _min indicates the minimum frame rate of the specifications of the image pickup apparatus 111. FR _max indicates the maximum frame rate of the specifications of the image pickup apparatus 111. V indicates the speed of the air vehicle 100. FR _min and FR _max differ depending on the specifications of the imaging device 111 and the like. The sensor control unit 16 uses Expression (1) to change the frame rate FR depending on the traveling speed in the direction in which the imaging device 111 is facing. For example, the sensor control unit 16 increases the frame rate FR of the image pickup device 111 in the traveling direction A of the flying object 100 and lowers or stops the frame rate of the other image pickup devices 111 as compared with the image pickup device 111 in the traveling direction A.

The sensor control unit 16 relaxes the sensor detection conditions of the sensor unit 110 as the degree of construction of the map information created by the map creation unit 14 increases. For example, it is not necessary to image a portion of the map information having a high degree of construction at a high frame rate, and it is not necessary to image a completed portion by the image pickup device 111. The degree of construction of the map information can be obtained, for example, based on the result of comparison between the created map information and the map information stored in the storage unit 15. Specifically, in the case of a map with a three-dimensional Voxel grid representation, the degree of construction of map information can be estimated by focusing on the number of newly created Voxels. The degree of construction of the map information may be estimated based on the coordinates in the space indicated by the map information and the comparison result for each area. The information processing device 10 calculates so as to reduce the frame rate of the imaging device 111 according to the degree of construction of the map information.

For example, the sensor control unit 16 calculates the frame rate FR using the following formula (2) when the map information is the three-dimensional Voxel grid representation.
FR=FR _max *(1-1/(N+1)) equation (2)

In Expression (2), FR _max indicates the maximum frame rate of the specifications of the image pickup apparatus 111. N indicates the number of newly created Voxels. The sensor control unit 16 calculates the frame rate FR of the imaging device 111 by using the equation (2). In addition, when it is known that it is not necessary to process the image captured by the image capturing apparatus 111 from the beginning such as a sky area by semantic segmentation, the sensor control unit 16 lowers the frame rate of the image capturing apparatus 111, or , Stop.

When the flying object 100 is flown to create map information, the sensor control unit 16 controls the frame rate so as to give priority to the imaging device 111 that captures the direction required to create the map information. For example, the sensor control unit 16 preferentially increases the frame rate of the image capturing apparatus 111 that captures the direction necessary to create the map information among the plurality of image capturing apparatuses 111, and lowers the frame rate of the other image capturing apparatuses 111. To do. For example, when creating map information on the left side in the traveling direction A of the air vehicle 100, the information processing apparatus 10 captures the frame rates of the

image capturing apparatuses

111a, 111c, and 111d for capturing the front, rear, and left sides, and the image capturing for the right side. Lower than device 111b. Alternatively, the information processing apparatus 10 sets the frame rate of the image capturing apparatus 111d that captures the left side to a higher frame rate of the front and rear

image capturing apparatuses

111a and 111c, and sets the frame rate of the image capturing apparatus 111b that captures the right side the lowest. May be. Note that the case of setting the frame rate to the lowest includes the case of setting the frame rate to 0 for a predetermined time, for example.

The operation control unit 17 controls the operation of the air vehicle 100 based on the sensor information acquired by the acquisition unit 11. The operation control unit 17 controls the drive unit 130 of the aircraft 100. For example, the operation control unit 17 performs flight control of the flying object 100 to realize the action plan. For example, the operation control unit 17 performs flight control of the flying object 100 for realizing the action plan stored in the storage unit 15. Then, the operation control unit 17 outputs an operation command or the like for driving the flying object 100 to the drive unit 130. The operation control unit 17 may have a function of planning and correcting an action movement plan regarding movement of the flying object 100 based on the sensor information of the sensor unit 110, for example.

The example of the functional configuration of the information processing device 10 according to the present embodiment has been described above. The configuration described above with reference to FIG. 2 is merely an example, and the functional configuration of the information processing device 10 according to the present embodiment is not limited to this example. The functional configuration of the information processing device 10 according to the present embodiment can be flexibly modified according to specifications and operation.

[Procedure of information processing according to the first embodiment]
Next, a procedure of information processing according to the first embodiment will be described with reference to FIG. 7. FIG. 7 is a flowchart showing an example of a processing procedure executed by the information processing apparatus 10 according to the first embodiment. The processing procedure illustrated in FIG. 7 is realized by the information processing device 10 executing a program. The processing procedure shown in FIG. 7 shows an example of the processing procedure for determining the detection conditions of the plurality of imaging devices 111 of the sensor unit 110.

As shown in FIG. 7, the information processing device 10 acquires the sensor information by the acquisition unit 11 (step S101). For example, the information processing device 10 acquires the sensor information of each of the imaging device 111 and the state sensor 112 by the acquisition unit 11. The information processing device 10 acquires control information from the operation control unit 17 (step S102). For example, the information processing device 10 acquires the control information immediately before controlling the operation of the flying object 100, the latest control information, and the like from the operation control unit 17.

The information processing device 10 executes three processes of step S110, step S120, and step S130 in parallel. The information processing device 10 may be configured to sequentially execute the three processes of step S110, step S120, and step S130.

The information processing device 10 recognizes the state of the flying object 100 based on the acquired sensor information (step S110). For example, the information processing device 10 analyzes the sensor information of the state sensor 112 to recognize the states such as the speed, the traveling direction, and the altitude of the flying object 100, and outputs the recognition result to the sensor control unit 16. Further, the information processing apparatus 10 recognizes the state of the flying body 100 such as the speed, the traveling direction, and the altitude based on the control information for controlling the driving unit 130. That is, the information processing device 10 enables prediction of the operation of the flying object 100 by recognizing the current state and the future state of the flying object 100. The information processing device 10 functions as the state recognition unit 12 by executing the process of step S110.

The information processing device 10 calculates the frame rate of each of the plurality of imaging devices based on the state of the flying vehicle 100 and the control information in step S110 (step S111). For example, the information processing device 10 calculates the frame rate of the imaging device 111 that images the traveling direction A of the flying object 100 based on the speed, attitude, and the like of the flying object 100. For example, the information processing apparatus 10 calculates the frame rate of the image capturing apparatus 111 other than the traveling direction A of the flying object 100 so as to be lower than that of the image capturing apparatus 111 capturing the traveling direction A. For example, the information processing apparatus 10 calculates so that the frame rate of the imaging device 111 increases when the speed of the flying object 100 increases, and the frame rate of the imaging device 111 decreases when the speed of the flying object 100 decreases. Then, when the processing of step S111 ends, the information processing apparatus 10 advances the processing to step S141 described below.

Further, the information processing device 10 recognizes the environmental information of the external environment based on the acquired sensor information (step S120). For example, the information processing device 10 analyzes images captured by each of the plurality of imaging devices 111, a distance image, and the like, and recognizes environmental information such as the presence or absence of an obstacle around the flying object 100 and the distance to the obstacle. .. For example, the information processing device 10 analyzes an image captured by the imaging device 111, a distance image, and the like, and recognizes environment information indicating the presence or absence of a dynamic object in the traveling direction A of the air vehicle 100. The information processing device 10 functions as the environment recognition unit 13 by executing the process of step S120.

The information processing device 10 calculates the frame rate of each of the plurality of imaging devices 111 based on the environment information in step S120 (step S121). For example, the information processing device 10 calculates the frame rate of the imaging device 111 based on the presence or absence of a dynamic object, an obstacle, or the like in the traveling direction A of the flying object 100. For example, the information processing apparatus 10 performs calculation so that the frame rate is higher when a dynamic object, an obstacle, or the like in the image capturing areas D1, D2, D3, and D4 captured by the image capturing apparatus 111 is present than when the object is not present. .. For example, since the dynamic object may approach the flying object 100, the information processing apparatus 10 sets the frame rate of the image capturing apparatus 111 capturing the dynamic object to be higher than the frame rates of other image capturing apparatuses 111. You may make it calculate high. When the process of step S111 ends, the information processing device 10 advances the process to step S141 described below.

Further, the information processing device 10 creates map information based on the acquired sensor information (step S130). For example, when the map information being created is stored in the storage unit 15, the information processing device 10 creates the map information by updating the map information based on the sensor information acquired from the sensor unit 110. .. For example, when the map information being created is not stored in the storage unit 15, the information processing device 10 creates new map information based on the sensor information acquired from the sensor unit 110. The information processing device 10 functions as the map creation unit 14 by executing the process of step S130.

The information processing device 10 calculates the frame rate of each of the plurality of imaging devices 111 based on the map information (step S131). For example, the information processing apparatus 10 calculates the image pickup apparatus 111 and its frame rate necessary for creating the map information based on the created map information. For example, the information processing apparatus 10 calculates so that the frame rate of the image capturing apparatus 111 decreases as the degree of construction of map information increases. For example, the information processing apparatus 10 controls the frame rate so that the image capturing apparatus 111 that captures the direction necessary for creating the map information is prioritized. When the process of step S111 ends, the information processing device 10 advances the process to step S141 described below.

The information processing device 10 determines the frame rate of each of the plurality of imaging devices based on the calculated frame rate (step S141). The process of step S141 is executed by the information processing device 10 when the processes of step S111, step S121, and step S131 are completed. For example, the information processing apparatus 10 compares the frame rate calculated from the state of the flying object 100, the frame rate calculated from the environment information, and the frame rate calculated from the degree of construction of the map information with the determination criterion, and the plurality of image pickup apparatuses. Determine the frame rate of 111. In the present embodiment, the case where the criterion is the highest frame rate determined for the imaging device 111 will be described, but the criterion is not limited to this. For example, a criterion may be set according to the state of the air vehicle 100. When the processing of step S141 ends, the information processing apparatus 10 advances the processing to step S142 described below.

The information processing device 10 changes each of the frame rates of the plurality of imaging devices 111 based on the determined frame rate (step S142). For example, the information processing device 10 instructs each of the plurality of imaging devices 111 via the acquisition unit 11 to change the frame rate. As a result, each of the plurality of imaging devices 111 changes the existing frame rate in use to the frame rate instructed to be changed. When the processing of step S142 ends, the information processing apparatus 10 advances the processing to step S150 described below.

The information processing device 10 functions as the sensor control unit 16 by executing the processes of step S111, step S121, step S131, and steps S141 to S142.

The information processing device 10 determines whether or not to end (step S150). For example, the information processing apparatus 10 determines to end when the operation of the flying object 100, a change of the flying object 100 to the standby state, and the like are detected. When the information processing apparatus 10 determines that the processing is not to be ended (No in step S150), the processing is returned to step S101 described above, and the series of processing is executed. If the information processing apparatus 10 determines to end the processing (Yes in step S150), the processing procedure illustrated in FIG. 7 ends.

[Operation of Information Processing Device According to First Embodiment]
Next, an example of the operation of the information processing device 10 according to the first embodiment will be described with reference to FIG. FIG. 8 is a diagram showing an example of determining the frame rate of the information processing device 10 according to the first embodiment.

The flying body 100 is flying in the traveling direction A by driving the driving unit 130 under the control of the information processing device 10. In this case, the information processing device 10 acquires the sensor information including the image captured by the imaging device 111 by the acquisition unit 11. In the information processing device 10, the state recognition unit 12 recognizes the state of the flying object 100, the environment recognition unit 13 recognizes the environment information, and the map creation unit 14 creates the map information.

The information processing device 10 calculates the frame rate for each of the four

image pickup devices

111a, 111b, 111c, and 111d, as shown in FIG. The frame rate calculated from the state of the flying object 100 is calculated as 60 fps (frames per second) for the imaging device 111a that images the front of the flying

object

100 and 0 fps for the

other imaging devices

111b, 111c, and 111d. The frame rate calculated from the environment information is calculated as 20 fps for the

image pickup apparatus

111a and 10 fps for the other

image pickup apparatuses

111b, 111c, and 111d. The frame rate calculated from the degree of construction of the map information is calculated as 20 fps for the

image pickup devices

111a and 111d and 1 fps for the

image pickup devices

111b and 111c. Here, the highest frame rate determined for each imaging device 111 is applied as the determination criterion. In this case, the information processing apparatus 10 changes the frame rate of the image pickup apparatus 111a to 60 fps, the frame rate of the image pickup apparatus 111b to 10 fps, the frame rate of the image pickup apparatus 111c to 10 fps, and the frame rate of the image pickup apparatus 111d to 20 fps. As a result, in the flying object 100, the image pickup device 111a picks up images at the front of the traveling direction A at 60 fps, the

image pickup devices

111b and 111c pick up at 10 fps, and the image pickup devices 111d pick up at 20 fps. Therefore, the flying object 100 can suppress the power consumption of the

imaging devices

111b, 111c, and 111d more than that of the imaging device 111a.

The information processing device 10 acquires from the sensor unit 110 sensor information including images captured at the frame rates of the

image capturing devices

111a, 111b, 111c, and 111d. That is, the information processing apparatus 10 reduces the data amount of the sensor information of the

imaging devices

111b, 111c, and 111d while maintaining the data amount of the sensor information of the imaging device 111a that images the front of the flying vehicle 100. It will be. As a result, the information processing apparatus 10 suppresses the sensor information of the four

image capturing apparatuses

111a, 111b, 111c, and 111d, so that the processing using the sensor information is suppressed and the processing load executed by the information processing apparatus 10 is reduced. Can be mitigated. Then, in the information processing device 10, the operation control unit 17 controls the operation of the air vehicle 100 based on the sensor information acquired from the sensor unit 110 of the air vehicle 100.

As described above, the information processing apparatus 10 according to the present embodiment uses the sensor unit 110 mounted on the air vehicle 100 based on the control information for controlling the air vehicle 100 and the environmental information indicating the surroundings of the air vehicle 100. The detection condition of each of the plurality of sensors is controlled. The information processing device 10 acquires sensor information detected by a plurality of sensors under detection conditions. As a result, the information processing device 10 can detect each of the plurality of sensors of the aircraft 100 under individual detection conditions according to the control information and the environment information of the aircraft 100. As a result, the information processing apparatus 10 can suppress the power consumption of the plurality of sensors mounted on the air vehicle 100 and the data amount of the acquired sensor information.

In addition, the information processing device 10 creates map information of the flying object 100 based on the acquired sensor information, and determines detection conditions for each of the plurality of sensors of the sensor unit 110 based on the control information, the environment information, and the map information. Control. Thereby, the information processing apparatus 10 can image each of the plurality of sensors of the air vehicle 100 under individual detection conditions based on the control information, the environment information, and the map information of the air vehicle 100. As a result, when creating the map information, the information processing apparatus 10 can suppress the power consumption of the plurality of sensors mounted on the air vehicle 100 and suppress the data amount of the acquired sensor information. it can.

Further, the information processing device 10 recognizes at least one of the speed and the traveling direction A of the flying object 100 and the environmental information. The information processing device 10 controls the imaging conditions of each of the plurality of sensors of the sensor unit 110 based on the recognized speed and/or traveling direction A and the environment information. Accordingly, the information processing device 10 can detect each of the plurality of sensors of the flying object 100 under individual detection conditions based on at least one of the speed and the traveling direction of the flying object 100. As a result, the information processing apparatus 10 can suppress the power consumption of the plurality of sensors mounted on the air vehicle 100 without lowering the safety of the air vehicle 100 and reduce the data amount of the sensor information to be acquired. Can be suppressed.

In addition, the information processing device 10 is configured to detect the four

image pickup devices

111a, 111b, 111c, and 111d mounted on the air vehicle 100 based on the control information for controlling the air vehicle 100 and the environmental information indicating the surroundings of the air vehicle 100. The respective imaging conditions are controlled. The information processing device 10 acquires the sensor information detected by the four

imaging devices

111a, 111b, 111c, and 111d under the detection conditions. As a result, the information processing apparatus 10 can image each of the four

imaging devices

111a, 111b, 111c, and 111d of the flying object 100 under individual imaging conditions according to the control information and the environment information of the flying object 100. it can. As a result, the information processing device 10 can suppress the power consumption of the four

imaging devices

111a, 111b, 111c, and 111d mounted on the air vehicle 100, and suppress the data of the acquired sensor information. ..

Further, the information processing apparatus 10 raises the frame rate of the image pickup apparatus 111 that picks up the traveling direction A of the flying object 100 to be higher than the frame rate of the image pickup apparatus 111 that picks up another direction. With this, the information processing apparatus 10 concentrates the image of the traveling direction A of the flying object 100, thereby preventing the safety from being lowered even if the frame rate of the image capturing apparatus 111 in the other direction is suppressed. .. As a result, the information processing device 10 can suppress the power consumption of the plurality of imaging devices 111 without lowering the safety of the flying object 100.

Further, the information processing device 10 controls the imaging conditions of each of the plurality of imaging devices 111 based on the environmental information in the traveling direction A of the flying object 100. As a result, the information processing apparatus 10 determines the imaging conditions of the imaging apparatus 111 that images the traveling direction A and the imaging apparatus 111 that captures the other directions in consideration of the external environment in the flight direction of the flying object 100. It can be controlled separately. As a result, the information processing apparatus 10 can suppress the power consumption of the plurality of imaging devices 111 by changing the imaging condition of the imaging device 111 that captures a direction different from the traveling direction A of the flying object 100.

Further, when there is no object in the traveling direction A of the flying object 100, the information processing apparatus 10 sets the frame rate of the imaging device 111 that captures the traveling direction A in the case where the object exists in the traveling direction A of the flying object 100. Lower than. As a result, the information processing apparatus 10 can change the image capturing condition of the image capturing apparatus 111 that captures the traveling direction A based on the presence/absence of an object in the direction in which the flying object 100 is flying. As a result, the information processing device 10 can suppress the power consumption of the imaging device 111 that images the traveling direction A of the flying object 100.

Further, the information processing apparatus 10 performs control to increase the frame rate of the imaging device 111 when the speed of the flying object 100 increases, and control to decrease the frame rate of the imaging device 111 when the speed of the flying object 100 decreases. To do. As a result, the information processing apparatus 10 can reduce the power consumption of the imaging device 111 by lowering the frame rate of the imaging device 111 when the speed of the flying object 100 decreases. As a result, the information processing device 10 can suppress the power consumption of the plurality of imaging devices 111 without lowering the safety of the flying object 100.

The information processing device 10 also controls the detection condition of the sensor of the sensor unit 110 based on the degree of construction of the map information M. As a result, the information processing apparatus 10 can change the detection condition of the sensor that detects a portion of the created map information having a high degree of construction. As a result, the information processing device 10 can suppress the power consumption of the sensor as the degree of construction of the map information being created increases.

When the aircraft 100 is moved and the map information is created, the information processing apparatus 10 controls the imaging conditions such that the imaging device 111 that takes an image in the direction required to create the map information has priority. As a result, the information processing apparatus 10 can relax the imaging conditions of the imaging apparatus 111 that captures an image in a direction other than the direction required for creating map information. As a result, the information processing device 10 can suppress the power consumption of the imaging device 1111 that is not necessary for creating the map information without reducing the efficiency of creating the map information.

In the information processing device 10, the operation control unit 17 controls the operation of the aircraft 100 based on the sensor information acquired by the acquisition unit 11 from the sensor unit 110 of the aircraft 100. As a result, the information processing device 10 acquires the sensor information detected by the sensor of the sensor unit 110 under the individual detection conditions according to the control information and the environment information of the flying object 100, and based on the sensor information, the flying object 100. Can be controlled. As a result, the information processing device 10 can suppress the power consumption of the flying body 100 that controls flight by controlling the detection conditions of the plurality of sensors mounted on the flying body 100.

The above-described first embodiment is an example, and various modifications and applications are possible.

[Modification (1) of the first embodiment]
For example, the information processing device 10 according to the first embodiment can change the imaging condition of the imaging device 111 included in the sensor unit 110 of the flying object 100.

The information processing device 10 according to the modified example (1) of the first embodiment is based on the attribute information of the imaging region D1 indicated by the sensor information of the imaging device 111 whose sensor control unit 16 images the traveling direction A of the flying object 100. The image capturing conditions of the image capturing device 111 may be controlled. For example, as illustrated in FIG. 4 described above, the sensor control unit 16 of the information processing device 10 executes the semantic segmentation of the image G included in the sensor information of the imaging device 111 acquired by the acquisition unit 11 to obtain the image. It is possible to classify the area in the imaging area indicated by G. Then, when the classified area includes an area having an attribute in which there is a high possibility that an object does not exist in the traveling direction A, the sensor control unit 16 performs control for lowering the frame rate of the imaging device 111. For example, when the area in the traveling direction A indicated by the image G has the sky attribute, the sensor control unit 16 can lower the frame rate of the image pickup apparatus 111 below the frame rate at the time of detecting an object.

As described above, the information processing apparatus 10 according to the present embodiment sets the image capturing condition of the image capturing apparatus 111 based on the attribute information of the image capturing area D1 indicated by the sensor information of the image capturing apparatus 111 that captures the traveling direction A of the flying object 100. Control. Thereby, the information processing device 10 can change the imaging condition of the imaging device 111 in the traveling direction A of the flying object 100 based on the possibility that an object exists. As a result, the information processing device 10 can suppress the power consumption of the aircraft 100 without lowering the safety of the aircraft 100 in the traveling direction A.

Note that the modification (1) of the first embodiment may be applied to the information processing device 10 of another embodiment or modification.

[Modification (2) of the first embodiment]
For example, the information processing apparatus 10 according to the first embodiment can change the imaging condition of the imaging device 111 included in the sensor unit 110 of the flying object 100 according to the attitude of the flying object 100.

In the information processing device 10 according to the modified example (2) of the first embodiment, when the sensor information acquired by the acquisition unit 11 has the information indicating the attitude of the flying object 100, the sensor control unit 16 causes the attitude. The control for changing the image pickup area of the image pickup apparatus 111 necessary for detecting the obstacle may be performed based on the above.

For example, as shown in FIG. 6 above, the sensor control unit 16 of the information processing device 10 detects that the sensor information of the imaging device 111 acquired by the acquisition unit 11 is the attitude P1 in which the flying object 100 flies without tilting. In the image that is included, the attention area R is near the center of the imaging area D1. Then, the sensor control unit 16 causes the environment recognition unit 13 to execute recognition processing based on the attention area R. Further, in the case where the flying body 100 tilts and flies in the attitude P2, the sensor control unit 16 sets the attention area R near the upper portion of the imaging area D1 in the image included in the sensor information of the imaging device 111 acquired by the acquisition unit 11. .. Then, the sensor control unit 16 causes the environment recognition unit 13 to execute recognition processing based on the attention area R. As a result, the environment recognition unit 13 does not need to analyze the entire image included in the sensor information, so that it is possible to reduce processing such as image analysis and image recognition.

As described above, when the sensor information has the information indicating the attitude of the flying object 100, the information processing apparatus 10 according to the present embodiment detects the obstacle of the image pickup apparatus 111 based on the attitude. Control for changing the imaging area is performed. As a result, the information processing apparatus 10 can change the image capturing area of the image capturing apparatus 111 necessary for detecting the obstacle according to the change in the attitude of the flying object 100. As a result, the information processing apparatus 10 may process the information of a partial area of the imaging area according to the attitude of the flying object 100 during flight, so that the processing load can be reduced.

The information processing apparatus 10 according to the modified example (2) of the first embodiment has been described as a case of focusing on the image area of the acquired sensor information as the control for changing the imaging area of the imaging apparatus 111, but the present invention is not limited to this. Not done. For example, the sensor control unit 16 of the information processing device 10 acquires only the information of the imaging region of the imaging device 111 necessary for detecting the obstacle from the image of the sensor information acquired from the sensor unit 110, the acquisition unit 11, and the environment recognition unit 13. May be extracted. For example, the sensor control unit 16 of the information processing device 10 may be configured to cause the sensor unit 110 to output sensor information indicating only the information of the imaging region of the imaging device 111 necessary for detecting an obstacle.

Note that the modified example (2) of the first embodiment may be applied to the information processing device 10 of other embodiments and modified examples.

(Second embodiment)
[Example of configuration of information processing apparatus according to second embodiment]
Next, a second embodiment will be described. FIG. 9 is a diagram illustrating an example of the configuration of the information processing device 10A according to the second embodiment. As shown in FIG. 9, the aircraft 100 includes a sensor unit 110, a communication unit 120, a drive unit 130, and an information processing device 10A. The sensor unit 110, the communication unit 120, the drive unit 130, and the information processing device 10A are connected to each other so that data and signals can be exchanged.

The information processing device 10A includes an acquisition unit 11, a state recognition unit 12, an environment recognition unit 13, a storage unit 15, a sensor control unit 16, and an operation control unit 17. That is, the information processing apparatus 10A does not include the map creation unit 14 of the information processing apparatus 10 according to the first embodiment. For each functional unit of the acquisition unit 11, the state recognition unit 12, the environment recognition unit 13, the sensor control unit 16, and the operation control unit 17, for example, a program stored inside the information processing device 10 is executed by the CPU, MPU, or the like. It is realized by executing the RAM or the like as a work area. Further, each processing unit may be realized by an integrated circuit such as an ASIC or FPGA, for example.

[Procedure of information processing according to the second embodiment]
Next, a procedure of information processing according to the second embodiment will be described with reference to FIG. FIG. 10 is a flowchart showing an example of a processing procedure executed by the information processing device 10 according to the second embodiment. The processing procedure illustrated in FIG. 10 is realized by the information processing device 10 executing a program. The processing procedure shown in FIG. 10 shows an example of the processing procedure for determining the detection conditions of the plurality of imaging devices 111 of the sensor unit 110.

As shown in FIG. 10, the information processing apparatus 10A acquires sensor information by the acquisition unit 11 (step S101). The information processing apparatus 10A acquires control information from the operation control unit 17 (step S102).

The information processing apparatus 10A executes two processes of step S110 and step S120 in parallel. The information processing device 10 may be configured to sequentially execute the two processes of step S110 and step S120.

The information processing device 10A recognizes the state of the flying object 100 based on the acquired sensor information (step S110). When the process of step S110 ends, the information processing device 10A advances the process to step S161. Further, the information processing device 10A recognizes the environmental information of the external environment based on the acquired sensor information (step S120). When the process of step S120 ends, the information processing apparatus 10A advances the process to step S161.

The information processing apparatus 10A determines the detection conditions of each of the plurality of sensors of the sensor unit 110 based on the state of the flying object 100, control information, and environment information (step S161). The process of step S161 is executed by the information processing apparatus 10A when the processes of step S110 and step S120 are completed. For example, the information processing apparatus 10A can detect an object among a plurality of sensors based on the current speed of the air vehicle 100, the traveling direction A, and the current speed of the air vehicle 100, the traveling direction A, and the like. The sensor to be used is specified, and the detection condition of the specified sensor is determined. For example, the information processing apparatus 10A determines a sensor to be operated and a sensor to be operated or stopped with power saving. When the process of step S161 ends, the information processing device 10A advances the process to step S162.

The information processing device 10A changes the detection condition of each of the plurality of sensors based on the determined detection condition (step S162). For example, the information processing device 10 instructs each of the plurality of sensors of the sensor unit 110 to change the detection condition via the acquisition unit 11. As a result, each of the imaging device 111, the state sensor 112, and the like of the sensor unit 110 changes the existing detection condition in use to the detection condition instructed to be changed. When the process of step S162 ends, the information processing device 10 advances the process to step S150 described below.

Note that the information processing device 10A functions as the sensor control unit 16 by executing the processes of step S161 and step S162.

The information processing device 10A determines whether or not to end (step S150). When the information processing apparatus 10A determines that the processing is not to be ended (No in step S150), the processing is returned to step S101 described above, and the series of processing is executed. If the information processing apparatus 10A determines to end the processing (Yes in step S150), the information processing apparatus 10A ends the processing procedure illustrated in FIG.

[Operation of Information Processing Device According to Second Embodiment]
Next, an example of the operation of the information processing device 10A according to the second embodiment will be described with reference to FIG. FIG. 11 is a diagram for explaining an example of determining the detection condition of the sensor of the information processing apparatus 10A according to the second embodiment.

In the scene SN1 shown in FIG. 11, the flying body 100 is flying in the traveling direction A by driving the driving unit 130 under the control of the information processing device 10A. In this case, the information processing apparatus 10A causes the image pickup apparatus 111a to function as a sensor used for detecting an object, and causes the other sensors to operate with low power consumption. Then, the information processing apparatus 10</b>A recognizes that the traveling direction A is changed to the traveling direction A′ based on the action plan of the aircraft 100 immediately before controlling the driving unit 130. In this case, the information processing apparatus 10A recognizes that it is necessary to recognize an object in the front and right directions of the air vehicle 100 based on the future speed and the traveling direction A′ of the air vehicle 100. As a result, the information processing apparatus 10A uses the detection conditions of the sensors for detecting the front and right sides of the sensor unit 110 for detecting an object based on the control information for controlling the moving body and the environment information indicating the surroundings of the moving body. And the detection conditions for operating the other sensors with power saving. Then, the information processing device 10A changes the detection condition of each of the plurality of sensors based on the determined detection condition.

In the scene SN2 shown in FIG. 11, the flying body 100 is flying from the traveling direction A to the traveling direction A′ by the driving unit 130 being driven by the control of the information processing device 10A. In this case, the information processing apparatus 10A causes the image pickup apparatus 111a and the image pickup apparatus 111b to function as sensors used for detecting an object, and causes the other sensors to operate with low power consumption.

As described above, the information processing apparatus 10A according to the present embodiment uses the sensor unit 110 mounted on the aircraft 100 based on the control information for controlling the aircraft 100 and the environmental information indicating the surroundings of the aircraft 100. The detection condition of each of the plurality of sensors is controlled. The information processing apparatus 10A acquires sensor information detected by a plurality of sensors under the detection conditions. As a result, the information processing apparatus 10A can detect each of the plurality of sensors of the air vehicle 100 under the individual detection conditions according to the control information and the environment information of the air vehicle 100. As a result, the information processing apparatus 10A can suppress the power consumption of the plurality of sensors mounted on the air vehicle 100 and the data amount of the acquired sensor information.

Further, in the information processing device 10A, the control information is information for controlling the flying body 100 based on the action plan of the flying body 100. Accordingly, the information processing apparatus 10A can control the detection conditions of the plurality of sensors of the sensor unit 110 based on the control information immediately before controlling the flying object 100. As a result, since the information processing apparatus 10A can change the detection condition of the sensor immediately before the operation of the flying object 100, the power consumption of the flying object 100 can be further suppressed. In addition, the information processing apparatus 10A can perform control without lowering the safety of the flying object 100 by detecting the sensor under the detection condition suitable for the operation of the flying object 100.

In the above-described present embodiment, the case where the information processing devices 10 and 10A are mounted on the flying body 100 has been described, but the present invention is not limited to this. For example, the information processing devices 10 and 10A are provided so as to be able to communicate with each other outside the flying body 100, acquire sensor information by communicating with the flying body 100 via the communication unit 120, and detect a plurality of sensors of the flying body 100. The condition may be controlled remotely.

In the above embodiment, the case where the moving body is the flying body 100 has been described, but the moving body is not limited to this. The moving body may be, for example, an autonomous moving type moving body that needs to monitor the surroundings of the vehicle (a motorcycle, a four-wheeled vehicle, a bicycle, etc.), a robot, a dolly, or the like.

[Hardware configuration]
The information processing apparatus according to the above-described embodiment is realized by, for example, a computer 1000 having a configuration shown in FIG. Hereinafter, the information processing apparatus 10 according to the embodiment will be described as an example. FIG. 12 is a hardware configuration diagram illustrating an example of a computer 1000 that realizes the functions of the information processing device 10. The computer 1000 includes a CPU 1100, a RAM 1200, a ROM (Read Only Memory) 1300, an HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600. The respective units of the computer 1000 are connected by a bus 1050.

The CPU 1100 operates based on a program stored in the ROM 1300 or the HDD 1400, and controls each part. For example, the CPU 1100 expands a program stored in the ROM 1300 or the HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

The ROM 1300 stores a boot program such as a BIOS (Basic Input Output System) executed by the CPU 1100 when the computer 1000 is started, a program dependent on the hardware of the computer 1000, and the like.

The HDD 1400 is a computer-readable recording medium that non-temporarily records a program executed by the CPU 1100, data used by the program, and the like. Specifically, the HDD 1400 is a recording medium that records a program according to the present disclosure, which is an example of the program data 1450.

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (for example, the Internet). For example, the CPU 1100 receives data from another device or transmits the data generated by the CPU 1100 to another device via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display, a speaker, a printer, etc. via the input/output interface 1600. The input/output interface 1600 may also function as a media interface for reading a program or the like recorded in a predetermined recording medium (medium). The medium is, for example, an optical recording medium such as a DVD (Digital Versatile Disc), a magneto-optical recording medium such as an MO (Magneto-Optical disk), a tape medium, a magnetic recording medium, or a semiconductor memory.

For example, when the computer 1000 functions as the information processing apparatus 10 according to the embodiment, the CPU 1100 of the computer 1000 executes the program loaded on the RAM 1200 to acquire the acquisition unit 11, the state recognition unit 12, and the environment recognition unit 13. The functions of the map creation unit 14, the sensor control unit 16, the operation control unit 17, and the like are realized. Further, the HDD 1400 stores the program according to the present disclosure and the data in the storage unit 15. Note that the CPU 1100 reads the program data 1450 from the HDD 1400 and executes the program data, but as another example, these programs may be acquired from another device via the external network 1550.

The preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is obvious that a person having ordinary knowledge in the technical field of the present disclosure can come up with various changes or modifications within the scope of the technical idea described in the claims. Of course, it is understood that the invention also belongs to the technical scope of the present disclosure.

Also, the effects described in the present specification are merely explanatory or exemplifying ones, and are not limiting. That is, the technique according to the present disclosure may have other effects that are apparent to those skilled in the art from the description of the present specification, in addition to or instead of the above effects.

Further, it is also possible to create a program for causing hardware such as a CPU, a ROM, and a RAM included in the computer to exhibit a function equivalent to the configuration of the information processing apparatus 10, and the program is recorded and read by the computer. Possible recording media may also be provided.

Also, each step related to the processing of the information processing apparatus in this specification does not necessarily have to be processed in time series in the order described in the flowchart. For example, the steps related to the processing of the information processing device may be processed in a different order from the order described in the flowchart or may be processed in parallel.

The following configurations also belong to the technical scope of the present disclosure.
(1)
Based on control information for controlling the moving body and environmental information indicating the surroundings of the moving body, a sensor control unit for controlling the detection conditions of each of the plurality of sensors mounted on the moving body,
An acquisition unit that acquires sensor information that each of the plurality of sensors has detected under the detection conditions,
An information processing apparatus including.
(2)
Based on the sensor information acquired by the acquisition unit, further comprising a creation unit that creates map information around the moving body,
The information processing device according to (1), wherein the sensor control unit controls the detection condition of each of the plurality of sensors based on the control information, the environment information, and the map information.
(3)
A state recognition unit that recognizes at least one of the speed and the traveling direction of the moving body based on the control information,
An environment recognition unit that recognizes the environment information based on the sensor information acquired by the acquisition unit,
Further equipped with,
The sensor control unit, based on the environmental information recognized by the environment recognition unit and at least one of the speed and traveling direction of the moving body recognized by the state recognition unit, the detection conditions of each of the plurality of sensors. Controlling The information processing apparatus according to (2) above.
(4)
The plurality of sensors include a plurality of imaging devices provided to image different regions around the moving body,
The information processing device according to (2) or (3), wherein the sensor control unit controls the imaging conditions of each of the plurality of imaging devices based on the control information and the environment information.
(5)
The information processing device according to (4), wherein the sensor control unit raises a frame rate of the imaging device that images the traveling direction of the moving body higher than a frame rate of the imaging device that images another direction.
(6)
The information processing device according to (4) or (5), wherein the sensor control unit controls the imaging condition of each of the plurality of imaging devices based on the environment information in the traveling direction of the moving body.
(7)
When the object does not exist in the traveling direction of the moving body, the sensor control unit lowers the frame rate of the imaging device that captures the moving direction than when the object exists in the traveling direction of the moving body. The information processing device according to any one of (4) to (6).
(8)
The sensor control unit performs control to increase the frame rate of the imaging device when the speed of the moving object increases, and performs control to decrease the frame rate of the imaging device when the speed of the moving object decreases. The information processing device according to any one of (4) to (7).
(9)
The information processing apparatus according to (2), wherein the sensor control unit controls the detection conditions of the plurality of sensors based on the degree of construction of the map information.
(10)
When the moving object is moved to create the map information, the sensor control unit controls the imaging condition so as to give priority to the imaging device that images a direction necessary for creating the map information. ) The information processing device described in.
(11)
The information according to (4), wherein the sensor control unit controls the image capturing condition of the image capturing device based on attribute information of an image capturing area indicated by the sensor information of the image capturing device capturing the traveling direction of the moving body. Processing equipment.
(12)
When the sensor information has information indicating the posture of the moving body, the sensor control unit performs control to change an image pickup area of the image pickup apparatus necessary for detecting an obstacle based on the posture. The information processing device according to (6).
(13)
The information processing apparatus according to any one of (1) to (12), further including an operation control unit that controls an operation of the moving body based on the sensor information acquired by the acquisition unit.
(14)
The said control information is information for controlling the said mobile body based on the action plan of the said mobile body, The information processing apparatus in any one of said (1) to (13).
(15)
Computer
Based on the control information for controlling the moving body and the environmental information indicating the surroundings of the moving body, controlling the respective detection conditions of the plurality of sensors mounted on the moving body,
An information processing method, wherein each of the plurality of sensors acquires sensor information detected under the detection condition.
(16)
Computer,
A sensor control unit that controls respective detection conditions of a plurality of sensors mounted on the moving body, based on control information for controlling the moving body and environment information indicating the periphery of the moving body,
A program for causing each of the plurality of sensors to function as an acquisition unit that acquires sensor information detected under the detection conditions.
(17)
A mobile body including a plurality of sensors and an information processing device,
The information processing device,
Based on control information for controlling the moving body and environmental information indicating the surroundings of the moving body, a sensor control unit for controlling the detection conditions of each of the plurality of sensors,
An acquisition unit that acquires sensor information that each of the plurality of sensors has detected under the detection conditions,
A mobile body equipped with.

10, 10A Information processing device 11 Acquisition unit 12 State recognition unit 13 Environment recognition unit 14 Map creation unit 15 Storage unit 16 Sensor control unit 17 Motion control unit 100 Aircraft 110 Sensor unit 111 Imaging device

111a Imaging device

111b Imaging device

111c Imaging device

111d imaging device 120 communication unit 130 drive unit A traveling direction

Claims

Based on control information for controlling the moving body and environmental information indicating the surroundings of the moving body, a sensor control unit for controlling the detection conditions of each of the plurality of sensors mounted on the moving body,
An acquisition unit that acquires sensor information that each of the plurality of sensors has detected under the detection conditions,
An information processing apparatus including.
Based on the sensor information acquired by the acquisition unit, further comprising a map creation unit that creates map information around the moving body,
The information processing device according to claim 1, wherein the sensor control unit controls the detection condition of each of the plurality of sensors based on the control information, the environment information, and the map information.
A state recognition unit that recognizes at least one of the speed and the traveling direction of the moving body based on the control information,
An environment recognition unit that recognizes the environment information based on the sensor information acquired by the acquisition unit,
Further equipped with,
The sensor control unit, based on at least one of the speed and traveling direction of the moving body recognized by the state recognition unit and the environment information recognized by the environment recognition unit, the detection condition of each of the plurality of sensors. The information processing device according to claim 2.
The plurality of sensors include a plurality of imaging devices provided to image different regions around the moving body,
The information processing device according to claim 2, wherein the sensor control unit controls an imaging condition of each of the plurality of imaging devices based on the control information and the environment information.
The information processing device according to claim 4, wherein the sensor control unit raises a frame rate of the imaging device that images the traveling direction of the moving body higher than a frame rate of the imaging device that images another direction.
The information processing device according to claim 4, wherein the sensor control unit controls the imaging conditions of each of the plurality of imaging devices based on the environment information in the traveling direction of the moving body.
When the object does not exist in the traveling direction of the moving body, the sensor control unit lowers the frame rate of the imaging device that captures the moving direction than when the object exists in the traveling direction of the moving body. Item 4. The information processing device according to item 4.
The sensor control unit performs control to increase the frame rate of the imaging device when the speed of the moving body increases, and performs control to decrease the frame rate of the imaging device when the speed of the moving body decreases. The information processing apparatus according to claim 4.
The information processing device according to claim 2, wherein the sensor control unit controls the detection conditions of the plurality of sensors based on a degree of construction of the map information.
The sensor control unit controls the imaging condition such that when the moving body is moved to create the map information, the imaging device that takes an image in a direction required to create the map information is prioritized. The information processing device according to 1.
The information processing according to claim 4, wherein the sensor control unit controls the imaging condition of the imaging device based on attribute information of an imaging region indicated by the sensor information of the imaging device that images the traveling direction of the moving body. apparatus.
When the sensor information has information indicating the posture of the moving body, the sensor control unit performs control to change the image pickup area of the image pickup apparatus necessary for detecting an obstacle based on the posture. Item 6. The information processing device according to item 6.
The information processing apparatus according to claim 1, further comprising an operation control unit that controls an operation of the moving body based on the sensor information acquired by the acquisition unit.
The information processing apparatus according to claim 1, wherein the control information is information for controlling the moving body based on an action plan of the moving body.
Computer
Based on the control information for controlling the moving body and the environmental information indicating the surroundings of the moving body, controlling the respective detection conditions of the plurality of sensors mounted on the moving body,
An information processing method, wherein each of the plurality of sensors acquires sensor information detected under the detection condition.
Computer,
A sensor control unit that controls respective detection conditions of a plurality of sensors mounted on the moving body, based on control information for controlling the moving body and environment information indicating the periphery of the moving body,
A program for causing each of the plurality of sensors to function as an acquisition unit that acquires sensor information detected under the detection conditions.