WO2024034284A1

WO2024034284A1 - Sensor system and movable body system

Info

Publication number: WO2024034284A1
Application number: PCT/JP2023/024174
Authority: WO
Inventors: 幹太佐藤; 達治芦谷
Original assignee: ソニーセミコンダクタソリューションズ株式会社
Priority date: 2022-08-10
Filing date: 2023-06-29
Publication date: 2024-02-15
Also published as: JP2024025238A

Abstract

[Problem] To provide an imaging device and a movable body system that enable vicinity information for a wide range to be acquired at a low cost. [Solution] A sensor system according to the present embodiment comprises: a rotating body that is capable of rotating about a first axis; a rotating-body control unit that controls the rotating body; a sensor that is disposed on the rotating body and acquires vicinity information; a sensor control unit that controls the sensor on the basis of the control timing of the rotating-body control unit; and a signal processing unit that processes the vicinity information acquired by the sensor.

Description

Sensor systems and mobile systems

The present disclosure relates to a sensor system and a mobile system.

Autonomous mobile objects such as AGVs (Automatic Guided Vehicles) or AMRs (Autonomous Mobile Robots) may have imaging sensors such as ToF (Time of Fight) sensors to obtain surrounding environmental information.

However, the position where the image sensor is placed on the moving body is limited, and the imaging range of the image sensor is limited. Furthermore, in order to widen the imaging range, it was necessary to mount a plurality of imaging sensors on the moving object. In this case, there are concerns about an increase in cost and an increase in the amount of image data.

Japanese Patent Application Publication No. 2021-148746 JP2021-105611A

To provide an imaging device and a mobile system that enable acquisition of a wide range of surrounding information at low cost.

A sensor system according to an aspect of the present disclosure includes a rotating body that is rotatable about a first axis, a rotating body control unit that controls the rotating body, a sensor that is arranged on the rotating body and acquires surrounding information, and a rotating body control unit that controls the rotating body. The sensor includes a sensor control section that controls the sensor based on control timing of the sensor, and a signal processing section that processes surrounding information acquired by the sensor.

When the rotating body control unit rotates the rotating body, the sensor control unit stops acquiring surrounding information of the sensor, and when the rotating body control unit stops rotating the rotating body, the sensor control unit stops acquiring the surrounding information of the sensor. Start acquiring surrounding information.

When the sensor stops acquiring surrounding information, the signal processing unit transmits a permission signal that enables rotation of the rotating body to the rotating body control unit.

The signal processing unit acquires an image or distance of an object in front of the sensor using surrounding information, and the rotating body control unit rotates the rotating body based on the image or distance of the object.

The signal processing unit detects the turning direction of the moving body based on the movement of the entire surrounding information, and the rotating body control unit rotates the rotating body in a direction opposite to the turning direction.

The signal processing unit calculates the rotation angle of the rotating body based on the movement of the entire surrounding information, and the rotating body control unit rotates the rotating body by the rotation angle in a direction opposite to the rotation direction.

The sensor includes CIS or ToF.

A mobile system according to one aspect of the present disclosure includes: a rotating body that is rotatable about a first axis; a rotating body control unit that controls the rotating body; and a sensor that is disposed on the rotating body and acquires surrounding information. A mobile body equipped with a sensor system including a sensor control unit that controls the sensor based on control timing of the rotating body control unit, and a signal processing unit that processes the surrounding information acquired by the sensor; A mobile body control section that controls the mobile body is provided.

When the moving body control unit turns the moving body, the rotating body control unit rotates the rotating body in the turning direction of the moving body before turning the moving body, and the sensor rotates the rotating body in the turning direction. The signal processing unit outputs a permission signal that enables the mobile body to turn based on the surrounding information to the mobile body control unit, and the mobile body control unit causes the mobile body to turn.

When the moving body control section rotates the moving body by the first angle, the rotating body control section rotates the rotating body by the first angle in the turning direction of the moving body.

When the moving body control unit turns the moving body, the rotating body control unit rotates the rotating body in a direction opposite to the turning direction of the moving body almost simultaneously with the turning of the moving body.

When the moving body control section rotates the moving body by the first angle, the rotating body control section rotates the rotating body by the first angle in a direction opposite to the turning direction of the moving body.

The sensor includes a CIS, ToF or stereo camera.

FIG. 1 is a schematic side view showing a configuration example of a mobile system according to a first embodiment. FIG. 1 is a schematic plan view showing a configuration example of a mobile system according to a first embodiment. A conceptual diagram showing the operation of the sensor system. The figure which shows the example of a structure of a sensor system. FIG. 2 is a block diagram showing a configuration example of a sensor system. FIG. 3 is a flow diagram showing an example of the operation of the sensor system according to the first embodiment. FIG. 7 is a conceptual diagram showing an example of the operation of the sensor system according to the second embodiment. FIG. 2 is a block diagram showing a configuration example of a sensor system according to a second embodiment. FIG. 7 is a flow diagram showing an example of the operation of the second embodiment. FIG. 7 is a conceptual diagram showing an example of the operation of the sensor system according to the third embodiment. FIG. 7 is a conceptual diagram showing an example of the operation of the sensor system according to the third embodiment. FIG. 7 is a block diagram showing a configuration example of a sensor system according to a third embodiment. FIG. 7 is a flowchart showing an example of the operation of the third embodiment. FIG. 7 is a conceptual diagram showing an example of the operation of the mobile system according to the fourth embodiment. FIG. 7 is a conceptual diagram showing an example of the operation of the mobile system according to the fourth embodiment. FIG. 7 is a block diagram showing a configuration example of a mobile system according to a fourth embodiment. FIG. 7 is a flowchart showing an example of the operation of the fourth embodiment. FIG. 7 is a flowchart showing an example of the operation of the fifth embodiment. FIG. 1 is a block diagram showing a schematic configuration example of a vehicle control system. The figure which shows the example of the installation position of an imaging part.

Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings. The drawings are schematic or conceptual, and the proportions of each part are not necessarily the same as in reality. In the specification and drawings, the same elements as those described above with respect to the existing drawings are denoted by the same reference numerals, and detailed description thereof will be omitted as appropriate.

(First embodiment)
FIG. 1 is a schematic side view showing a configuration example of a mobile system 1 according to the first embodiment. FIG. 2 is a schematic plan view showing a configuration example of the mobile system 1 according to the first embodiment.

The mobile system 1 is, for example, an autonomous mobile such as an AGV or an AMR, and is capable of autonomously traveling in a distribution warehouse, factory, or the like. Furthermore, the mobile system 1 is also applicable to automobiles, airplanes, drones, and the like. The mobile system 1 includes a sensor system 2 and a mobile body 3. The moving body 3 is configured to be movable on the floor, the ground, space, etc., and is equipped with the sensor system 2. The moving body 3 has a box-shaped main body, and can be loaded with luggage and the like and moved. The mobile object 3 can autonomously travel based on surrounding information detected by the sensor system 2. Note that the moving body 3 may be a moving body that does not operate autonomously.

The sensor system 2 may be a 3D sensing device, such as an image sensor such as CIS (CMOS (Complementary Metal Oxide Semiconductor) Image Sensor), a distance measurement sensor such as ToF (Time of Fright), a stereo camera, Lider, etc. , or a combination of these. The sensor system 2 is placed at the end of the main body of the moving body 3 so as not to interfere with luggage or the like.

FIG. 3 is a conceptual diagram showing the operation of the sensor system 2. The sensor system 2 is attached to the moving body 3 so as to be rotatable about the Z axis as a first axis. For example, the sensor system 2 detects the moving direction (X direction) of the moving object 3, and when the moving object 3 turns in the ±Y direction with respect to the moving direction, the sensor system 2 rotates by an angle θ in the ±Y direction. to detect surrounding information. The surrounding information may be, for example, image information or distance measurement information around the mobile system 1.

FIG. 4 is a diagram showing a configuration example of the sensor system 2. The sensor system 2 includes a sensor section 20 and a rotating section 30. The rotating section 30 includes a base body 34 and a rotating body 35. The base body 34 is fixed to the movable body 3 and rotates a rotating body 35 around the Z axis. The rotating body 35 extends from the base body 34 in the Z direction, and is configured to be rotatable about the Z axis with respect to the base body 34 . The sensor section 20 is disposed on the rotating body 35 and is rotatable about the Z-axis together with the rotating body 35. The sensor unit 20 can acquire surrounding information.

FIG. 5 is a block diagram showing a configuration example of the sensor system 2. The sensor section 20 includes a sensor 21, a sensor control section 22, and a signal processing section 23. The rotating unit 30 includes a rotating body control unit 31, a rotating body motor 32, and a rotating body 35. Note that illustration of the base body 34 is omitted in FIG. 5.

The sensor control unit 22 transmits an imaging control signal to the sensor 21 while being synchronized with the rotating body control unit 31 of the rotating unit 30 using a synchronization signal. That is, the sensor control section 22 controls the sensor 21 based on the control timing of the rotating body control section 31. The sensor 21 acquires surrounding information according to an imaging control signal from the sensor control unit 22. The sensor 21 may be, for example, a CIS chip or a ToF chip. The signal processing section 23 includes an image generation section 24 and a timing control section 25. The image generation unit 24 processes imaging data as surrounding information acquired by the sensor 21 and generates an image of the surroundings of the moving body 3. When the timing control unit 25 finishes acquiring the imaging data, it transmits a synchronization signal indicating the end of imaging to the rotating body control unit 31 of the rotating unit 30. Thereby, the sensor section 20 can start or stop imaging at the timing of the rotation of the rotating body 35 (that is, the rotation of the sensor section 20) in synchronization with the rotating section 30. Further, the rotating unit 30 can start or stop the rotation of the rotating body 35 (that is, the rotation of the sensor unit 20) at the timing of imaging in synchronization with the sensor unit 20. Note that the sensor 21, the sensor control section 22, and the signal processing section 23 of the sensor section 20 may be composed of one semiconductor chip or may be composed of a plurality of semiconductor chips. When configured with a plurality of semiconductor chips, the plurality of semiconductor chips may be stacked and the wirings may be bonded to each other (Cu--Cu bonding).

The rotating body control unit 31 transmits a rotating body control signal and an angle signal to the rotating body motor 32, and controls the rotating body motor 32 and the rotating body 35. The rotating body motor 32 is controlled by a rotating body control signal from the rotating body control section 31, and rotates the rotating body 35 to an angle based on the angle signal. The rotating body control signal indicates, for example, the rotation direction, rotation speed, rotation start timing, etc. of the rotating body 35. The angle signal indicates the rotation angle of the rotating body 35, and indicates, for example, the angle θ in FIG. The rotating body control unit 31 synchronizes with the sensor control unit 22 and the signal processing unit 23, for example, when the sensor control unit 22 stops imaging in the sensor 21 and the signal processing unit 23 finishes generating an image. Then, the rotation of the rotating body 35 can be started.

Next, the operation of the sensor system 2 according to this embodiment will be explained. Note that the sensor system 2 will be described below as an imaging system that captures images of the surroundings. However, the sensor system 2 may also be a ranging system that measures distances of surrounding objects.

FIG. 6 is a flow diagram showing an example of the operation of the sensor system 2 according to the first embodiment. In this embodiment, after the rotating body 35 rotates and stops, the sensor section 20 images the surroundings thereof. After the sensor unit 20 captures the image, the rotating body control unit 31 enables the rotating body 35 to rotate. The sensor system 2 according to this embodiment obtains surrounding information by repeating such rotation of the rotating body 35 and imaging by the sensor unit 20.

For example, first, the rotating body control unit 31 transmits a rotating body control signal and an angle signal to the rotating body motor 32, and controls the rotating body motor 32. Thereby, the rotating body 35 and the sensor section 20 are rotated by a desired angle θ from the reference position (S10). At this time, the rotary body control section 31 transmits a signal indicating the rotation of the rotary body 35 together with a synchronization signal to the sensor control section 22 of the sensor section 20 to synchronize with the sensor control section 22 . Thereby, the sensor control unit 22 can prohibit imaging of the sensor 21 during the rotation period of the rotating body 35 (S20). That is, when the rotating body control unit 31 rotates the rotating body 35, the sensor control unit 22 stops acquiring images of the sensor 21.

Next, when the rotating body 35 and the sensor unit 20 rotate by the angle θ, the rotating body control unit 31 transmits a rotating body control signal to the rotating body motor 32 to control the rotating body motor 32. As a result, the rotation of the rotating body 35 and the sensor section 20 is stopped (S30). At this time, the rotating body control section 31 transmits a signal indicating the stop of the rotating body 35 together with a synchronization signal to the sensor control section 22 of the sensor section 20 to synchronize with the sensor control section 22 . Thereby, the sensor control unit 22 prohibits imaging of the sensor 21 during the rotation period of the rotating body 35, and allows imaging of the sensor 21 after the rotation of the rotating body 35 (S30). That is, when the rotating body control unit 31 stops rotating the rotating body 35, the sensor control unit 22 can start acquiring images of the sensor 21.

Next, the sensor control unit 22 transmits an imaging control signal to the sensor 21, and the sensor 21 starts imaging the surroundings (S40). As a result, imaging data is transmitted from the sensor 21 to the signal processing section 23. At this time, the sensor control section 22 transmits a signal indicating the start of imaging to the rotating body control section 31 together with a synchronization signal, and synchronizes with the rotating body control section 31 . The rotating body control unit 31 prohibits driving of the rotating body motor 32 and prohibits rotation of the rotating body 35 and the sensor unit 20 during the imaging period of the sensor 21 (S50).

The signal processing unit 23 acquires imaging data from the sensor 21 and generates an image of the surroundings (S60). This surrounding image is stored in a memory (not shown) or the like as surrounding information of the mobile system 1, or is transmitted to the outside. The mobile system 1 can autonomously travel using this surrounding information.

Further, the timing control unit 25 of the signal processing unit 23 transmits a signal indicating that an image is being generated to the rotating body control unit 31 of the rotating unit 30 along with a synchronization signal, and synchronizes with the rotating body control unit 31. As a result, the rotating body control unit 31 prohibits driving of the rotating body motor 32 during image generation, and prohibits rotation of the rotating body 35 and the sensor unit 20 (S70).

Next, the sensor control unit 22 transmits an imaging control signal to the sensor 21, and stops imaging by the sensor 21 (S80). At this time, the sensor control section 22 transmits a signal indicating the stop of imaging to the rotating body control section 31 together with a synchronization signal, and synchronizes with the rotating body control section 31 . Furthermore, when the acquisition of image data from the sensor 21 is completed in the signal processing unit 23, the timing control unit 25 transmits a signal indicating the end of image generation together with a synchronization signal to the rotating body control unit 31 of the rotating unit 30, and It is synchronized with the body control section 31. As a result, the rotary body control unit 31 allows the rotary body motor 32 to be driven after image generation is completed based on the signals from the sensor control unit 22 and the timing control unit 25, and controls the rotation of the rotary body 35 and the sensor unit 20. Permission is granted (S90). That is, when the sensor 21 stops capturing images, the signal processing unit 23 transmits a permission signal to the rotating body control unit 31 to enable the rotating body 35 to rotate.

Further, the sensor system 2 returns to step S10 and repeats steps S10 to S90. Thereby, the sensor system 2 can obtain an image of the surroundings at a position where the sensor unit 20 is further rotated by an angle θ. Steps S10 to S90 are repeated until the termination condition is reached (NO in S100). The termination condition may be any arbitrary condition, such as repeating N times (N is a natural number) or N×θ reaching 360 degrees. If the end condition is reached (YES in S100), the series of operations of the sensor system 2 ends.

As described above, according to the present embodiment, the sensor unit 20 stops (prohibits) imaging during the rotation period of the rotation body 35 in synchronization with the operation of the rotation body 35, and during the stop period of the rotation body 35, Imaging can be executed (permitted). On the other hand, in synchronization with the operation of the sensor unit 20, the rotating unit 30 stops (prohibits) the rotating operation of the rotating body 35 during the period when the sensor 21 is capturing an image and the signal processing unit 23 is processing the captured data. do. Further, the rotating unit 30 starts (permits) the rotating operation of the rotating body 35 during a period in which the sensor 21 finishes capturing an image and the signal processing unit 23 finishes processing the captured image data. Thereby, the sensor system 2 can image the entire surrounding situation with only one sensor section 20 while alternately performing image capturing of the sensor section 20 and rotation of the rotating section 30. That is, the sensor system 2 can acquire a wide-angle surrounding image using a small number of sensor units 20.

(Second embodiment)
FIG. 7 is a conceptual diagram showing an example of the operation of the sensor system 2 according to the second embodiment. The sensor system 2 according to the second embodiment detects the moving direction (X direction) of the moving body 3, and when an obstacle W exists in the moving direction of the moving body 3, the sensor system 2 moves the sensor unit 20 in the ±Y direction. Detect surrounding information by rotating by an angle θ. Thereby, the sensor system 2 can acquire surrounding information about the moving object 3 and the obstacle W.

Note that, similarly to the first embodiment, the rotation of the sensor unit 20 may be repeated, for example, N times, and may be further repeated until N×θ reaches 360 degrees.

FIG. 8 is a block diagram showing a configuration example of the sensor system 2 according to the second embodiment. In the second embodiment, a recognition unit 26 that recognizes an obstacle W from image information or distance information is provided in the signal processing unit 23. The recognition unit 26 identifies the obstacle W in the image using software. For example, the recognition unit 26 recognizes as an obstacle W when there is an object approaching the moving body 3 and the size of the object is larger than a predetermined value through image processing. The obstacle W may be, for example, a wall of a building. When the obstacle W is recognized, the signal processing unit 23 transmits the recognition information of the obstacle W to the rotating body control unit 31 of the rotating unit 30. The rotating body control unit 31 determines whether to rotate the rotating body 35 and the sensor unit 20 based on the recognition information. The other configurations of the second embodiment may be the same as the corresponding configurations of the first embodiment.

Next, the operation of the second embodiment will be explained.

FIG. 9 is a flow diagram showing an example of the operation of the second embodiment. First, the moving body 3 starts moving (S11). The moving direction of the moving body 3 is assumed to be the X direction. Further, the sensor control unit 22 transmits an imaging control signal to the sensor 21, and the sensor 21 starts imaging the surroundings (S21). Thereby, the sensor system 2 acquires imaging data while moving together with the mobile object 3. Imaging data is transmitted from the sensor 21 to the signal processing section 23. At this time, the sensor control section 22 transmits a signal indicating the start of imaging to the rotating body control section 31 together with a synchronization signal, and synchronizes with the rotating body control section 31 . The rotating body control unit 31 may prohibit or allow rotation of the rotating body 35 and the sensor unit 20 during the imaging period of the sensor 21.

The sensor 21 images the moving direction of the moving body 3, and when there is an object in the moving direction, images the object.

The signal processing unit 23 acquires imaging data from the sensor 21 and generates an image of the surroundings (S31). This surrounding image is stored in a memory (not shown) or the like as surrounding information of the mobile system 1, or is transmitted to the outside. The signal processing unit 23 acquires imaging data from the sensor 21, and when there is an object in the traveling direction of the moving body 3, acquires an image of the object obtained from the imaging data.

Further, the timing control unit 25 of the signal processing unit 23 transmits a signal indicating that an image is being generated to the rotating body control unit 31 of the rotating unit 30 along with a synchronization signal, and synchronizes with the rotating body control unit 31. Thereby, the rotating body control unit 31 may prohibit or allow rotation of the rotating body 35 and the sensor unit 20 during image generation.

Next, the recognition unit 26 determines whether the obstacle W is recognized (S41). As described above, the recognition unit 26 recognizes the obstacle W as the obstacle W, for example, when the distance between the moving body 3 and the obstacle W is short and the obstacle W has a size equal to or larger than a predetermined value.

If the recognition unit 26 does not recognize the obstacle W (NO in S41), steps S11 to S31 are repeated, the moving body 3 continues to move, and the sensor 21 continues to capture images.

On the other hand, when the recognition unit 26 recognizes the obstacle W (YES in S41), the recognition unit 26 determines whether the distance from the sensor 21 to the obstacle W is less than a predetermined value (S51). The distance from the sensor 21 to the obstacle W may be determined using image processing software or the like. Alternatively, if the sensor 21 is ToF, the results of distance measurement by ToF may be used.

If the distance from the sensor 21 to the obstacle W is equal to or greater than the predetermined value (NO in S51), steps S11 to S31 are repeated, the moving body 3 continues to move, and the sensor 21 continues to capture images.

On the other hand, if the distance from the sensor 21 to the obstacle W is less than the predetermined value (YES in S51), the rotating body control unit 31 transmits a rotating body control signal and an angle signal to the rotating body motor 32, and Then, the sensor unit 20 is rotated by a desired angle θ from the current position (S61). At this time, the rotary body control section 31 transmits a signal indicating the rotation of the rotary body 35 together with a synchronization signal to the sensor control section 22 of the sensor section 20 to synchronize with the rotary body sensor control section 22 . Thereby, the sensor control unit 22 may prohibit or permit imaging of the sensor 21 during the rotation period of the rotating body 35. Although not shown, if the distance from the sensor 21 to the obstacle W is less than a predetermined value, the moving body 3 may temporarily stop.

Next, when the rotating body 35 and the sensor unit 20 rotate by the angle θ, the rotating body control unit 31 transmits a rotating body control signal to the rotating body motor 32, and stops the rotation of the rotating body 35 and the sensor unit 20. let At this time, the rotating body control section 31 transmits a signal indicating the stop of the rotating body 35 together with a synchronization signal to the sensor control section 22 of the sensor section 20 to synchronize with the sensor control section 22 . Thereby, the sensor control unit 22 can permit the sensor 21 to take an image after the rotating body 35 has rotated.

Next, the sensor control unit 22 transmits an imaging control signal to the sensor 21, and the sensor 21 executes imaging of the surroundings (S71). As a result, imaging data is transmitted from the sensor 21 to the signal processing section 23. At this time, the sensor control section 22 transmits a signal indicating the start of imaging to the rotating body control section 31 together with a synchronization signal, and synchronizes with the rotating body control section 31 . At this time, the rotating body control section 31 may prohibit rotation of the rotating body 35 and the sensor section 20 during the imaging period of the sensor 21.

The signal processing unit 23 acquires imaging data from the sensor 21 and generates an image of the surroundings (S81). This surrounding image is stored in a memory (not shown) or the like as surrounding information of the mobile system 1, or is transmitted to the outside. As a result, an image in a direction different from the traveling direction (the direction in which the obstacle W exists) is acquired, and the recognition unit 26 can determine whether or not there is an obstacle in that direction.

Further, the timing control unit 25 of the signal processing unit 23 transmits a signal indicating that an image is being generated to the rotating body control unit 31 of the rotating unit 30 along with a synchronization signal, and synchronizes with the rotating body control unit 31. Thereby, the rotating body control unit 31 may prohibit rotation of the rotating body 35 and the sensor unit 20 during image generation.

According to the second embodiment, the signal processing unit 23 acquires the image or distance of the object in front of the sensor 21 from the surrounding image. Then, when an obstacle W is recognized in the traveling direction of the moving body 3 and the distance from the moving body 3 to the obstacle W becomes less than a predetermined value, the rotating body control unit 31 controls the rotating body 35 and the sensor unit 20 to a desired position. Rotate by angle θ. That is, the rotating body control unit 31 rotates the rotating body 35 based on the image or distance of the obstacle W. Thereby, the sensor 21 can acquire surrounding information in a direction rotated by an angle θ from the traveling direction of the moving body 3 in which the obstacle W exists. This surrounding information can be used to execute the next operation of the moving body 3, such as stopping the moving body 3 or changing the direction of movement of the moving body 3.

(Third embodiment)
10 and 11 are conceptual diagrams showing an example of the operation of the sensor system 2 according to the third embodiment. The sensor system 2 according to the third embodiment detects the traveling direction (X direction) of the moving body 3, and when the moving body 3 changes the traveling direction, the signal processing unit 23 detects the movement of the entire surrounding image. The turning direction of the moving body 3 is detected. The rotating body control unit 31 rotates the rotating body 35 in a direction opposite to the turning direction of the moving body 3. Thereby, the sensor system 2 can suppress image shift (blur) caused by the turning of the moving body 3.

For example, as shown in FIG. 10, the moving body 3 is initially moving in the X direction, and the sensor 21 is capturing an image in the moving direction (X direction) of the moving body 3. As shown in FIG. 11, when the moving body 3 turns by an angle θ1, the signal processing unit 23 detects that the moving body 3 turns due to the movement of the entire image and detects the turning angle θ1. Then, the rotating body control unit 31 rotates the rotating body 35 and the sensor unit 20 by an angle θ2 in a direction opposite to the turning direction of the moving body 3. As a result, even if the moving body 3 turns, the imaging direction of the sensor 21 does not deviate much from the initial traveling direction (X direction) of the moving body 3. When the rotation angle θ2 of the rotating body 35 is substantially the same as the turning angle θ1 of the moving body 3, the imaging direction of the sensor 21 hardly deviates from the initial traveling direction (X direction) of the moving body 3. Note that the rotation angle θ2 of the rotating body 35 does not necessarily have to be the same as the turning angle θ1 of the moving body 3. Even in this case, the effect of suppressing image blur due to the rotation of the moving body 3 can be obtained.

FIG. 12 is a block diagram showing a configuration example of the sensor system 2 according to the third embodiment. According to the third embodiment, the signal processing section 23 includes a correction calculation section 27 that calculates the turning direction and turning angle of the moving body 3 from the deviation of the entire image. The correction calculation unit 27 detects the turning of the moving body 3 based on the deviation of the entire image, and calculates the turning direction and turning angle of the moving body 3 from the deviation direction and deviation width of the entire image. When the signal processing unit 23 detects the turning of the moving body 3, it transmits the calculation results of the turning direction (for example, −Y direction) and the turning angle θ1 of the moving body 3 to the rotating body control unit 31 of the rotating unit 30. The rotating body control unit 31 determines whether or not to rotate the rotating body 35 based on the calculation results of the turning direction and turning angle of the moving body 3 from the signal processing unit 23, and when rotating the rotating body 35, the rotating body 35 is rotated. 35 and the sensor section 20 are rotated in a direction opposite to the turning direction (eg, +Y direction). The rotation angle θ2 of the rotating body 35 and the sensor unit 20 is preferably the same as the turning angle θ1, but may be different. The other configurations of the second embodiment may be the same as the corresponding configurations of the first embodiment.

Next, the operation of the third embodiment will be explained.

FIG. 13 is a flow diagram showing an example of the operation of the third embodiment. First, similarly to the second embodiment, steps S11 to S31 are executed. As a result, the moving body 3 moves, and the sensor system 2 captures an image in the moving direction (X direction) of the moving body 3. At this time, the sensor control section 22 transmits a signal indicating the start of imaging to the rotating body control section 31 together with a synchronization signal, and synchronizes with the rotating body control section 31 . The rotating body control unit 31 may prohibit or allow rotation of the rotating body 35 and the sensor unit 20 during the imaging period of the sensor 21.

Next, the correction calculation unit 27 detects the turning of the moving body 3 when the entire image shifts in one direction (S42). If the image is partially moving or if the entire image is not moving in one direction, the correction calculation unit 27 does not determine that the moving body 3 is turning (NO in S42) and skips steps S11 to S31. May be repeated.

If the entire image shifts (moves) in one direction, the correction calculation unit 27 detects that the moving body 3 has turned (YES in S42). For example, if the entire image shifts in either of the ±Y directions, the correction calculation unit 27 determines that the moving body 3 has turned in either the left or right direction.

Next, the correction calculation unit 27 calculates the turning direction and turning angle of the moving body 3 from the deviation direction and deviation width of the entire image (S52). For example, if the image is entirely shifted in the +Y direction, the correction calculation unit 27 determines that the moving body 3 has turned in the -Y direction. If the image is entirely shifted in the -Y direction, the correction calculation unit 27 determines that the moving body 3 has turned in the +Y direction. Further, the image shift width corresponds to the turning angle θ1 of the moving body 3. Therefore, for example, the correction calculation unit 27 uses the distance between the imaged arbitrary object and the sensor 21 and the image shift width (the movement width of the object in the image) to determine the turning angle of the moving body 3. Calculate θ1.

Next, the signal processing unit 23 transmits the signal indicating the turning of the moving body 3, the turning direction of the moving body 3 (for example, −Y direction), and the calculation result of the turning angle θ1 to the rotating body of the rotating unit 30, together with a synchronization signal. The information is transmitted to the control unit 31 (S62).

Next, the rotating body control unit 31 determines whether or not to rotate the rotating body 35 and the sensor unit 20 based on the calculation result of the turning direction or turning angle of the moving body 3 from the signal processing unit 23 (S72 ). For example, when the magnitude of the rotation angle is equal to or greater than the threshold value, the rotating body control unit 31 may determine that the rotating body 35 and the sensor unit 20 are to be rotated. Conversely, when the magnitude of the rotation angle is less than the threshold value, the rotating body control unit 31 may determine not to rotate the rotating body 35 and the sensor unit 20.

If the rotating body 35 and the sensor unit 20 are not to be rotated (NO in S72), the process returns to step S11.

On the other hand, when rotating the rotating body 35 and the sensor unit 20 (YES in S72), the rotating body control unit 31, based on the calculation results of the turning direction and turning angle of the moving body 3 from the signal processing unit 23, The rotating body 35 and the sensor section 20 are rotated in a direction opposite to the turning direction (for example, in the +Y direction) (S82). The rotation angle θ2 of the rotating body 35 and the sensor unit 20 is preferably the same as the turning angle θ1, but may be different. Note that the sensor 21 continues to capture images in steps S42 to S82.

Steps S11 to S82 are repeated until the movement of the moving body 3 is completed or until the imaging by the sensor 21 is completed (NO in S92). When the movement of the mobile body 3 ends or the imaging by the sensor 21 ends (YES in S92), the operation of the mobile system 1 ends.

According to the third embodiment, the correction calculation unit 27 detects the turning of the moving object 3 based on the deviation of the entire image, and calculates the turning direction and turning angle of the moving object 3 from the deviation direction and deviation width of the entire image, respectively. . The rotating body control unit 31 rotates the rotating body 35 and the sensor unit 20 in a direction opposite to the rotating direction based on the calculation results of the rotating direction and the rotating angle of the moving body 3. Thereby, even if the moving body 3 turns, the imaging direction of the sensor 21 does not deviate much from the initial traveling direction of the moving body 3. If the rotation angle θ2 of the rotating body 35 is made equal to the turning angle θ1 of the moving body 3, the imaging direction of the sensor 21 will hardly deviate from the original traveling direction (X direction) of the moving body 3. As a result, blurring of the image of the sensor 21 is further suppressed.

(Fourth embodiment)
14 and 15 are conceptual diagrams showing an example of the operation of the mobile system 1 according to the fourth embodiment. The sensor system 2 according to the fourth embodiment directly acquires motion information indicating the traveling direction or turning direction of the movable body 3 from the movable body 3, and rotates the rotating body 35 and the sensor unit 20 based on the motion information of the movable body 3. let Furthermore, since the sensor system 2 directly acquires the motion information of the moving body 3 from the moving body 3, the rotating body 35 and the sensor unit 20 are rotated to obtain an image in the turning direction of the moving body 3 before the movement of the moving body 3 starts. can be obtained in advance. Thereby, even when the moving body 3 turns, the mobile system 1 can turn after confirming the surrounding information in the turning direction.

For example, as shown in FIG. 14, the moving body 3 is initially moving in the X direction, and the sensor 21 is capturing an image in the moving direction (X direction) of the moving body 3. When the moving body 3 attempts to turn by an angle θ1, the signal processing unit 23 acquires from the moving body 3 information that the moving body 3 is turning, the turning direction, and the turning angle θ1. Then, the rotating body control section 31 rotates the rotating body 35 and the sensor section 20 by an angle θ2 in the turning direction of the moving body 3. Thereby, before the moving body 3 turns, an image of the turning direction of the moving body 3 can be acquired and the turning direction of the moving body 3 can be confirmed in advance. Note that the rotation angle θ2 of the rotating body 35 may or may not necessarily be the same as the turning angle θ1 of the moving body 3.

FIG. 16 is a block diagram showing a configuration example of the mobile system 1 according to the fourth embodiment. According to the fourth embodiment, the movable body 3 includes a movable body control section 41 and a movable body motor 42. The mobile body control unit 41 transmits a mobile body control signal to the mobile body motor 42. The moving object control signal is a signal indicating the operation of the moving object 3, and includes control signals such as a traveling speed, a turning direction, and a turning angle. The moving body motor 42 drives the wheels of the moving body 3 based on the moving system signal. This allows the moving body 3 to move or turn.

Further, the moving body control unit 41 transmits a moving body control signal to the rotating body control unit 31 while synchronizing with the rotating body control unit 31 of the rotating unit 30 using a synchronization signal. The rotating body control unit 31 can know operation information of the moving body 3 such as the turning direction and turning angle of the moving body 3 from the moving body control signal. It is preferable that the rotating body control unit 31 acquires the moving body control signal before the moving body motor 42. Thereby, before the moving body 3 starts turning, the sensor 21 can image the situation in the turning direction. The signal processing unit 23 may generate an image from the imaging data from the sensor 21, and use the image to determine whether or not the moving body 3 can turn. For example, if it is determined that there is an obstacle W or the like in the turning direction of the moving object 3, the sensor 21 can image the situation in the turning direction before the moving object 3 starts turning. The signal processing unit 23 may transmit a signal indicating that the mobile body 3 is prohibited from turning to the mobile body control unit 41 of the mobile body 3 together with a synchronization signal. In this case, the movable body control section 41 can prevent the movable body 3 from turning without transmitting the movable body control signal to the movable body motor 42.

Next, the operation of the fourth embodiment will be explained.

FIG. 17 is a flow diagram showing an example of the operation of the fourth embodiment. First, similarly to the second embodiment, steps S11 to S31 are executed. As a result, the moving body 3 moves, and the sensor system 2 captures an image in the moving direction (X direction) of the moving body 3. At this time, the sensor control section 22 transmits a signal indicating the start of imaging to the rotating body control section 31 together with a synchronization signal, and synchronizes with the rotating body control section 31 . The rotating body control unit 31 may prohibit or allow rotation of the rotating body 35 and the sensor unit 20 during the imaging period of the sensor 21.

The moving body control unit 41 is synchronized with the rotating body control unit 31 and transmits a moving body control signal to the rotating body control unit 31 (S43). When the moving body 3 is about to turn, the moving body control signal includes information such as the turning direction and turning angle of the moving body 3. Therefore, the rotating body control unit 31 can know the turning direction and turning angle of the moving body 3 in advance.

If the moving body 3 does not turn (NO in S53), steps S11 to S31 and S43 are repeated. At this time, the rotating body control section 31 does not need to rotate the rotating body 35.

On the other hand, when the moving body 3 is about to turn (YES in S53), the rotating body control unit 31 rotates the rotating body 35 and the sensor unit 20 in the turning direction of the moving body 3 before the moving body 3 starts turning. (S63). The rotation angle θ2 of the rotating body 35 is preferably the same as the turning angle θ1 of the moving body 3, but does not necessarily have to be the same.

Next, the sensor system 2 images the turning direction of the moving body 3 (S73). At this time, the rotating body 35 and the sensor section 20 are rotated by an angle θ2 from the traveling direction (for example, the X direction) of the moving body 3 to the turning direction (for example, the +Y direction). Thereby, the sensor 21 can image the surroundings while rotating in the turning direction of the moving body 3.

Next, the signal processing unit 23 determines whether the moving body 3 can turn based on the image generated from the image data from the sensor 21 (S83). The signal processing unit 23 determines the presence or absence of an obstacle W from the image in the turning direction of the moving body 3, and determines whether the moving body 3 can turn (S83). Although not shown in FIG. 16, the recognition unit 26 may determine from the image whether or not the moving body 3 can turn.

If the moving body 3 can turn (YES in S83), the signal processing unit 23 transmits a signal indicating that it can turn to the moving body control unit 41 and the rotating body control unit 31 together with a synchronization signal, and 3 is permitted to turn (S93). The mobile body control unit 41 transmits a mobile body control signal for controlling the turning operation of the mobile body 3 to the mobile body motor 42 . As a result, the moving body 3 starts turning.

Further, the rotating body control unit 31 can know the turning operation of the moving body 3 in advance based on a signal from the signal processing unit 23 instead of recognizing the turning of the moving body 3 based on the image from the sensor 21. Therefore, the rotating body control section 31 can rotate the rotating body 35 and the sensor section 20 before the moving body 3 starts turning.

Thereafter, the rotating body control unit 31 may rotate the rotating body 35 and the sensor unit 20 back to their original positions according to the turning operation of the moving body 3. For example, the rotating body 35 and the sensor unit 20 may be rotated by an angle θ2 in the direction opposite to the turning direction of the moving body 3, and the sensor 21 may be directed again in the direction in which the moving body 3 moves after the turning operation. Thereby, the sensor 21 can continuously image the moving direction of the moving body 3.

On the other hand, if the moving body 3 cannot turn (NO in S83), the signal processing unit 23 transmits a signal indicating that turning is not possible together with a synchronization signal to the moving body control unit 41 and the rotating

body control unit

31, and 3 turning is prohibited (S103). The moving object control unit 41 prohibits the turning operation of the moving object 3 and does not transmit a moving object control signal to the moving object motor 42. As a result, the moving body 3 does not turn.

Further, the rotating body control unit 31 can know from the signal from the signal processing unit 23 that the moving body 3 will not turn. Therefore, the rotating body control unit 31 can return the orientation of the rotating body 35 and the sensor unit 20 to the traveling direction of the moving body 3 (X direction). For example, the rotating body 35 and the sensor unit 20 may be rotated by an angle θ2 in the direction opposite to the turning direction of the moving body 3, and returned to the traveling direction (X direction) of the moving body 3.

Steps S11 to S103 are repeated until the movement of the moving body 3 is completed or until the imaging by the sensor 21 is completed (NO in S113). When the movement of the mobile body 3 ends or the imaging by the sensor 21 ends (YES in S113), the operation of the mobile system 1 ends.

According to the fourth embodiment, the rotating body control unit 31 can know the operation information of the moving body 3 such as the turning direction and the turning angle of the moving body 3 from the moving body control signal before the moving body 3 turns. Thereby, before the moving body 3 starts turning, the sensor 21 can take an image of the situation in the turning direction in advance. The signal processing unit 23 generates an image using the imaging data from the sensor 21, and can determine whether or not the moving body 3 can turn based on the image. The moving body 3 can start turning based on the determination by the signal processing unit 23 as to whether or not it can turn, or can proceed straight without turning.

(Fifth embodiment)
FIG. 18 is a flow diagram showing an example of the operation of the fifth embodiment. The configuration of the mobile system 1 according to the fifth embodiment may be the same as that of the fourth embodiment. In the fifth embodiment, the sensor system 2 directly acquires motion information indicating the traveling direction or turning direction of the movable body 3 from the movable body 3, and controls the rotating body 35 and the sensor unit 20 based on the motion information of the movable body 3. It is similar to the fourth embodiment in that it is rotated. However, in the fifth embodiment, the rotating body 35 and the sensor unit 20 are rotated in the opposite direction to the turning direction of the moving body 3 in order to suppress image blur due to the turning of the moving body 3.

First, similarly to the fourth embodiment, steps S11 to S53 are executed. As a result, the moving body 3 moves, and the sensor system 2 captures an image in the moving direction (X direction) of the moving body 3.

On the other hand, when the moving body 3 is about to turn (YES in S53), almost simultaneously with the start of the turning of the moving body 3, the rotating body control unit 31 controls the rotating body 35 and the sensor in a direction opposite to the turning direction of the moving body 3. The section 20 is rotated (S64). As a result, even if the moving body 3 turns, the imaging direction of the sensor 21 does not deviate much from the initial traveling direction (X direction) of the moving body 3. When the rotation angle θ2 of the rotating body 35 is substantially the same as the turning angle θ1 of the moving body 3, the imaging direction of the sensor 21 hardly deviates from the initial traveling direction (X direction) of the moving body 3. Note that the rotation angle θ2 of the rotating body 35 does not necessarily have to be the same as the turning angle θ1 of the moving body 3. Even in this case, the effect of suppressing image blur due to the rotation of the moving body 3 can be obtained.

Thereafter, steps S11 to S64 are repeated until the movement of the moving body 3 is completed or until the imaging of the sensor 21 is completed (NO in S113). When the movement of the mobile body 3 ends or the imaging by the sensor 21 ends (YES in S113), the operation of the mobile system 1 ends.

As described above, according to the fifth embodiment, even when the moving body 3 turns, the imaging direction of the sensor 21 does not deviate much from the initial traveling direction of the moving body 3. Therefore, the fifth embodiment can obtain the same effects as the third embodiment.

Further, according to the fifth embodiment, the rotating body control unit 31 directly obtains the moving body control signal from the moving body 3, and determines the turning direction, turning angle, etc. of the moving body 3 from the moving body control signal. Operation information can be known before the moving body 3 turns. Thereby, the sensor system 2 can rotate the rotating body 35 almost simultaneously with the turning of the moving body 3 without any delay.

The mobile system 1 according to the embodiments described above can acquire a wide range of surrounding information using images or by rotating the sensor 21 in synchronization with the control signal of the mobile body 3. Thereby, the mobile system 1 can acquire a wide range of surrounding information at low cost.

<Example of application to mobile objects>
The technology according to the present disclosure (this technology) can be applied to various products. For example, the technology according to the present disclosure may be realized as a device mounted on any type of moving body such as a car, electric vehicle, hybrid electric vehicle, motorcycle, bicycle, personal mobility, airplane, drone, ship, robot, etc. It's okay.

FIG. 19 is a block diagram showing a schematic configuration example of a vehicle control system, which is an example of a mobile object control system to which the technology according to the present disclosure can be applied.

The vehicle control system 12000 includes a plurality of electronic control units connected via a communication network 12001. In the example shown in FIG. 19, the vehicle control system 12000 includes a drive system control unit 12010, a body system control unit 12020, an outside vehicle information detection unit 12030, an inside vehicle information detection unit 12040, and an integrated control unit 12050. Further, as the functional configuration of the integrated control unit 12050, a microcomputer 12051, an audio/image output section 12052, and an in-vehicle network I/F (Interface) 12053 are illustrated.

The drive system control unit 12010 controls the operation of devices related to the drive system of the vehicle according to various programs. For example, the drive system control unit 12010 includes a drive force generation device such as an internal combustion engine or a drive motor that generates drive force for the vehicle, a drive force transmission mechanism that transmits the drive force to wheels, and a drive force transmission mechanism that controls the steering angle of the vehicle. It functions as a control device for a steering mechanism to adjust and a braking device to generate braking force for the vehicle.

The body system control unit 12020 controls the operations of various devices installed in the vehicle body according to various programs. For example, the body system control unit 12020 functions as a keyless entry system, a smart key system, a power window device, or a control device for various lamps such as a headlamp, a back lamp, a brake lamp, a turn signal, or a fog lamp. In this case, radio waves transmitted from a portable device that replaces a key or signals from various switches may be input to the body control unit 12020. The body system control unit 12020 receives input of these radio waves or signals, and controls the door lock device, power window device, lamp, etc. of the vehicle.

The external information detection unit 12030 detects information external to the vehicle in which the vehicle control system 12000 is mounted. For example, an imaging section 12031 is connected to the outside-vehicle information detection unit 12030. The vehicle exterior information detection unit 12030 causes the imaging unit 12031 to capture an image of the exterior of the vehicle, and receives the captured image. The external information detection unit 12030 may perform object detection processing such as a person, car, obstacle, sign, or text on the road surface or distance detection processing based on the received image.

The imaging unit 12031 is an optical sensor that receives light and outputs an electrical signal according to the amount of received light. The imaging unit 12031 can output the electrical signal as an image or as distance measurement information. Further, the light received by the imaging unit 12031 may be visible light or non-visible light such as infrared rays.

The in-vehicle information detection unit 12040 detects in-vehicle information. For example, a driver condition detection section 12041 that detects the condition of the driver is connected to the in-vehicle information detection unit 12040. The driver condition detection unit 12041 includes, for example, a camera that images the driver, and the in-vehicle information detection unit 12040 detects the degree of fatigue or concentration of the driver based on the detection information input from the driver condition detection unit 12041. It may be calculated, or it may be determined whether the driver is falling asleep.

The microcomputer 12051 calculates control target values for the driving force generation device, steering mechanism, or braking device based on the information inside and outside the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040, Control commands can be output to 12010. For example, the microcomputer 12051 realizes ADAS (Advanced Driver Assistance System) functions, including vehicle collision avoidance or impact mitigation, following distance based on vehicle distance, vehicle speed maintenance, vehicle collision warning, vehicle lane departure warning, etc. It is possible to perform cooperative control for the purpose of

In addition, the microcomputer 12051 controls the driving force generating device, steering mechanism, braking device, etc. based on information about the surroundings of the vehicle acquired by the vehicle exterior information detection unit 12030 or the vehicle interior information detection unit 12040. It is possible to perform cooperative control for the purpose of autonomous driving, etc., which does not rely on operation.

Furthermore, the microcomputer 12051 can output a control command to the body system control unit 12030 based on the information outside the vehicle acquired by the outside information detection unit 12030. For example, the microcomputer 12051 controls the headlamps according to the position of the preceding vehicle or oncoming vehicle detected by the vehicle exterior information detection unit 12030, and performs cooperative control for the purpose of preventing glare, such as switching from high beam to low beam. It can be carried out.

The audio and image output unit 12052 transmits an output signal of at least one of audio and images to an output device that can visually or audibly notify information to the occupants of the vehicle or to the outside of the vehicle. In the example of FIG. 19, an audio speaker 12061, a display section 12062, and an instrument panel 12063 are illustrated as output devices. The display unit 12062 may include, for example, at least one of an on-board display and a head-up display.

FIG. 20 is a diagram showing an example of the installation position of the imaging section 12031.

In FIG. 20, the imaging unit 12031 includes

imaging units

12101, 12102, 12103, 12104, and 12105.

The

imaging units

12101, 12102, 12103, 12104, and 12105 are provided at, for example, the front nose of the vehicle 12100, the side mirrors, the rear bumper, the back door, and the upper part of the windshield inside the vehicle. An imaging unit 12101 provided in the front nose and an imaging unit 12105 provided above the windshield inside the vehicle mainly acquire images in front of the vehicle 12100.

Imaging units

12102 and 12103 provided in the side mirrors mainly capture images of the sides of the vehicle 12100. An imaging unit 12104 provided in the rear bumper or back door mainly captures images of the rear of the vehicle 12100. The imaging unit 12105 provided above the windshield inside the vehicle is mainly used to detect preceding vehicles, pedestrians, obstacles, traffic lights, traffic signs, lanes, and the like.

Note that FIG. 20 shows an example of the imaging range of the imaging units 12101 to 12104. An imaging range 12111 indicates the imaging range of the imaging unit 12101 provided on the front nose, imaging ranges 12112 and 12113 indicate imaging ranges of the

imaging units

12102 and 12103 provided on the side mirrors, respectively, and an imaging range 12114 shows the imaging range of the imaging unit 12101 provided on the front nose. The imaging range of the imaging unit 12104 provided in the rear bumper or back door is shown. For example, by overlapping the image data captured by the imaging units 12101 to 12104, an overhead image of the vehicle 12100 viewed from above can be obtained.

At least one of the imaging units 12101 to 12104 may have a function of acquiring distance information. For example, at least one of the imaging units 12101 to 12104 may be a stereo camera including a plurality of image sensors, or may be an image sensor having pixels for phase difference detection.

For example, the microcomputer 12051 determines the distance to each three-dimensional object within the imaging ranges 12111 to 12114 and the temporal change in this distance (relative speed with respect to the vehicle 12100) based on the distance information obtained from the imaging units 12101 to 12104. By determining the following, it is possible to extract, in particular, the closest three-dimensional object on the path of vehicle 12100, which is traveling at a predetermined speed (for example, 0 km/h or more) in approximately the same direction as vehicle 12100, as the preceding vehicle. can. Furthermore, the microcomputer 12051 can set an inter-vehicle distance to be secured in advance in front of the preceding vehicle, and perform automatic brake control (including follow-up stop control), automatic acceleration control (including follow-up start control), and the like. In this way, it is possible to perform cooperative control for the purpose of autonomous driving, etc., in which the vehicle travels autonomously without depending on the driver's operation.

For example, the microcomputer 12051 transfers three-dimensional object data to other three-dimensional objects such as two-wheeled vehicles, regular vehicles, large vehicles, pedestrians, and utility poles based on the distance information obtained from the imaging units 12101 to 12104. It can be classified and extracted and used for automatic obstacle avoidance. For example, the microcomputer 12051 identifies obstacles around the vehicle 12100 into obstacles that are visible to the driver of the vehicle 12100 and obstacles that are difficult to see. Then, the microcomputer 12051 determines a collision risk indicating the degree of risk of collision with each obstacle, and when the collision risk exceeds a set value and there is a possibility of a collision, the microcomputer 12051 transmits information via the audio speaker 12061 and the display unit 12062. By outputting a warning to the driver via the vehicle control unit 12010 and performing forced deceleration and avoidance steering via the drive system control unit 12010, driving support for collision avoidance can be provided.

At least one of the imaging units 12101 to 12104 may be an infrared camera that detects infrared rays. For example, the microcomputer 12051 can recognize a pedestrian by determining whether the pedestrian is present in the images captured by the imaging units 12101 to 12104. Such pedestrian recognition involves, for example, a procedure for extracting feature points in images captured by the imaging units 12101 to 12104 as infrared cameras, and a pattern matching process is performed on a series of feature points indicating the outline of an object to determine whether it is a pedestrian or not. This is done through a procedure that determines the When the microcomputer 12051 determines that a pedestrian is present in the images captured by the imaging units 12101 to 12104 and recognizes the pedestrian, the audio image output unit 12052 creates a rectangular outline for emphasis on the recognized pedestrian. The display unit 12062 is controlled to display the . Furthermore, the audio image output unit 12052 may control the display unit 12062 to display an icon or the like indicating a pedestrian at a desired position.

An example of a vehicle control system to which the technology according to the present disclosure can be applied has been described above. The technology according to the present disclosure can be applied to, for example, the imaging unit 12031 among the configurations described above.

Note that the present technology can have the following configuration.
(1)
a rotating body rotatable around a first axis;
a rotating body control section that controls the rotating body;
a sensor disposed on the rotating body to acquire surrounding information;
a sensor control unit that controls the sensor based on control timing of the rotating body control unit;
A sensor system comprising: a signal processing unit that processes the surrounding information acquired by the sensor.
(2)
When the rotating body control unit rotates the rotating body, the sensor control unit stops acquiring the surrounding information of the sensor,
The sensor system according to (1), wherein when the rotating body control unit stops rotation of the rotating body, the sensor control unit starts acquiring the surrounding information of the sensor.
(3)
According to (1) or (2), when the sensor stops acquiring the surrounding information, the signal processing unit transmits a permission signal that enables rotation of the rotating body to the rotating body control unit. sensor system.
(4)
The signal processing unit acquires an image or a distance of an object in front of the sensor using the surrounding information,
The sensor system according to (1) or (2), wherein the rotating body control unit rotates the rotating body based on an image or distance of the object.
(5)
The signal processing unit detects a turning direction of the mobile body based on movement of the entire surrounding information,
The sensor system according to (1) or (2), wherein the rotating body control unit rotates the rotating body in a direction opposite to the turning direction.
(6)
The signal processing unit calculates a turning angle of the rotating body based on the movement of the entire surrounding information,
The sensor system according to (5), wherein the rotating body control unit rotates the rotating body by the rotating angle in a direction opposite to the rotating direction.
(7)
The sensor system according to any one of (1) to (5), wherein the sensor includes CIS or ToF.
(8)
A rotating body rotatable about a first axis, a rotating body control section that controls the rotating body, a sensor arranged on the rotating body that acquires surrounding information, and a control timing of the rotating body control section. A mobile body equipped with a sensor system including a sensor control unit that controls the sensor, and a signal processing unit that processes the surrounding information acquired by the sensor, and
A mobile body system comprising a mobile body control unit that controls the mobile body.
(9)
When the moving body control unit turns the moving body, the rotating body control unit rotates the rotating body in the turning direction of the moving body before turning the moving body,
The sensor acquires the surrounding information while the rotating body is rotated in the turning direction,
The signal processing unit outputs a permission signal that enables the mobile body to turn based on the surrounding information to the mobile body control unit,
The mobile system according to (8), wherein the mobile body control unit turns the mobile body.
(10)
According to (9), when the moving body control unit rotates the moving body by a first angle, the rotating body control unit rotates the rotating body by the first angle in the turning direction of the moving body. mobile system.
(11)
When the moving body control unit turns the moving body, the rotating body control unit rotates the rotating body in a direction opposite to the turning direction of the moving body almost simultaneously with the turning of the moving body. 8) The mobile system according to item 8).
(12)
When the moving body control section rotates the moving body by a first angle, the rotating body control section rotates the rotating body by the first angle in a direction opposite to the turning direction of the moving body. 11) The mobile system according to item 11).
(13)
The mobile system according to any one of (8) to (12), wherein the sensor includes a CIS, ToF, or stereo camera.

Note that the present disclosure is not limited to the embodiments described above, and various changes can be made without departing from the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not limited, and other effects may also be present.

Claims

a rotating body rotatable around a first axis;
a rotating body control section that controls the rotating body;
a sensor disposed on the rotating body to acquire surrounding information;
a sensor control unit that controls the sensor based on control timing of the rotating body control unit;
A sensor system comprising: a signal processing unit that processes the surrounding information acquired by the sensor.
When the rotating body control unit rotates the rotating body, the sensor control unit stops acquiring the surrounding information of the sensor,
The sensor system according to claim 1, wherein when the rotating body control unit stops rotation of the rotating body, the sensor control unit starts acquiring the surrounding information of the sensor.
The sensor system according to claim 1, wherein when the sensor stops acquiring the surrounding information, the signal processing unit transmits a permission signal that enables rotation of the rotating body to the rotating body control unit.
The signal processing unit acquires an image or a distance of an object in front of the sensor using the surrounding information,
The sensor system according to claim 1, wherein the rotating body control unit rotates the rotating body based on an image or a distance of the object.
The signal processing unit detects a turning direction of the mobile body based on movement of the entire surrounding information,
The sensor system according to claim 1, wherein the rotating body control unit rotates the rotating body in a direction opposite to the turning direction.
The signal processing unit calculates a turning angle of the rotating body based on the movement of the entire surrounding information,
The sensor system according to claim 5, wherein the rotating body control unit rotates the rotating body by the rotating angle in a direction opposite to the rotating direction.
The sensor system according to claim 1, wherein the sensor includes CIS (CMOS (Complementary Metal Oxide Semiconductor) Image Sensor) or ToF (Time of Fright).
A rotating body rotatable about a first axis, a rotating body control section that controls the rotating body, a sensor arranged on the rotating body that acquires surrounding information, and a control timing of the rotating body control section. A mobile body equipped with a sensor system including a sensor control unit that controls the sensor, and a signal processing unit that processes the surrounding information acquired by the sensor, and
A mobile body system comprising a mobile body control unit that controls the mobile body.
When the moving body control unit turns the moving body, the rotating body control unit rotates the rotating body in the turning direction of the moving body before turning the moving body,
The sensor acquires the surrounding information while the rotating body is rotated in the turning direction,
The signal processing unit outputs a permission signal that enables the mobile body to turn based on the surrounding information to the mobile body control unit,
The moving body system according to claim 8, wherein the moving body control unit turns the moving body.
According to claim 9, when the moving body control unit rotates the moving body by a first angle, the rotating body control unit rotates the rotating body by the first angle in a turning direction of the moving body. mobile system.
When the moving body control unit turns the moving body, the rotating body control unit rotates the rotating body in a direction opposite to the turning direction of the moving body almost simultaneously with the turning of the moving body. The mobile system according to item 8.
When the moving body control unit rotates the moving body by a first angle, the rotating body control unit rotates the rotating body by the first angle in a direction opposite to a turning direction of the moving body. 12. The mobile system according to item 11.
The mobile system according to claim 8, wherein the sensor includes a CIS, ToF, or stereo camera.