JP6685713B2 - Imaging system and its control method, communication device, and mobile imaging device - Google Patents

Description

  The present invention relates to a mobile imaging device having an imaging function and a moving function, and a communication device that communicates with the mobile imaging device.

  There is a video conference as a system for moving and controlling a video camera having a zoom function. Patent Document 1 discloses a technique of controlling the moving speed of a video camera according to the zoom position of the camera. When a video camera is moved and controlled in a video conference, if the moving speed is controlled to be constant regardless of the zoom magnification, the amount of movement of the subject image is small in wide-angle shooting and large in telephoto shooting. Therefore, in order to solve the problem that the user viewing the received image feels uncomfortable, Patent Document 1 controls the moving speed of the video camera according to the zoom control amount.

  Further, as a moving image pickup device, there is a device in which a camera is attached to a small drone such as a multicopter having three or more rotary wings, and a computer or a smartphone can be easily operated as a controller. A drone is a flying body that is operated by a crew member by remote control or autonomous control without boarding, and in recent years, means a drone in general. The controller of the small drone has operation keys for moving the drone and operation keys for zooming the camera.

JP-A-6-268899

  In a small drone equipped with a camera, it is necessary for the user to perform framing while operating the drone and zooming the camera. Therefore, the operation is complicated, and it is difficult for the user to perform the desired framing. For example, if the user wants to quickly adjust the framing during the moving operation of the drone, he must immediately switch from the drone operation to the camera zoom operation. Therefore, the timing of shooting may be missed during the switching of the operation.

Further, in the technique disclosed in Patent Document 1, the moving speed of the video camera is controlled in association with the zoom control amount. Therefore, it is impossible to deal with the operation of matching the framing by a simple operation.
An object of the present invention is to allow a user to perform desired framing with an easy operation in an imaging system including a mobile imaging device and a communication device.

An image pickup system according to an embodiment of the present invention is an image pickup system including a moving image pickup apparatus having an image pickup function and a moving function, wherein a receiving unit that receives a zoom instruction and the zoom instruction that is received by the receiving unit are provided. depending on the change rate of the zoom shown, a first control for movement control, determination means for determining whether to one of the control of the second control for zoom control, based on a determination result by the determining means And a control means for performing the first control and the second control.

  According to the imaging system of the present invention, the user can perform desired framing with an easy operation by adaptively controlling the movement of the moving imaging device and the imaging operation.

FIG. 3 is a relationship diagram between a communication device and a mobile imaging device according to the embodiment of the present invention. It is a block diagram of the communication apparatus concerning the embodiment of the present invention. It is a block diagram of the mobile imaging device concerning the embodiment of the present invention. It is explanatory drawing of the zoom function in the communication apparatus of 1st Embodiment. 3 is an operation flowchart of a zoom function in the first embodiment. It is explanatory drawing of the automatic imaging position alignment function in 2nd Embodiment. 9 is an operation flowchart of an automatic shooting position adjusting function in the second embodiment. It is explanatory drawing of the zoom system switching according to the communication condition in 3rd Embodiment. 9 is an operation flowchart of zoom method switching according to a communication state in the third embodiment.

  Hereinafter, each embodiment of the present invention will be described with reference to the drawings. In each embodiment, a mobile imaging system using an unmanned aircraft will be described as an imaging system including a mobile imaging device having an imaging function and a moving function and a communication device.

[First Embodiment]
FIG. 1 is a conceptual diagram illustrating the relationship between the communication device 200 and the mobile imaging device 100 according to the embodiment of the present invention. The mobile imaging device 100 is an unmanned aerial vehicle, and in this embodiment, a flying body including four motors M1 to M4 is illustrated. The flying body includes a drive control unit that realizes flight by independently rotating a rotor (rotary blade) connected to each motor in a well-balanced manner. The flying object includes an image capturing unit that captures an image of a subject through a camera lens and an image sensor, and performs an image capturing operation according to an operation instruction signal from the communication device 200. The mobile imaging device 100 is a device that realizes movement of the machine body and a shooting operation.

  The communication device 200 has a communication function with the mobile imaging device 100. The communication device 200 is a smartphone, a computer, a dedicated remote controller, or the like. The communication device 200 communicates with the mobile imaging device 100 using a communication method such as wireless LAN (Local Area Network), Bluetooth (registered trademark), or radio control. Specifically, the communication device 200 and the mobile imaging device 100 are connected by the following four wireless communication means.

-Movement control information communication means The transmission unit of the communication device 200 transmits the movement control information to the mobile imaging device 100. The receiving unit of the mobile imaging device 100 receives the movement control information from the communication device 200, and the movement control is performed according to the movement control information.
-Movement state communication means The transmission unit of the mobile imaging device 100 transmits the movement performance, current position information, and the like as the information related to the movement state, and the reception unit of the communication device 200 receives the information.
Imaging Control Information Communication Means The transmitting unit of the communication device 200 transmits the imaging control information to the mobile imaging device 100. The receiving unit of the mobile imaging device 100 receives the imaging control information from the communication device 200, and the imaging unit is controlled according to the imaging control information.
-Imaging State Communication Means The transmitting unit of the mobile imaging device 100 transmits the imaging information including the captured image data and the imaging state to the mobile imaging device 100, and the receiving unit of the communication device 200 receives the information.

A specific example of each wireless communication means will be described with reference to FIG. FIG. 2 is a block diagram showing a hardware configuration example of the communication device 200.
A main CPU (Central Processing Unit) 201 is a central part that controls the entire communication device 200. A DRAM (Dynamic Random Access Memory) 202 is a main memory in the communication device 200. The image stream, the frame image data, and the communication data with the mobile imaging device 100, which are large amounts of data, are temporarily stored in the DRAM 202.

  A Codec (encoding / decoding unit) 203 decodes the compressed image stream to generate a frame image. A GPU (Graphics Processing Unit) 204 draws widgets such as GUI buttons and icons presented to the user by displaying them on the screen on the DRAM 202 by drawing them. GUI is an abbreviation for "Graphical User Interface".

  An LCD (Liquid Crystal Display) 205 is a device that constitutes a display unit and combines and displays the frame image stored in the DRAM 202 and the widget. The LCD 205 is provided with a touch panel and has a function of detecting the coordinate position touched by the user. The memory 206 is a storage device that stores an application program that can be executed by the CPU 201. For example, a flash memory (semiconductor non-volatile memory) is used. A program RAM (Random Access Memory) 207 stores programs and the like that can be executed by the CPU 201.

  The movement control information transmitting unit 208 wirelessly transmits the movement control information to the mobile imaging device 100. The movement control information is information used to control the movement, posture, and the like of the mobile imaging device 100. The mobile imaging device 100 includes a wireless reception unit for mobile control information. The moving state receiving unit 209 wirelessly receives moving state information such as moving performance and current position information from the mobile imaging device 100. The mobile imaging device 100 includes a wireless transmission unit for moving state information.

The imaging control information transmission unit 210 wirelessly transmits the imaging control information to the mobile imaging device 100. The imaging control information is information used for controlling the imaging device mounted on the mobile imaging device 100 and includes zoom control information and the like. The mobile imaging device 100 includes a wireless receiving unit for imaging control information. The imaging state receiving unit 211 wirelessly receives the imaging information including the captured image stream from the mobile imaging device 100. The mobile imaging device 100 includes a wireless transmission unit for imaging information.
The main data bus 212 connects the respective units of the communication device 200 to each other so as to be able to transmit and receive.

Next, the software in this embodiment will be described. The memory 206 stores the following programs that can be executed by the CPU 201.
-Imaging operation input program: A program that uses the GPU 204 to draw a GUI screen that accepts an imaging operation instruction from the user, and that accepts a user operation such as a zoom operation or an imaging start instruction by the touch panel function of the LCD 205.
Zoom instruction change amount calculation program: A program that calculates the change amount of the zoom instruction based on the temporal history of the zoom operation instruction input by the imaging operation input program.
-Zoom position detection program: A program that detects the zoom position in the moving image pickup apparatus 100 via the image pickup state reception unit 211.
-Imaging frame image generation program: A program that uses the Codec 203 to generate a frame image from the image stream after imaging, which is received via the imaging state receiving unit 211.
-Imaging range detection program: A program that compares the frame images for a predetermined time obtained by the imaging frame image generation program and calculates the amount of change in the current imaging range.
Communication state detection program: a program for detecting the communication states of the movement control information transmission unit 208 and the imaging control information transmission unit 210.
-Movement performance calculation program: A program that calculates the movement performance of the mobile imaging device 100 from the instruction transmitted by the movement control information transmission unit 208 and the position information received by the movement state reception unit 209.
The main CPU (hereinafter, simply referred to as CPU) 201 realizes various processes described below by interpreting and executing the program read from the memory 206 and expanded in the program RAM 207.

  With reference to FIG. 3, a camera attachment type drone will be described as a configuration example of the mobile imaging device 100. Each component inside the mobile imaging device 100 is connected to the system bus 320 and can transmit and receive necessary data to and from each other. The CPU 301 that controls the entire mobile imaging device 100 controls each unit and controls the flight and imaging of the mobile imaging device 100. The memory 302 is a rewritable storage device, and is used as a work area of a program that controls the mobile imaging apparatus 100. The non-volatile memory 303 stores a program for controlling the mobile imaging device 100 and data used by the program. Data that needs to be retained even when the power is off, such as the setting information of the mobile imaging device 100, is stored in the non-volatile memory 303. When the power supply unit 306 is turned on, the CPU 301 reads the control program from the non-volatile memory 303 and starts controlling the mobile imaging device 100.

  The operation unit 304 includes buttons, dials, switches and the like used by the user for operations. For example, the user uses the operation unit 304 when setting ON / OFF of the power supply unit 306. The display unit 305 includes, for example, a light emitting diode (LED) and a driver unit that controls the light emitting diode (LED). The display unit 305 displays the state of the mobile imaging device 100 under the control of the CPU 301.

  The cradle 307 controls the image pickup of the image pickup unit 308. The cradle 307 is connected to the image pickup unit 308 by a USB (Universal Serial Bus) or the like, and controls the image pickup unit 308 based on a control signal from the CPU 301. In the present embodiment, the optical axis of the image pickup unit 308 is installed so as to be in the front direction of the moving image pickup apparatus 100. The imaging unit 308 receives light from a subject through the imaging optical system, photoelectrically converts the optical image signal into an image signal, and acquires image data. The CPU 301 can control the image capturing unit 308 via the cradle 307 to execute the image capturing operation, and acquire the data of the captured image.

  The wireless communication unit 309 is a module that wirelessly communicates with the communication device 200 via a wireless LAN or the like, and includes a transmission unit and a reception unit. For example, the CPU 301 acquires image data captured by the image capturing unit 308 and controls the wireless communication unit 309 to transmit the image data to the communication device 200. The motor 310 is a rotary drive unit connected to the rotor 311 of the moving image pickup apparatus 100, and rotates during movement and at rest according to a control signal from the CPU 301.

  The gyroscope 312 detects the posture of the moving image pickup apparatus 100, and detects the angular velocity of the rotational movement of the moving image pickup apparatus 100 and the inclination with respect to the vertical direction. The gyroscope 312 can detect rotations of three axes in each of pitch, yaw, and roll directions, and is used together with an acceleration sensor 315 described later. Three-axis detection signals from the gyroscope 312 are used for flight attitude control of the mobile imaging device 100 and for emergency flight stop when external force is detected during flight.

  The acceleration sensor 315 detects the acceleration of translational motion of the moving image pickup apparatus 100. The acceleration sensor 315 can detect triaxial acceleration and is used together with the gyroscope 312. The altitude detection unit 313 detects the flight altitude of the mobile imaging device 100. For example, an ultrasonic sensor is used, an ultrasonic wave is emitted toward the ground, and the flying height can be detected by measuring the reflected wave. The position detection unit 314 detects the position information of the mobile imaging device 100. The position detection unit 314 is a positioning device that uses, for example, GPS (Global Positioning System). The face image recognition unit 316 performs image processing for recognizing the face image of the subject in the image captured by the image capturing unit 308.

  Next, the zoom function of the communication device 200 will be described with reference to FIG. FIG. 4 is an image diagram illustrating a zoom function of the communication device 200 in a tabular format by exemplifying two cases (Case 1 and Case 2). Regarding the zoom function in the communication device 200, the user performs a zoom operation using the touch panel function on the LCD 205. Case 1 is a case where the user performs a slow zoom operation, that is, a change amount of the zoom operation is equal to or less than a threshold value. In this case, the communication device 200 transmits the movement control information to the mobile imaging device 100 to perform the first control for moving the mobile imaging device 100 to capture an image. In Case 2, the user performs a steep zoom operation, that is, the change amount of the zoom operation is larger than the threshold value. In this case, the communication device 200 transmits the imaging control information to the mobile imaging device 100, so that the second control is performed in which the imaging unit 308 is controlled to perform imaging without changing the position of the mobile imaging device 100. Hereinafter, a specific example of the zoom function as the imaging function will be described.

  In Case 1, when the user performs a slow zoom operation, the CPU 201 executes the zoom instruction change amount calculation program and calculates the zoom change speed (denoted as Sz). The zoom change speed Sz corresponds to the change amount of the zoom instruction signal when changing the zoom. Next, the CPU 201 executes the movement performance calculation program and calculates the movement performance (denoted as Sd) of the mobile imaging device 100. The CPU 201 compares the zoom change speed Sz and the movement performance Sd to determine whether or not a response can be made by the movement of the moving image pickup apparatus 100. That is, when the change amount of the zoom operation is less than or equal to the threshold value corresponding to the moving performance Sd, the angle of view can be controlled by moving the moving image capturing apparatus 100 toward or away from the subject. In that case, the movement control information transmitting unit 208 transmits the movement control information and the moving image pickup apparatus 100 receives the movement control information according to a command from the CPU 201. For example, the moving image capturing apparatus 100 moves in the subject direction (the image capturing direction of the image capturing unit). At this time, since the imaging control information transmitting unit 210 does not transmit the imaging control information corresponding to the zoom operation, the zoom position of the mobile imaging device 100 is not changed.

  In Case 2, when the user performs a sharp zoom operation, the CPU 201 executes the zoom instruction change amount calculation program to calculate the zoom change speed Sz. Next, the CPU 201 executes the movement performance calculation program and calculates the movement performance Sd. The CPU 201 compares the zoom change speed Sz with the moving performance Sd and determines that the moving speed of the moving image pickup apparatus 100 is unresponsive. In that case, the imaging control information transmitting unit 210 transmits the imaging control information including the zoom control information according to the instruction of the CPU 201, and the mobile imaging apparatus 100 receives the imaging control information. For example, in the moving image pickup apparatus 100, the zoom position of the image pickup unit is on the tele side (the telephoto side where the focal length of the zoom lens is longer than a predetermined value). The zoom position of the image pickup unit 308 is changed by the CPU 301 and the cradle 307 of the moving image pickup apparatus 100. At this time, since the moving state receiving unit 209 does not transmit the moving control information, the moving image pickup apparatus 100 does not move from the current position.

  As described above, when the change amount of the zoom operation is equal to or less than the threshold value, the view angle is changed by the movement control of the moving image pickup apparatus 100. When the change amount of the zoom operation is larger than the threshold value, the view angle is changed by controlling the zoom position of the image pickup unit 308 without performing the movement control of the moving image pickup apparatus 100. The threshold value is set according to the moving performance of the mobile imaging device 100.

  In FIG. 4, the zoom operation to the tele side based on the user's operation instruction has been described. On the wide side (on the wide-angle side where the focal length of the zoom lens is shorter than a predetermined value), in Case 1, processing for transmitting movement control information for moving the moving image pickup apparatus 100 in a direction away from the subject is performed. In Case 2, the zoom control information transmission processing of the image capturing unit 308 is performed so that the zoom position is on the wide side.

  Next, control by the communication device 200 will be described with reference to FIG. FIG. 5 is a flow chart for explaining the operation relating to the zoom function, and the operation shown below is executed every predetermined time.

  First, the CPU 201 executes the image capturing operation input program and determines whether or not the user has performed a zoom operation (S401). When it is determined in S401 that the user has not performed the zoom operation, the operation of the zoom function is ended. If it is determined in S401 that the user has performed the zoom operation, the process proceeds to S402.

  The CPU 201 executes the zoom instruction change amount calculation program to calculate the zoom change speed Sz (S402). Next, the CPU 201 executes the movement performance calculation program to calculate the movement performance Sd of the mobile imaging device 100 (S403). The CPU 201 compares the zoom change speed Sz calculated in S402 and S403 with the movement performance Sd, and determines whether or not the movement of the moving image pickup apparatus 100 cannot respond to the zoom operation instruction from the user (S404). If it is determined that the amount of change in the zoom operation is large and the moving imaging device 100 cannot move to respond, the process proceeds to step S405. When it is determined that the change amount of the zoom operation is small and it is possible to respond by moving the moving image capturing apparatus 100, the process proceeds to S406.

  In step S405, the CPU 201 causes the imaging control information transmission unit 210 to transmit zoom control information corresponding to the zoom change speed Sz to the mobile imaging device 100. In step S406, the CPU 201 causes the movement control information transmission unit 208 to transmit movement control information based on the zoom change speed Sz and the movement performance Sd to the moving image pickup apparatus 100. The movement control information is movement control information of the moving image pickup apparatus 100 for changing the shooting position. After the processing of S405 and S406, the operation of the zoom function ends.

  In the present embodiment, an example has been described in which the imaging control information transmitting unit 210 and the movement control information transmitting unit 208 exclusively transmit the control information according to the determination result in S404. Depending on the determination result, both the imaging control information transmitting unit 210 and the movement control information transmitting unit 208 may transmit the control information. For example, it is assumed that the CPU 201 determines that the zoom change speed Sz cannot respond by the zoom response performance of the image capturing unit of the moving image capturing apparatus 100 as a result of comparing the zoom change velocity Sz and the zoom response performance of the image capturing unit 308. In this case, a process of transmitting both the zoom control information of the image capturing unit 308 (zoom position control instruction information) and the movement control information of the mobile image capturing apparatus 100 is executed. In this case, the first control includes control of the imaging operation that uses the movement control of the moving imaging device 100 and the zoom control of the imaging unit 308 together. As a result, it becomes possible to respond at a speed higher than the zoom response performance of the image capturing unit 308. Further, in the present embodiment, the LCD 205 displays the image pickup state information corresponding to the image pickup control information (zoom position, etc.) of the image pickup unit 308 or the moving state information of the moving image pickup apparatus 100 corresponding to the movement control information in accordance with the zoom operation of the user. May be displayed on the screen. This allows the user to visually recognize the operating state (imaging state, moving state) of the mobile imaging device 100. These matters are the same in the embodiments described later.

  According to the present embodiment, by adaptively controlling the movement control of the moving image pickup apparatus 100 and the zoom control of the image pickup unit 308, the user can perform desired framing with an easy operation. That is, the user can realize a desired framing by only performing the zoom operation without performing the operation of moving the mobile imaging device 100.

[Second Embodiment]
In the second embodiment of the present invention, an automatic photographing position adjusting function in the communication device 200 will be described. The automatic shooting position adjustment function is a function of changing the shooting position and zoom position of the moving image pickup apparatus 100 so that the zoom position of the camera is always at the center position. Since the configuration of the device in this embodiment is the same as that in the first embodiment, by using already used reference numerals, detailed description thereof will be omitted, and different points will be described.

  The automatic photographing position adjusting function will be described with reference to FIG. As described with reference to FIG. 4 in the first embodiment, the zoom function in the communication device 200 is for the user to perform a zoom operation using the touch panel function of the LCD 205. Case 1 and Case 2 are the same as in FIG. In this embodiment, Case 3 is added as a case where the user has not performed a zoom operation for a predetermined time or longer.

  In Case 3, it is assumed that the camera zoom position of the mobile imaging device 100 is on the tele side. When the user does not perform the zoom operation in this state, the zoom change speed Sz calculated by the zoom instruction change amount calculation program becomes zero. In this case, the CPU 201 executes the zoom position detection program and acquires the camera zoom position of the moving image pickup apparatus 100. From the acquisition result, the CPU 201 determines that the camera zoom position is on the tele side, transmits the movement control information via the movement control information transmission unit 208, and transmits the zoom position control information using the imaging control information transmission unit 210. To do. The movement control information at this time is control information for moving the moving image pickup apparatus 100 toward the subject. The zoom control information is control information for setting the zoom position to the center position, and is controlled so that the change amount of the image pickup range approaches zero based on the change amount of the image pickup range obtained by using the image pickup range detection program. This is control information to be executed.

  In Case 3 of FIG. 6, the case where the zoom operation is not performed in the state where the zoom position of the imaging unit 308 is on the tele side has been described. When the zoom operation is not performed in the state where the zoom position is on the wide side, the movement control information is control information for moving the moving image pickup apparatus 100 in a direction away from the subject. The zoom control information is control information for setting the camera zoom position to the center position.

The operation of the automatic photographing position adjusting function in this embodiment will be described with reference to the flowchart of FIG. 7. The following operation is performed every predetermined time.
In step S601, the CPU 201 executes the image capturing operation input program and determines whether the user has performed a zoom operation. If the user has performed a zoom operation, the process proceeds to S602. Since S602 to S606 are the same as the processes of S402 to S406 in FIG. 5, their description will be omitted. When it is determined in S601 that the user has not performed the zoom operation for the predetermined time or longer, the process proceeds to S607.

  In step S607, the CPU 201 executes the zoom position detection program and acquires the camera zoom position of the moving image pickup apparatus 100. Then, the CPU 201 determines whether or not the camera zoom position acquired in S607 is on the tele side from the center position (S608). If it is determined in S608 that the camera zoom position is on the tele side of the center position, the process proceeds to S609, and if it is determined that the camera zoom position is not on the tele side of the center position, the process proceeds to S610.

  In step S <b> 609, the CPU 201 uses the movement control information transmission unit 208 to transmit movement control information for moving the moving image pickup apparatus 100 toward the subject (moving the photographing position forward). Further, in step S610, the CPU 201 determines whether the camera zoom position acquired in step S607 is on the wide side from the center position. If it is determined that the camera zoom position is not on the wide side of the center position, the operation of the automatic shooting position adjustment function ends. If it is determined in S610 that the camera zoom position is wider than the center position, the process proceeds to S611. The CPU 201 uses the movement control information transmission unit 208 to transmit movement control information for moving the moving image pickup apparatus 100 in the direction opposite to the subject direction (retracting the shooting position) (S611).

  After the processing of S609 and S611, the CPU 201 executes the imaging range detection program and acquires the amount of change in the imaging range (S612). Next, the CPU 201 transmits the zoom control information using the imaging control information transmission unit 210 (S613). The zoom control information at this time is control information of the camera zoom position in which the amount of change in the imaging range acquired in S612 is close to zero and the zoom position is the center position. After the processing of S613, the operation of the automatic photographing position adjusting function is ended.

  In the present embodiment, when it is detected by the automatic shooting position adjusting function that the user does not perform the zoom operation, the camera zoom position of the moving image pickup apparatus 100 is set to the center without suppressing the change of the image pickup range and changing the image pickup range as much as possible. Set to position. For example, when the change amount (change speed) of the zoom operation is large, the zoom position of the image capturing unit 308 is changed, but thereafter, control is performed to move the moving image capturing apparatus 100 so that the zoom position becomes the center position. Therefore, the telephoto side and the wide side can quickly respond to the zoom operation performed by the user next time, so that the framing response performance can be improved.

[Third Embodiment]
In the third embodiment of the present invention, a switching operation of control information according to a communication status in the communication device 200 will be described. In the present embodiment, the communication state between the mobile imaging device 100 and the communication device 200 is determined, and when the communication state of the imaging control information decreases and the communication state of the movement control information is good, the position of the mobile imaging device 100 is determined. Control to move is performed. Further, when the communication state of the movement control information is deteriorated and the communication state of the image pickup control information is good, control for changing the zoom position of the image pickup unit 308 is performed.

  FIG. 8 is an image diagram for explaining the zoom method switching operation according to the communication status in the communication device 200 according to the present embodiment in a table format. In the zoom operation performed by the user using the touch panel function of the LCD 205, Case 4 and Case 5 are illustrated. Case 4 is a case where the communication state of the imaging control information has deteriorated. Case 5 is a case where the communication state of the mobility control information is degraded.

  In Case 4, when the user performs a zoom operation to the tele side, the CPU 201 executes the communication state detection program and detects the communication states of the movement control information transmission unit 208 and the imaging control information transmission unit 210. When it is determined that the communication state of the imaging control information has deteriorated as the detected communication state, the imaging control information transmitting unit 210 cannot transmit the zoom control information. Therefore, the zoom position of the moving image pickup device 100 cannot be controlled. On the other hand, it is assumed that the communication state of the mobile control information is good. The CPU 201 uses the movement control information transmission unit 208 to transmit the movement control information to the mobile imaging device 100. The moving image pickup apparatus 100 moves toward the subject according to the movement control information.

  In Case 5, when the user performs a zoom operation to the tele side, the CPU 201 executes the communication state detection program and detects the communication states of the movement control information transmission unit 208 and the imaging control information transmission unit 210. When it is determined that the communication state of the mobility control information has deteriorated as the detected communication state, the mobility control information transmitting unit 208 cannot transmit the mobility control information. Therefore, the mobile imaging device 100 cannot move from the current position. On the other hand, it is assumed that the communication state of the imaging control information is good. The CPU 201 transmits the zoom control information to the mobile image pickup apparatus 100 by using the image pickup control information transmitting unit 210. The zoom position of the moving image pickup apparatus 100 is changed to the position on the tele side according to the zoom control information.

  In FIG. 8, the example in which the zoom operation by the user is the operation to the tele side has been described, but in the case of the zoom operation to the wide side, in Case 4, the moving image pickup apparatus 100 is moved in the direction away from the subject, and the movement control information is transmitted. It In Case 5, zoom control information for setting the zoom position to the wide side is transmitted.

The switching operation of the zoom method according to the communication status will be described with reference to the flowchart in FIG. 9. The following operation is performed every predetermined time.
The CPU 201 executes the imaging operation input program and determines whether the user has performed a zoom operation (S801). If it is determined in S801 that the user has not performed a zoom operation, the zoom method switching operation ends. If it is determined in S801 that the user has performed the zoom operation, the process proceeds to S802. The CPU 201 executes the communication state detection program to detect the communication state of the imaging control information transmission unit 210 (hereinafter, the state value is referred to as Tz) (S802).
In the next step S803, the CPU 201 executes the communication state detection program to detect the communication state of the movement control information transmission unit 208 (hereinafter, the state value is referred to as Td). The CPU 201 compares the communication state value Tz of the imaging control information transmission unit 210 detected in S802 with the communication state value Td of the movement control information transmission unit 208 detected in S803, and the movement control information transmission unit 208 can be used. It is determined whether or not (S804). If it is determined in S804 that the movement control information transmitting unit 208 is not usable, that is, if the communication state value Td has decreased, the process proceeds to S805. If it is determined that the movement control information transmitting unit 208 is usable, the process proceeds to S807.

  In step S805, the CPU 201 determines whether the imaging control information transmission unit 210 can be used. When it is determined that the imaging control information transmitting unit 210 is not usable, that is, when the communication state value Tz has decreased, the communication process ends. If it is determined in S805 that the imaging control information transmitting unit 210 is usable, the process proceeds to S806.

  In step S806, the CPU 201 uses the imaging control information transmission unit 210 to transmit zoom control information to the mobile imaging device 100 in order to change the camera zoom position. Further, in step S807, the CPU 201 uses the movement control information transmission unit 208 to transmit movement control information for moving the mobile imaging device 100. After the processing of S806 and S807, the zoom method switching operation ends.

  In the present embodiment, when the user performs a zoom operation, the control information of the camera zoom position, the movement information of the mobile imaging device 100, and the detection result of the communication state detected by the communication state detection program are displayed on the screen of the LCD 205. To be done. This allows the user to recognize the operating state and the communication state of the mobile imaging device 100.

  According to the present embodiment, the zoom method switching operation is performed according to the communication status, and it is possible to cope with the situation in which the communication status of the movement control information transmission unit 208 or the imaging control information transmission unit 210 is lowered. That is, since the process of adaptively changing the framing matching method (movement of the unmanned device or the zoom state of the camera) is performed according to the communication state, the user can realize the desired framing only by performing the zoom operation.

[Other Embodiments]
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Mobile Imaging Device 200 Communication Device 201 Main CPU
208 Mobility Control Information Transmitter 209 Mobility State Receiver 210 Imaging Control Information Transmitter 211 Imaging State Receiver

Claims (31)

An imaging system comprising a moving imaging device having an imaging function and a moving function,
A receiving means for receiving a zoom instruction,
Depending on the rate of change of the zoom indicated zoom instruction accepted by the receiving with means, determination determines a first control for movement control, whether to perform one of the control of the second control for zoom control Means and
An imaging system comprising: a control unit that performs the first control and the second control based on a determination result by the determination unit. An imaging system comprising a moving imaging device having an imaging function and a moving function,
A receiving means for receiving a zoom instruction,
A determination unit that determines which of a first control that performs movement control and a second control that performs zoom control, according to the zoom instruction received by the reception unit;
Control means for performing the first control and the second control based on the determination result by the determination means;
When the first control is performed, the moving state information of the moving imaging device corresponding to the movement control is displayed, and when the second control is performed, the imaging state information corresponding to the zoom control is displayed. An image pickup system comprising: a display unit for displaying. An imaging system comprising a moving imaging device having an imaging function and a moving function,
A receiving means for receiving a zoom instruction,
A determination unit that determines which of a first control that performs movement control and a second control that performs zoom control, according to the zoom instruction received by the reception unit;
A control unit that performs the first control and the second control based on a determination result by the determination unit,
When the zoom instruction is not issued and the zoom position is closer to the telephoto side than the center position, the control means performs the control of moving the moving image pickup apparatus forward with respect to the subject in the first control, and the zoom instruction is issued. When not performed and when the zoom position is on a wider angle side than the central position, the moving image pickup device is controlled to retract with respect to the subject by the first control. An imaging system comprising a moving imaging device having an imaging function and a moving function,
A receiving means for receiving a zoom instruction,
A determination unit that determines which of a first control that performs movement control and a second control that performs zoom control, according to the zoom instruction received by the reception unit;
A control unit that performs the first control and the second control based on a determination result by the determination unit,
When the zoom instruction is not issued, the control means moves the moving image pickup device under the first control and performs control for reducing the change of the image pickup range due to the movement of the moving image pickup device by the zoom control. Imaging system characterized by. An imaging system including a mobile imaging device having an imaging function and a moving function, and a communication device communicating with the mobile imaging device,
A receiving means for receiving a zoom instruction,
A determination unit that determines which of a first control that performs movement control and a second control that performs zoom control, according to the zoom instruction received by the reception unit;
A control unit that performs the first control and the second control based on a determination result by the determination unit,
The communication device is
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
The receiving means ,
The determining means detects the communication states of the first and second transmitting means, the value indicating the communication state of the first transmitting means decreases, and the value indicating the communication state of the second transmitting means. Is determined not to decrease, the second control is performed, and if the value indicating the communication state of the first transmission unit is not decreased, the first control is determined to be performed. Imaging system. A communication device for communicating with a mobile imaging device having an imaging function and a moving function,
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
Depending on the change rate of the zoom indicated by the zoom instruction received by the receiving with means, first control and performing move control of the moving imager by sending the movement control information by said first transmission means Determination means for determining which of the second control for performing zoom control of the moving image pickup apparatus by transmitting the zoom control information by the second transmission means,
A communication device comprising: a control unit that performs the first control and the second control based on a determination result by the determination unit. The determination means determines to perform the first control when the change speed is equal to or less than a threshold value and to perform the second control when the change speed is greater than the threshold value. Item 7. The communication device according to item 6. The determination means acquires the information on the moving performance of the moving image pickup device and compares the changing speed with the information on the moving performance to perform the first control or the second control. The communication device according to claim 6, wherein the communication device determines whether or not it is. The control means performs the first control when the changing speed is less than or equal to the threshold with respect to a threshold value corresponding to the moving performance of the moving image pickup device, and the first control when the changing speed is greater than the threshold value. 9. The communication device according to claim 8, wherein the communication device performs control of 2. When the first control is performed, the moving state information of the moving image pickup device corresponding to the movement control information is displayed, and when the second control is performed, the image pickup state corresponding to the zoom control information is displayed. The communication device according to claim 6, further comprising a display unit that displays information. A communication device for communicating with a mobile imaging device having an imaging function and a moving function,
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
First control for performing movement control of the moving image pickup apparatus by transmitting the movement control information by the first transmission means in response to a zoom instruction received by the reception means; and second transmission means. Determining means for determining which of the second control for performing zoom control of the moving image pickup apparatus by transmitting the zoom control information by means of:
Control means for performing the first control and the second control based on the determination result by the determination means;
When the first control is performed, the moving state information of the moving image pickup device corresponding to the movement control information is displayed, and when the second control is performed, the image pickup state corresponding to the zoom control information is displayed. A communication device comprising: display means for displaying information. The determining means detects the communication states of the first and second transmitting means, the value indicating the communication state of the first transmitting means decreases, and the value indicating the communication state of the second transmitting means. Is determined not to decrease, the second control is performed, and if the value indicating the communication state of the first transmission unit is not decreased, the first control is determined to be performed. The communication device according to claim 11. A receiver for acquiring the zoom position of the moving image pickup device,
The control unit advances the moving image pickup apparatus to the subject by the first control when the zoom instruction is not issued and the zoom position acquired by the receiving unit is on the telephoto side from the center position. Control is performed, a zoom instruction is not issued, and when the zoom position acquired by the receiving unit is on the wide-angle side with respect to the central position, the moving image pickup device is moved to the subject by the first control. The communication device according to claim 6, wherein the communication device is controlled to move backward. The control means further compares the zoom position acquired by the reception means with the center position, and if the zoom position is not the center position, the second control causes the zoom position to approach the center position. The communication device according to claim 13, wherein A communication device for communicating with a mobile imaging device having an imaging function and a moving function,
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
First control for performing movement control of the moving image pickup apparatus by transmitting the movement control information by the first transmission means in response to a zoom instruction received by the reception means; and second transmission means. Determining means for determining which of the second control for performing zoom control of the moving image pickup apparatus by transmitting the zoom control information by means of:
Control means for performing the first control and the second control based on the determination result by the determination means;
A receiving unit for acquiring the zoom position of the moving image pickup device,
The control unit advances the moving image pickup apparatus to the subject by the first control when the zoom instruction is not issued and the zoom position acquired by the receiving unit is on the telephoto side from the center position. Control is performed, a zoom instruction is not issued, and when the zoom position acquired by the receiving unit is on the wide-angle side with respect to the central position, the moving image pickup device is moved to the subject by the first control. A communication device characterized by performing control for retreating. The control means further compares the zoom position acquired by the reception means with the center position, and if the zoom position is not the center position, the second control causes the zoom position to approach the center position. The communication device according to claim 15, wherein The determining means detects the communication states of the first and second transmitting means, the value indicating the communication state of the first transmitting means decreases, and the value indicating the communication state of the second transmitting means. Is determined not to decrease, the second control is performed, and if the value indicating the communication state of the first transmission unit is not decreased, the first control is determined to be performed. The communication device according to claim 15 or 16. A receiving unit for acquiring the imaging range of the mobile imaging device,
When the zoom instruction is not issued, the control means moves the moving image pickup device under the first control and performs control for reducing the change of the image pickup range due to the movement of the moving image pickup device by the zoom control. The communication device according to any one of claims 6 to 17, characterized in that. A communication device for communicating with a mobile imaging device having an imaging function and a moving function,
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
The first transmission unit transmits the movement control information to control the movement of the mobile imaging device, and the second transmission unit transmits the zoom control information to perform the movement imaging. Control means for performing a second control for zoom control of the device;
A receiving unit for acquiring the imaging range of the mobile imaging device,
When the zoom instruction is not issued, the control means moves the moving image pickup device under the first control and performs control for reducing the change of the image pickup range due to the movement of the moving image pickup device by the zoom control. A communication device characterized by. A determination unit that determines which of the first control and the second control is to be performed,
When the zoom instruction is given, the determination means detects the communication state of the first and second transmission means, the value indicating the communication state of the first transmission means decreases, and the second The second control is performed when the value indicating the communication state of the transmitting means is not reduced, and the first control is performed when the value indicating the communication state of the first transmitting means is not reduced. To determine
20. The communication device according to claim 19, wherein the control unit performs the first control and the second control based on a determination result by the determination unit. A communication device for communicating with a mobile imaging device having an imaging function and a moving function,
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
The first transmission unit transmits the movement control information to control the movement of the mobile imaging device, and the second transmission unit transmits the zoom control information to perform the movement imaging. Determination means for determining which control is to be performed, the second control for performing zoom control of the apparatus,
A control unit that performs the first control and the second control based on a determination result by the determination unit,
The determining means detects the communication states of the first and second transmitting means, the value indicating the communication state of the first transmitting means decreases, and the value indicating the communication state of the second transmitting means. Is determined not to decrease, the second control is performed, and if the value indicating the communication state of the first transmission unit is not decreased, the first control is determined to be performed. Communication device. Imaging means for imaging the subject,
Drive means controlled when the moving image pickup apparatus moves,
Communication means for receiving movement control information and zoom control information from the communication device, and transmitting information relating to the movement state or zoom state,
First control means for controlling said image pickup means and said drive means, a moving image pickup apparatus Ru provided with,
First transmitting means for transmitting the movement control information to the mobile imaging device;
Second transmitting means for transmitting the zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
A first control for controlling the movement of the moving image pickup apparatus by transmitting the movement control information by the first transmitting means in accordance with a zoom change speed indicated by a zoom instruction received by the receiving means; Determination means for determining which of the second control for performing zoom control of the moving image pickup apparatus by transmitting the zoom control information by the second transmission means,
Second control means for performing the first control and the second control based on the determination result by the determination means;
The moving image pickup apparatus, wherein the first control unit controls the image pickup unit and the driving unit in accordance with an instruction from the communication device. Imaging means for imaging the subject,
Drive means controlled when the moving image pickup apparatus moves,
Communication means for receiving movement control information and zoom control information from the communication device, and transmitting information relating to the movement state or zoom state,
A moving image pickup apparatus comprising: a first control unit that controls the image pickup unit and the driving unit,
First transmitting means for transmitting the movement control information to the mobile imaging device;
Second transmitting means for transmitting the zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
First control for performing movement control of the moving image pickup apparatus by transmitting the movement control information by the first transmission means in response to a zoom instruction received by the reception means; and second transmission means. Determining means for determining which of the second control for performing zoom control of the moving image pickup apparatus by transmitting the zoom control information by means of:
Second control means for performing the first control and the second control based on the determination result by the determination means;
When the first control is performed, the moving state information of the moving image pickup device corresponding to the movement control information is displayed, and when the second control is performed, the image pickup state corresponding to the zoom control information is displayed. Display means for displaying information,
The moving image pickup apparatus, wherein the first control unit controls the image pickup unit and the driving unit in accordance with an instruction from the communication device. Imaging means for imaging the subject,
Drive means controlled when the moving image pickup apparatus moves,
Communication means for receiving movement control information and zoom control information from the communication device, and transmitting information relating to the movement state or zoom state,
A moving image pickup apparatus comprising: a first control unit that controls the image pickup unit and the driving unit,
The communication device is
First transmitting means for transmitting the movement control information to the mobile imaging device;
Second transmitting means for transmitting the zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
First control for performing movement control of the moving image pickup apparatus by transmitting the movement control information by the first transmission means in response to a zoom instruction received by the reception means; and second transmission means. Determining means for determining which of the second control for performing zoom control of the moving image pickup apparatus by transmitting the zoom control information by means of:
Second control means for performing the first control and the second control based on the determination result by the determination means;
A receiving unit for acquiring the zoom position of the moving image pickup device,
The first control means is
When the zoom instruction is not issued, and when the zoom position acquired by the receiving unit is on the telephoto side with respect to the center position, control for advancing the moving imaging device with respect to the subject is performed by the first control, When the zoom instruction is not issued and the zoom position acquired by the receiving unit is on the wide-angle side of the center position, the first image pickup control is performed to move the moving image pickup device backward with respect to the subject. A moving image pickup device characterized by controlling the image pickup means and the drive means in accordance with an instruction from the communication device. Imaging means for imaging the subject,
Drive means controlled when the moving image pickup apparatus moves,
Communication means for receiving movement control information and zoom control information from the communication device, and transmitting information relating to the movement state or zoom state,
A moving image pickup apparatus comprising: a first control unit that controls the image pickup unit and the driving unit,
The communication device is
First transmitting means for transmitting the movement control information to the mobile imaging device;
Second transmitting means for transmitting the zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
The first transmission unit transmits the movement control information to control the movement of the moving image pickup device, and the second transmission unit transmits the zoom control information to move the moving image. Second control means for performing second control for performing zoom control of the device;
A receiving unit for acquiring the imaging range of the mobile imaging device,
The first control means is
When the zoom instruction is not issued, the instruction from the communication device that moves the moving imaging device under the first control and reduces the change in the imaging range due to the movement of the moving imaging device by the zoom control. A moving image pickup apparatus characterized by controlling the image pickup means and the driving means according to the above. Imaging means for imaging the subject,
Drive means controlled when the moving image pickup apparatus moves,
Communication means for receiving movement control information and zoom control information from the communication device, and transmitting information relating to the movement state or zoom state,
A moving image pickup apparatus comprising: a first control unit that controls the image pickup unit and the driving unit,
The communication device is
First transmitting means for transmitting the movement control information to the mobile imaging device;
Second transmitting means for transmitting the zoom control information to the mobile imaging device;
Receiving means for receiving a zoom instruction which is an operation instruction for the zoom function of the moving image pickup device;
The first transmission unit transmits the movement control information to control the movement of the mobile imaging device, and the second transmission unit transmits the zoom control information to perform the movement imaging. Determination means for determining which control is to be performed, the second control for performing zoom control of the apparatus,
A second control unit that performs the first control and the second control based on a determination result by the determination unit,
The first control means is
The communication state of the first and second transmitting means is detected, the value indicating the communication state of the first transmitting means is lowered, and the value indicating the communication state of the second transmitting means is not lowered. In the case where the second control is performed, and the determination unit determines to perform the first control when the value indicating the communication state of the first transmission unit does not decrease, the instruction from the communication device A moving image pickup apparatus characterized by controlling the image pickup means and the driving means according to the above. A control method executed in an imaging system including a moving imaging device having an imaging function and a moving function,
A step of receiving a zoom instruction,
A determination step of determining whether to perform a first control for performing a movement control or a second control for performing a zoom control according to the zoom change speed indicated by the received zoom instruction ;
A control method of an imaging system, comprising: a control step of performing the first control and the second control based on a determination result of the determination step. A control method executed in an imaging system including a moving imaging device having an imaging function and a moving function,
A step of receiving a zoom instruction,
A determination step of determining which of a first control for performing a movement control and a second control for performing a zoom control to be performed according to the received zoom instruction;
A control step of performing the first control and the second control based on the determination result of the determination step;
When the first control is performed, the moving state information of the moving imaging device corresponding to the movement control is displayed, and when the second control is performed, the imaging state information corresponding to the zoom control is displayed. And a display step of displaying the image. A control method executed in an imaging system including a moving imaging device having an imaging function and a moving function,
A step of receiving a zoom instruction,
A determination step of determining which of a first control for performing a movement control and a second control for performing a zoom control to be performed according to the received zoom instruction;
A control step of performing the first control and the second control based on the determination result of the determination step,
In the control step, when the zoom instruction is not issued and the zoom position is closer to the telephoto side than the center position, the control for moving the moving imaging device forward with respect to the subject is performed in the first control, and the zoom instruction is performed. Is not performed, and when the zoom position is on the wide-angle side of the center position, control for retracting the moving imaging device with respect to the subject is performed by the first control. Control method. A control method executed in an imaging system including a moving imaging device having an imaging function and a moving function,
A step of receiving a zoom instruction,
A determination step of determining which of a first control for performing a movement control and a second control for performing a zoom control to be performed according to the received zoom instruction;
A control step of performing the first control and the second control based on the determination result of the determination step,
In the control step, when the zoom instruction is not issued, the control for moving the moving image pickup apparatus by the first control and reducing the change of the image pickup range due to the movement of the moving image pickup apparatus by the zoom control is performed. A method for controlling an imaging system, comprising: A control method executed in an imaging system including a mobile imaging device having an imaging function and a moving function, and a communication device communicating with the mobile imaging device,
The communication device is
First transmission means for transmitting movement control information to the mobile imaging device;
Second transmitting means for transmitting zoom control information to the mobile imaging device;
And a receiving means,
The control method is
A step of receiving a zoom instruction by the receiving means,
A determination step of determining which of a first control for performing a movement control and a second control for performing a zoom control to be performed according to the received zoom instruction;
A control step of performing the first control and the second control based on the determination result of the determination step,
In the determining step, the communication state of the first and second transmitting means is detected, the value indicating the communication state of the first transmitting means decreases, and the value indicating the communication state of the second transmitting means decreases. Is determined not to decrease, the second control is determined to be performed, and if the value indicating the communication state of the first transmission unit is not reduced, it is determined to perform the first control. A method for controlling an imaging system, comprising:

Images