JP2019200398A - Imaging system, control method therefor, program, and storage medium - Google Patents

Abstract

To realize a smooth focus adjustment control in a system for controlling a camera attached to a robot, even when a distance to an object varies.SOLUTION: Provided is an imaging system comprising an imaging device attached to a robot arm and a controller for controlling the robot arm and the imaging device. The controller includes a notification unit for notifying the imaging device of subject distance information relative to the imaging device. The imaging device includes: an imaging unit for capturing a subject image; a drive unit for driving a focus lens that adjusts the focus of an imaging optical system; and a control unit for exercising first drive control for driving, on the basis of the subject distance information received from the controller, the focus lens to a position that corresponds to the subject distance information and then acquiring a history of change of focus evaluation values from the image obtained by the imaging unit, and second drive control for determining, on the basis of a history of change of focus evaluation values obtained by first drive control, a direction in which the focus lens is driven and performing automatic focus adjustment.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an imaging system that controls a robot arm based on a captured image of a camera.

  For example, in an assembly robot or the like, a robot is used to control a robot when a workpiece is picked and placed by a robot arm. In this control, the work is photographed by a camera attached to the hand part of the robot arm, the position of the work is detected from the photographed image data, the robot arm is moved to the detection position, and the work is gripped by the hand. Is.

  As this type of robot control device, Patent Document 1 discloses that a camera has an optical system adjustment unit having a focus adjustment function, a focal length adjustment function, an aperture adjustment function, and the like, and is optical based on photographed image data obtained by photographing a workpiece. An apparatus for controlling the system adjustment means is described.

JP 2003-211382 A

  However, in the above-described prior art, the feature amount of the object is stored as target data by visual feedback control called visual servo, and the focus, focal length, and aperture are reduced so that the error is less than when the target data is stored. It has a configuration to adjust.

  In this control, the object is photographed, the photographed photographed image is transmitted to the visual recognition device, the visual recognition device compares the target data with the current data, calculates the error, and reduces the error (becomes 0). ) Repeat the robot control and camera control again. Therefore, it takes time to reduce the error. In that case, the smooth movement of the operation of the robot arm is hindered.

  Also, when focusing on an object, if the subject is photographed and the focus position is adjusted while checking whether the subject is in focus or not, the distance to the object varies. In this case, it is necessary to perform the same control at each distance, which takes time.

  For this reason, it may be possible to provide an autofocus function that automatically focuses, but when searching for a focus position from a position away from the focus position, there is a problem that it takes time to focus. .

  The present invention has been made in view of the above-described problems, and its purpose is to realize a smooth focus adjustment control even when the distance to an object is various in a system for controlling a camera attached to a robot. It is to be.

  An imaging system according to the present invention is an imaging system that includes an imaging device attached to a robot arm, and a control device that controls the robot arm and the imaging device. Notification means for notifying subject distance information to the imaging device, the imaging device from an imaging means for imaging a subject image, a driving means for driving a focus lens for focusing an imaging optical system, and the control device First drive control for acquiring a history of changes in focus evaluation values from an image obtained by the imaging unit while driving the focus lens to a position corresponding to the subject distance information based on the received subject distance information And determining the direction in which the focus lens is driven based on the history of changes in the focus evaluation value obtained by the first drive control, And a control means for performing a second drive control for performing dynamic focusing, and wherein the.

  According to the present invention, in a system for controlling a camera attached to a robot, smooth focus adjustment control can be realized even when the distance to an object is various.

1 is a block diagram illustrating a configuration of an imaging system according to an embodiment of the present invention. The figure which represented typically the external appearance of an imaging device, an image processing controller, and a robot arm. The flowchart which shows operation | movement of a focus manual drive process. The flowchart which shows the operation | movement of a one-shot AF process. The figure which shows the change of AF evaluation value with respect to the position of a focus lens.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

<Configuration of imaging system>
An imaging system according to an embodiment of the present invention will be described. In this imaging system, focus adjustment control of the imaging device installed in the robot arm is performed based on the history of the AF evaluation value signal.

  FIG. 1A is a block diagram illustrating a configuration of an imaging system according to an embodiment of the present invention. This imaging system includes an imaging device 100, an image processing controller 120, and a robot arm 130 that are communicably connected.

  The imaging apparatus 100 includes a first fixed lens group 101, a variable power lens 102, a diaphragm 103, a second fixed lens group 104, and a focus lens 105 (focus compensator lens) as an imaging optical system. By moving the zoom lens 102 in the direction along the optical axis, zooming can be performed and the focal length can be changed. The focus lens 105 has both a function of correcting the movement of the focal plane due to zooming and a focusing function.

  A zoom drive source 109 is a drive source for moving the variable magnification lens 102. The aperture drive source 110 is a drive source for moving the aperture 103. A focusing drive source 111 is a drive source for moving the focus lens 105. The zoom drive source 109, the aperture drive source 110, and the focusing drive source 111 are configured by actuators such as a stepping motor, a DC motor, a vibration motor, and a voice coil motor.

  The imaging apparatus 100 further includes an imaging element 106, a CDS / AGC circuit 107, a camera signal processing unit 108, a timing generator 112, an AF signal processing unit 113, a recording unit 115, a camera microcomputer 114 (hereinafter simply referred to as a camera microcomputer), communication A device 116 is provided.

  The image sensor 106 is an element that photoelectrically converts a subject image, and includes a CCD, a CMOS sensor, or the like. The imaging element 106 has a plurality of pixel portions in a matrix. The light beam that has passed through the imaging optical system forms an image on the light receiving surface of the imaging element 106 and is converted into a signal charge corresponding to the amount of incident light by a photodiode (photoelectric conversion unit) included in each pixel unit. The signal charge accumulated in each photodiode is sequentially read out from the image sensor 106 as a voltage signal corresponding to the signal charge based on a drive pulse supplied from the timing generator 112 in accordance with a command from the camera microcomputer 114.

  The output signal read from the image sensor 106 is input to a CDS / AGC circuit 107 that samples and adjusts the gain. The camera signal processing unit 108 performs various types of image processing on the image signal output from the CDS / AGC circuit 107 to generate an image signal. The image signal in the first embodiment is a still image or moving image signal used for robot control. The storage unit 115 stores the image signal from the camera signal processing unit 108.

  The AF signal processing unit 113 outputs a high-frequency component or a luminance difference component (passed through the AF gate) from a signal that has passed through the AF gate that passes only the signal in the region used for focus detection among the output signals of all the pixels from the CDS / AGC circuit 107. The difference between the maximum and minimum values of the luminance level of the signal is extracted to generate an AF evaluation value signal. The AF evaluation value signal is output to the camera microcomputer 114. The AF evaluation value signal represents the sharpness (contrast state) of the video signal generated based on the output signal from the image sensor 106. The sharpness changes depending on the focus state of the image pickup optical system. Specifically, the AF evaluation value signal is a signal representing the focus state of the imaging optical system.

  The camera microcomputer 114 controls the overall operation of the imaging apparatus 100 and performs AF control (automatic focus adjustment control) for moving the focus lens 105 by controlling the focusing drive source 111. The camera microcomputer 114 performs AF control in the TV-AF method (hereinafter simply referred to as “TV-AF”) as AF control. The TV-AF method is a focus detection method that performs focus detection by contrast detection as described above. The camera microcomputer 114 controls the zoom drive source 109 to move the zoom lens 102 to perform focal length control, and controls the aperture drive source 110 to perform aperture control to operate the aperture 103.

  The communication device 116 is a device for communicating with the image processing controller 120. The image processing controller 120 instructs the imaging apparatus 100 to take a picture, analyze a shot image, and give instructions for each camera control (focus control, exposure control, focal length control). In addition, the distance to the subject is transmitted (notified) to the imaging device 100 based on the operation state of the robot arm 130 and the position state of the robot arm 130, and the reception state of the camera is received. Furthermore, an operation instruction to the robot arm 130 is performed. The operation state of the robot arm 130 and the position state of the robot arm 130 are determined from detection values of various sensors provided in the robot arm 130, instruction commands sent from the image processing controller 120 to the robot arm 130, and the like. The Data transmission / reception is performed according to a predetermined communication protocol. Here, data is transmitted and received using commands determined in advance for each type of instruction.

  The image processing controller 120 includes a CPU 121, a primary storage device 122, a secondary storage device 123, a communication device 124, a display unit 125, and an operation unit 126. The CPU 121 controls the entire image processing controller 120. The secondary storage device 123 is composed of a hard disk or the like, and stores a program for operating the CPU 121. The primary storage device 122 includes a RAM or the like, and stores a program read from the secondary storage device 123. The communication device 124 communicates with the imaging device 100 and the robot arm 130. The display unit 125 displays captured images, characters for interactive operation, and the like. The operation unit 126 receives a user operation.

  The imaging device 100 is fixed to the robot arm 130. When the robot arm 130 moves, the imaging range of the imaging device 100 is changed. Further, the robot arm 130 can hold an object to be operated using a robot hand at the tip of the arm. The CPU 131 controls the entire robot arm 130. The communication device 132 communicates with the image processing controller 120.

  FIG. 1B is a diagram schematically illustrating the appearance of the imaging apparatus 100, the image processing controller 120, and the robot arm 130. In this embodiment, it is assumed that the image processing controller 120 is a parent and sends instructions to the imaging apparatus 100 and the robot arm 130.

<Focus manual drive processing>
In the present embodiment, before performing AF (autofocus) processing, which will be described later, of the imaging apparatus, the focus lens 105 is manually driven according to a user instruction, and a focus evaluation value at each position of the focus lens 105 is acquired ( First drive control) is performed. FIG. 2 is a flowchart showing the operation of the focus manual driving process in the present embodiment.

  First, when a user inputs an instruction to start focus manual drive processing to the image processing controller 120 via the operation unit 126, the CPU 121 transmits subject distance information to the communication device 116 of the imaging device 100 via the communication device 124. The subject distance information may be data input from the operation unit 126, or data stored in the secondary storage device 123 may be read from the CPU 121.

  In S <b> 201, the camera microcomputer 114 of the imaging apparatus 100 determines whether subject distance information has been received from the image processing controller 120. If the subject distance information is received, the process proceeds to S202. If not, the process repeats S201 until the subject distance information is received.

  In S202, the camera microcomputer 114 calculates a target focus lens position corresponding to the subject distance information received in S201. Then, based on the positional relationship between the current focus lens position and the calculated target focus lens position, it is determined whether the focus lens 105 should be moved in the closest direction or the infinity direction in order to focus. The determined drive direction is stored in the storage unit 115.

  In S <b> 203, the camera microcomputer 114 sends a drive instruction to the focusing drive source 111 to drive the focus lens 105 in the drive direction determined in S <b> 202, and the focusing drive source 111 starts driving the focus lens 105. In the subsequent operations, the processing of S204 to S208 is repeated at a predetermined cycle until the focus lens 105 reaches the target focus lens position calculated in S202, and the focus evaluation value is acquired each time.

  In S <b> 204, the camera microcomputer 114 acquires the AF evaluation value output from the AF signal processing unit 113 and stores it in the storage unit 115 as AF evaluation value history information.

  In S205, the camera microcomputer 114 determines whether or not the AF evaluation value has passed the peak (that is, the focus lens position that is in focus) based on the AF evaluation value history information stored in S204. Whether or not the focus lens has passed the in-focus position is determined based on whether or not the latest AF evaluation value has decreased by a threshold value or more with respect to the maximum AF evaluation value in the AF evaluation value history information. Then, when the latest AF evaluation value is lower than the threshold value, the focus lens position when the image having the maximum or maximum AF evaluation value is accumulated is set as the in-focus position.

  In S206, if there is an AF evaluation value peak in S205, the process proceeds to S207, and if there is no AF evaluation value peak yet, the process proceeds to S208.

  In S207, the camera microcomputer 114 stores the focus lens position in the storage unit 115 as the peak lens position when the video having the peak AF evaluation value is accumulated.

  In S208, the camera microcomputer 114 determines whether or not the focus lens 105 has reached the target focus lens position calculated in S202. If it is determined that the target focus lens position has been reached, the process ends. If it is determined that the target focus lens position has not been reached, the process proceeds to S204 to continue the process.

<One-shot AF processing>
FIG. 3 is a flowchart illustrating the operation of the one-shot AF (autofocus) process (second drive control) of the imaging apparatus 100 according to the present embodiment. The one-shot AF process is executed after the focus manual driving process described with reference to FIG. 2, thereby shortening the one-shot AF process.

  First, when the user inputs a one-shot AF process start instruction to the image processing controller 120 via the operation unit 126, the CPU 121 transmits a one-shot AF start instruction to the communication apparatus 116 of the imaging apparatus 100 via the communication apparatus 124. To do.

  The camera microcomputer 114 of the imaging apparatus 100 determines whether or not a one-shot AF command is received from the image processing controller 120 in S301. If the one-shot AF command is received, the process proceeds to S302. If not received, S301 is repeated until the one-shot AF command is received.

  In S302, the camera microcomputer 114 determines whether the storage unit 115 stores the peak lens position stored in S207 of FIG. If the peak lens position is stored, the information on the peak lens position is read and the process proceeds to S303. If the peak lens position is not stored, the process proceeds to S306.

  In S303, it is determined whether or not the arm unit 142 has been stopped between the time when the peak lens position information read in S302 is stored in the storage unit 115 and the current time.

  In this embodiment, it is assumed that information indicating whether or not the arm unit 142 has been operated is stored as flag information in a temporary storage area (not shown) inside the camera microcomputer 114. As an example of the flag information, for example, the flag information is set to a “continue stop” status at the timing when the peak lens position is stored in S207, and the flag information is set to the “operated” information when the robot arm 130 is driven.

  Alternatively, when the target lens position is reached in S208 without saving the peak lens position, the camera microcomputer 114 sets the flag information to the “continue stop” status at the timing when it is determined that the focus lens has reached the target lens position. To do. Therefore, by the above operation, if the flag information is “continuation of stop” in S303, the process proceeds to S304, and if the flag information is “operated”, the process proceeds to S305.

  In S304, the camera microcomputer 114 sets the start direction of the one-shot AF to a direction with the peak lens position based on the positional relationship between the current focus lens position and the peak lens position read from the storage unit 115, and starts the one-shot AF. (Transition to one-shot AF).

  The above will be described with reference to FIG. The vertical axis in FIG. 4 indicates the AF evaluation value, and the larger the value, the higher the AF evaluation value. The horizontal axis indicates the focus lens position. In this embodiment, the lens position in the closest direction is shown as it goes to the right, and the lens position in the infinity direction is shown as it goes to the left.

  Furthermore, “S” on the horizontal axis indicates the focus lens position when the focus lens starts to move to the specified position (Start) in S203. “E” indicates a focus position when the focus manual process is ended (End) when the focus lens reaches the target focus lens position in S208. “P” indicates the focus lens position when it is determined that the in-focus position (Peak) from the AF evaluation value history information during the focus manual driving process. In FIG. 4A, it can be seen from the curve 401 that the in-focus position (P) is found between the start (S) and the end (E) of the focus manual control.

  Therefore, in S304, since the start instruction of the one-shot AF is received when the focus lens position is “E”, the start direction of the one-shot AF is set to infinity and started in S304.

  On the other hand, in S305, since the arm is moving from when the peak lens position is stored until the one-shot AF instruction is received, the pattern in which the subject is present is not known. Therefore, the one-shot AF is started from a predetermined direction. The predetermined direction may be direction information stored in the secondary storage device 123 of the image processing controller 120 or may be a direction designated by the user.

  Next, S306 to S313 show the one-shot AF operation when it is determined in S302 that there is no peak lens position during the focus manual driving process.

  In S306, the camera microcomputer 114 determines whether or not the AF evaluation value has increased from the AF evaluation value history information stored in S204. If it is determined that the AF evaluation value has increased, the process proceeds to S307. If it is determined that the AF evaluation value has not increased, the process proceeds to S310.

Here, an example of a method for determining whether or not the AF evaluation value is increased will be described with reference to FIG. FIG. 4B shows that the focus position (P) has not been found between the start of focus manual control (position S) and the end (position E).
A curve 402 is a graph showing a change in AF evaluation value from the lens position (S) at the start of focus manual control to the lens position (E) at the end of focus manual control. From this graph, it can be seen that the AF evaluation value monotonously increases from S to E. Reference numeral 403 represents a change amount of the AF evaluation value from S to E.

  As the above-described determination of “increasing”, there is a method of determining that the amount of change 403 is increasing when the amount of change 403 is equal to or greater than a predetermined amount. As another example, it is determined that the AF evaluation value for a predetermined period including E is monotonously increasing or that the difference between the minimum value and the maximum value is equal to or greater than a predetermined value. There is a method. Furthermore, as another example, it may be determined that the increase rate of the AF evaluation value during a predetermined period including E is equal to or greater than a predetermined value as “increase”.

  Next, in S307, the camera microcomputer 114 determines whether or not the arm has stopped between the time when the focus lens 105 arrives at the target focus position and the time when the one-shot AF command is received. That is, it is determined whether the flag information described above is in a “continuation of stop” status or an “operated” status.

  If the arm has stopped, that is, if the flag information has a “continuation of stop” status, the process proceeds to S308. If the arm has operated, that is, if the flag information has a “operated” status, the process proceeds to S309.

  In S308, the camera microcomputer 114 reads the drive direction information stored in the storage unit 115 in S202, sets the same direction as the read drive direction as the start direction of the one-shot AF, and starts the one-shot AF. FIG. 4B shows a state in which the focus lens 105 is driven from position S to position E by focus manual drive.

  As described above, when it is determined whether or not the AF evaluation value is increased, it is expected that there is a possibility of focusing at a lens position closer to E as represented by the dotted line in FIG. Is done. Therefore, when a one-shot AF start instruction is received at the position E, the one-shot AF processing time can be shortened by setting the closest direction to the one-shot AF start direction and starting the one-shot AF. It becomes.

  On the other hand, in S309, since the arm is moving from the time when the flag information is stored until the one-shot AF instruction is received, the pattern in which the subject is present is not known. Therefore, the one-shot AF is started from a predetermined direction. The predetermined direction may be a direction stored in the secondary storage device 123 of the image processing controller 120 or a direction designated by the user.

  Next, in S310, the camera microcomputer 114 determines whether or not the AF evaluation value has decreased from the AF evaluation value history information stored in S204. If it is determined that the AF evaluation value is decreasing, the process proceeds to S311. If it is determined that the AF evaluation value is not decreasing, the process proceeds to S313.

  Here, an example of a method for determining whether or not the AF evaluation value is decreasing will be described with reference to FIG.

  FIG. 4C shows that the focal point (P) has not been found between the start of focus manual control (position S) and the end (position E). A curve 404 is a graph showing a change in AF evaluation value from the lens position (S) at the start of focus manual control to the lens position (E) at the end of focus manual control. From this graph, it can be seen that the AF evaluation value monotonously decreases from S to E. Reference numeral 405 represents the amount of change in the AF evaluation value from S to E.

  As the determination of “decreasing”, there is a method of determining that the amount of change 405 is decreasing when the amount of change 405 is equal to or greater than a predetermined amount. As another example, it is determined that the AF evaluation value for a predetermined period including S is monotonously decreasing or that the difference between the minimum value and the maximum value is equal to or greater than a predetermined value. There is a method. Furthermore, as another example, it may be determined that the rate of decrease in the AF evaluation value during a predetermined period including S is equal to or greater than a predetermined value as “decreasing”.

  In S311, the camera microcomputer 114 determines whether or not the arm has stopped between the time when the focus lens 105 arrives at the target focus position and the time when the one-shot AF command is received. That is, it is determined whether the flag information described above is in a “continuation of stop” status or an “operated” status. If the arm is stopped, that is, if the flag information is in the “continuation of stop” status, the process proceeds to S312. If the arm is operated, that is, if the flag information is in the “operated” status, the process proceeds to S313.

  In S312, the camera microcomputer 114 reads the drive direction information stored in the storage unit 115 in S202, sets the direction opposite to the read drive direction as the start direction of the one-shot AF, and starts the one-shot AF. FIG. 4C shows a state in which the focus lens 105 is driven from position S to position E by focus manual drive.

  As described above, when it is determined whether or not the AF evaluation value is decreasing, there is a possibility that the lens is focused at a lens position closer to infinity than S as represented by the dotted line in FIG. Is expected. Accordingly, when a one-shot AF start instruction is received at the position E, the one-shot AF processing time can be reduced by setting the infinity direction as the one-shot AF start direction and starting the one-shot AF. It becomes possible.

On the other hand, in S313, the one-shot AF is started from a predetermined direction. This process is a process executed when described below.
(1) When all of the following conditions are satisfied: The focus position cannot be specified during focus manual drive, and the peak lens position is not stored.
In S306, it has not been determined that the AF evaluation value history stored during focus manual drive has increased.
In S310, it was not determined that the AF evaluation value history saved during focus manual drive was decreasing.
Or
(2) When the arm is moving from when the flag information is saved until the one-shot AF instruction is received, and the direction in which the subject is present cannot be known.
(3) Or when the AF evaluation value history is not saved because the focus manual drive is not executed.

  When information indicating that the arm has moved is received, the stored AF evaluation value history may be cleared. The predetermined direction may be a direction stored in the secondary storage device 123 of the image processing controller 120 or a direction designated by the user.

  As described above, in the present embodiment, the one-shot AF start direction when the one-shot AF start instruction is received is changed based on the AF evaluation value history information during the focus manual control. As a result, the time required for one-shot AF can be shortened, and smooth robot control can be realized.

  Although the present invention has been described in detail based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined.

(Other embodiments)
In addition, the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read the program. It can also be realized by executing processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100: Imaging device, 102: Variable magnification lens, 105: Focus lens, 106: Imaging device, 108: Camera signal processing part, 109: Zoom drive source, 111: Focusing drive source, 113: AF signal processing part, 114: Camera Microcomputer 116: Communication device 120: Image processing controller 130: Robot arm

Claims (20)

An imaging system comprising: an imaging device attached to a robot arm; and a control device that controls the robot arm and the imaging device,
The controller is
Notifying means for notifying subject distance information for the imaging device to the imaging device;
The imaging device
Imaging means for capturing a subject image;
Driving means for driving a focus lens for focusing the imaging optical system;
Based on subject distance information received from the control device, a focus evaluation value change history is acquired from an image obtained by the imaging means while driving the focus lens to a position corresponding to the subject distance information. 1 drive control, and a second drive control for determining the direction of driving the focus lens based on the history of changes in the focus evaluation value obtained by the first drive control and performing automatic focus adjustment. Control means for performing,
An imaging system characterized by that.   The imaging system according to claim 1, wherein the first drive control and the second drive control are performed according to a user instruction.   The imaging system according to claim 2, wherein the control unit performs the second drive control after the first drive control based on an instruction from the user.   When the control means receives the second drive control instruction, and there is no history of change in the focus evaluation value, the drive direction of the focus lens in the second drive control is a predetermined direction. The imaging system according to any one of claims 1 to 3, wherein:   When the control means receives the second drive control instruction, and there is an in-focus position in the history of changes in the focus evaluation value, the second drive based on the direction of the in-focus position. The imaging system according to claim 1, wherein a driving direction of the focus lens in control is determined.   The control means receives the second drive control instruction, and if there is no in-focus position in the focus evaluation value change history, the focus evaluation value is calculated in the focus evaluation value change history. When increasing according to the driving of the focus lens by the first driving control, the driving direction of the focus lens by the second driving control is the same as the driving direction by the first driving control. The imaging system according to any one of claims 1 to 3, wherein:   The control means receives the second drive control instruction, and if there is no in-focus position in the focus evaluation value change history, the focus evaluation value is calculated in the focus evaluation value change history. When it decreases according to the driving of the focus lens by the first driving control, the driving direction of the focus lens by the second driving control is opposite to the driving direction by the first driving control. The imaging system according to any one of claims 1 to 3, wherein:   When the control means receives information indicating that the robot arm is not operating between the time when the focus lens is driven by the first drive control and the time when the second drive control instruction is received. The imaging system according to claim 5, wherein a direction determined based on a change history of the focus evaluation value is a direction in which the focus lens is driven.   When the control means receives information indicating that the robot arm is operating after the focus lens is driven by the first drive control and before the second drive control instruction is received. The imaging system according to claim 5, wherein a driving direction of the focus lens in the second driving control is set to a predetermined direction. An imaging apparatus having an imaging means for imaging an object that is an operation target by a robot,
Obtaining means for obtaining information on the distance from the imaging device to the object based on information for controlling the operation of the robot;
Control means for controlling movement of a focus lens to focus on the object,
The control means includes first drive control for acquiring a history of changes in focus evaluation values from an image obtained by the imaging means while moving the focus lens based on information on a distance to the object. And a second drive control for determining a direction for driving the focus lens based on a history of changes in the focus evaluation value obtained by the first drive control and performing an automatic focus adjustment. An imaging device.   In the second drive control, the control means sets a driving direction of the focus lens in the second drive control to a predetermined direction when there is no history of changes in the focus evaluation value. The imaging apparatus according to claim 10, wherein the imaging apparatus is characterized.   In the second drive control, when there is an in-focus position in the history of changes in the focus evaluation value, the control means performs the focus in the second drive control based on the direction of the in-focus position. The imaging device according to claim 10, wherein a driving direction of the lens is determined.   In the second drive control, when there is no in-focus position in the change history of the focus evaluation value, the control means determines that the focus evaluation value is the first evaluation value in the change history of the focus evaluation value. The driving direction of the focus lens by the second drive control is set to be the same as the direction of the drive by the first drive control when increasing according to the drive of the focus lens by the drive control. The imaging apparatus according to claim 10.   In the second drive control, when there is no in-focus position in the change history of the focus evaluation value, the control means determines that the focus evaluation value is the first evaluation value in the change history of the focus evaluation value. When it is decreased in accordance with the driving of the focus lens by the drive control, the driving direction of the focus lens by the second driving control is set to a direction opposite to the driving direction by the first driving control. The imaging apparatus according to claim 10, wherein the imaging apparatus is characterized.   When the control means receives information indicating that the robot is not operating during the period from when the focus lens is driven by the first drive control to when the focus lens is shifted to the second drive control, 15. The imaging apparatus according to claim 12, wherein a direction determined based on a change history of the focus evaluation value is a direction for driving the focus lens.   When the control means receives information indicating that the robot is operating between the time when the focus lens is driven by the first drive control and the time when the instruction for the second drive control is received. The imaging apparatus according to claim 12, wherein a driving direction of the focus lens in the second driving control is set to a predetermined direction. A method for controlling an imaging system comprising: an imaging device attached to a robot arm; and a control device for controlling the robot arm and the imaging device,
A notification step in which the control device notifies the imaging device of object distance information with respect to the imaging device;
Based on the subject distance information received from the control device, the control unit of the imaging device changes the focus evaluation value from the image obtained by the imaging unit while driving the focus lens to a position corresponding to the subject distance information. The first drive control for acquiring the history of the first and the focus evaluation value change history obtained by the first drive control are used to determine the direction in which the focus lens is driven and to perform automatic focus adjustment. A control process for performing drive control of 2;
A control method for an imaging system, comprising: A method for controlling an imaging apparatus having an imaging means for imaging an object that is an operation target by a robot,
An acquisition step of acquiring information related to a distance from the imaging device to the object based on information for controlling the operation of the robot;
Controlling the movement of a focus lens to focus on the object,
In the control step, a first drive control that acquires a history of changes in focus evaluation values from an image obtained by the imaging unit while moving the focus lens based on information about a distance to the object; And a second drive control for determining a direction for driving the focus lens based on a history of changes in the focus evaluation value obtained by the first drive control and performing an automatic focus adjustment. Control method for imaging apparatus.   The program for making a computer perform each process of the control method of Claim 17 or 18.   A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 17 or 18.

Images