JP2012129709A - Information processor, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly and stably capture a target object without adding a special configuration like a motion prediction part and without depending on the motion of the target object.SOLUTION: A low-resolution ROI (region of interest) determination part 1222 determines a first region of interest being the region of interest of low-resolution image data to be imaged at a succeeding time, based on image data imaged at a preceding time. An image input part 121 acquires low-resolution image data applicable to the first region of interest between the low-resolution image data from a camera 11. A high-resolution ROI determination part 1223 determines a second region of interest being the region of interest of high-resolution image data to be imaged at a succeeding time, based on the low-resolution image data applicable to the first region of interest.

Description

  The present invention relates to a technique for determining a region of interest in image data so as to capture a target object to be tracked.

  In recent years, robot technology has made rapid progress, and not only industrial robots but also pet robots are actively developed as toys for general use. Industrial assembly robots are required to produce a variety of products in small quantities, and the products to be assembled are becoming more complex. In order to cope with such a situation, it is necessary to reduce the number of special dedicated tools and dedicated sensors according to individual products as much as possible. Therefore, attention has been focused on combining cameras and articulated robots, recognizing target objects by image processing, and performing assembly work. Also, in general-purpose robots such as pet robots, usable sensors are limited, and there are some that use a camera as a sensor that can acquire a lot of information, and extract and acquire various information by image processing. .

  One of the most basic functions of a visual sensor that combines such a camera and image processing is tracking (tracking) a target object. When paying attention to the technique of tracking a target object using a camera, there are several methods. One is to keep a camera fixed and continue to capture a certain range, and to track a target object within the imaging range. This is often used in the field of monitoring, and is used for monitoring suspicious persons and abnormal objects. It is also used for traffic volume measurement of people and cars. Some recent digital cameras combine facial image recognition and target object tracking to always focus on a person's face. Another method is to move the camera or change the direction while capturing the target object according to the movement of the target object or according to the movement of the robot. It is used for 3D shape measurement of target objects, position and orientation recognition, and the like. In addition, a method in which both are combined is conceivable, but in any method, the range of the target object is specified from the captured image, and various processes are performed by the subsequent image processing apparatus. The tracking processing of the target object is to determine the imaging range after the next frame and / or to determine the image range to be sent to the image processing apparatus.

  Here, a conventional object tracking imaging system extracts an object region to be tracked from a camera and image data captured by the camera, and determines a region of interest (ROI) from the image data of the object region. It is comprised by. The object tracking processing device generates image data corresponding to the determined attention area, and passes the generated image data to the image processing device. In the image processing apparatus, a process of reading a part number stamped on a target object to be tracked from image data corresponding to a region of interest, or a process of determining an insertion position of a manipulator when the robot grips the target object I do.

  In this way, image data corresponding to a region of interest processed in the image processing apparatus can be obtained. However, there is generally an upper limit (bandwidth) of the amount of data that flows in the communication path connecting the camera and the object tracking processing apparatus. . For example, in the case of transferring 60 screens per second with an RGB color image (8 bits for each color) of VGA (640 × 480 [pixel]) size, 640 × 480 × 8 × 3 × 60 = 4422368000 [bit / sec] ≈ A width of about 0.4 [Gbit / sec] is sufficient. Assuming that the ROI size is ¼ the horizontal and vertical size of the imaging size, it is 160 × 120 [pixels]. However, depending on the application, it may be desired to perform image processing using a higher definition image. For example, if a VGA-size ROI is required, the size of the captured image is 2560 × 1920 [pixels]. In this case, traffic of 7 [Gbit / sec] or more is generated on the communication path. Increasing the frame rate (number of screens transferred per second) will require wider bandwidth. In other words, the image size (ROI size) and the frame rate of the image data processed by the image processing apparatus are suppressed depending on the bandwidth of the communication path. However, if only the image data required by the image processing apparatus can be sent from the camera to the communication path, the bandwidth of the communication path can be used effectively.

  Patent Document 1 discloses a technique for acquiring ROI image data by controlling the position and orientation of the camera itself. In this case, the imaging range of the camera matches the ROI size.

  Here, paying attention to the determination of ROI, it is determined based on the image data of the previous screen. For this reason, it is difficult to accurately capture the target object when the target object to be tracked is moving faster than the frame size and frame rate. Therefore, it is necessary to predict the position of the target object on the next screen from the past movement of the target object.

JP-A-2005-57743

  However, the above-described method of predicting the position of the target object on the next screen from the past movement of the target object has a problem that a motion prediction unit is required even if the frame rate can be increased. In addition, when the target object moves differently from the movement assumed by the motion prediction unit, there is a problem that it is difficult to accurately and stably capture the target object.

  Therefore, an object of the present invention is to accurately and stably capture a target object regardless of the movement of the target object without adding a special configuration such as a motion prediction unit.

  The information processing apparatus according to the present invention is configured to determine a first attention area, which is an attention area of low-resolution image data captured at the next time point, based on image data captured at the previous time point. And, based on the low-resolution image data corresponding to the first region of interest, the acquisition means for acquiring the low-resolution image data corresponding to the first region of interest among the low-resolution image data, And second determination means for determining a second region of interest that is a region of interest of the high-resolution image data to be imaged at the next time point.

  According to the present invention, it is possible to accurately and stably capture a target object regardless of the movement of the target object without adding a special configuration such as a motion prediction unit.

It is a figure which shows the structure of the object tracking imaging system which concerns on embodiment of this invention. It is a figure which shows the example of the pixel which comprises the low resolution ROI image data on a CMOS image sensor, and the pixel which comprises high resolution ROI image data. It is a figure which shows the relationship between the low resolution ROI of the time t, and the high resolution ROI of the time t, when the target object is moving. It is a figure for demonstrating the determination method of the low resolution ROI of the time t in embodiment of this invention. It is a flowchart which shows the procedure which calculates | requires an image center. It is a figure which shows the image data which imaged the target object in the time t-1 and the time t. It is a figure which shows the example of the object tracking imaging system for comparing with the object tracking imaging system which concerns on embodiment of this invention. It is a figure for demonstrating the ROI determination method in the comparative example of FIG.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments to which the invention is applied will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram showing a configuration of an object tracking imaging system according to an embodiment of the present invention. As shown in FIG. 1, in the object tracking imaging system according to the present embodiment, a camera 11 and an object tracking processing device 12 are connected via a communication unit 13. The camera 11 includes an imaging unit 111, an image output unit 112, an imaging control unit 113, and a resolution / ROI control unit 114. The object tracking processing device is a configuration that is an application example of the information processing device of the present invention.

  The imaging unit 111 includes a lens, a CMOS image sensor, and the like, and generates image data by imaging processing. The image output unit 112 transmits the image data generated by the imaging unit 111 to the object tracking processing device 12 via the communication path 13. The imaging control unit 113 controls, for example, exposure timing and timing for erasing image data held in the CMOS image sensor. For example, the imaging control unit 113 performs exposure at regular time intervals, or erases the image data held in the CMOS image sensor after the image output unit 112 outputs the image data to the communication path 13. The resolution / ROI control unit 114 obtains two pieces of information of resolution information and ROI information from the object tracking processing device 12 and outputs them to the image output unit 112. The resolution information here is information for designating low resolution image data or information for designating high resolution image data. In the present embodiment, it is assumed that the image data in the same imaging range is 1280 × 960 pixels for high resolution image data and 640 × 480 pixels for low resolution image data. The ROI information is information that designates a region of interest (ROI) in captured image data. The ROI is, for example, a minimum rectangular area including the object area, or a rectangular area having a predetermined size with the center of the object area as the image center.

  The object tracking processing device 12 includes an image input unit 121, an ROI image generation unit 123, and an ROI calculation unit 122. The ROI calculation unit 122 includes a target extraction unit 1221, a low resolution ROI determination unit 1222, and a high resolution ROI determination unit 1223. The object tracking processing device 12 obtains image data from the communication path 13 by the image input unit 121. The image input unit 121 holds the image data obtained from the communication path 13 in a state where it can be accessed from other components in the object tracking processing device 12. The low-resolution ROI determination unit 1222 is a configuration that is an application example of the first determination unit, the image input unit 121 is a configuration that is an application example of the acquisition unit, and the high-resolution ROI determination unit 1223 is a first example. This is a configuration that is an application example of the second determination means.

  In the following, for convenience of explanation, it is assumed that the imaging times are..., T-2, t-1, t, t + 1,. The target extraction unit 1221 obtains low resolution ROI image data or high resolution ROI image data from the image input unit 121, and extracts an object region to be tracked. The low resolution ROI determination unit 1222 determines an ROI (hereinafter referred to as a low resolution ROI) from an object region extracted from the low resolution ROI image data. The high resolution ROI determination unit 1223 determines an ROI (hereinafter referred to as a high resolution ROI) from an object region extracted from the high resolution ROI image data. Note that the image input unit 121 has already acquired the low-resolution ROI image data and the high-resolution ROI image data from the camera 11 through the communication path 13 until time t−1.

  Now, a method for obtaining ROI image data (low resolution ROI image data, high resolution ROI image data) at time t will be described. The target extraction unit 1221 extracts an object region from the low-resolution ROI image data at time t-1. As an object region extraction method, for example, a method for extracting a ping-pong ball-like object region when a white ping-pong ball-like object is moving around in a black background is as follows. First, the target extraction unit 1221 obtains luminance image data from the low resolution ROI image data. A method of calculating the luminance value Y from the RGB color image data is well known, and a method of generating luminance image data from the luminance value calculated by Y = 0.299R + 0.587G + 0.114B for each pixel is often used. Assuming that the pixel value of the luminance image data is normalized by [0, 1], binarization is performed with a pixel having a luminance value of a certain threshold value (for example, 0.5) or less and other pixels. When binary image data is generated with the former being a black pixel and the latter being a white pixel, the ping-pong ball-like object region is represented by a white pixel. Therefore, the object region can be extracted with the smallest circular region including all white pixels.

  The target extraction unit 1221 sends information such as the position and size of the extracted object region to the low resolution ROI determination unit 1222. When the object extraction unit 1221 extracts the object region by the above-described method, for example, the center of the object region is set to “position” and the diameter of the circular region is set to “size”.

  The low resolution ROI determination unit 1222 determines the low resolution ROI (first attention area) at time t based on the “position” and “size” of the object area in the low resolution ROI image data at time t−1. To do. When the frame rate is sufficiently higher than the motion of the target object (the imaging interval is sufficiently short), the “position” and “size” of the object region are the ROI image data at time t−1 and the ROI image data at time t. It is almost the same. Therefore, the low resolution ROI determination unit 1222 determines, for example, the minimum rectangular area including the object area as the low resolution ROI at time t. When “position” and “size” are used, for example, a square region whose side is “size” centered on “position” may be set as the low resolution ROI. Then, the low resolution ROI determination unit 1222 uses two pieces of information, ROI information indicating the low resolution ROI at time t and resolution information (in this case, information for designating low resolution image data) as the resolution / ROI of the camera 11. Notify the control unit 114.

  The resolution / ROI control unit 114 transmits the ROI information and resolution information obtained from the object tracking processing device 12 to the image output unit 112. The image output unit 122 acquires low-resolution image data at time t captured by the imaging unit 111 according to the resolution information. Then, the image output unit 122 extracts the range of the low resolution ROI specified by the ROI information from the low resolution image data at the time t, and sends it to the object tracking processing unit 12 via the communication path 13 as the low resolution ROI image data at the time t. Send it out.

  The image input unit 121 obtains the low-resolution ROI image data at this time t and sends it to the target extraction unit 1221. The target extraction unit 1221 extracts an object region from the low-resolution ROI image data at time t. Then, the target extraction unit 1221 sends information such as “position” and “size” of the extracted object region to the high-resolution ROI determination unit 1223. Generally, a high resolution ROI is included in a region of high resolution image data corresponding to the low resolution ROI. Therefore, the high resolution ROI determination unit 1223 determines the high resolution ROI (second attention area) at time t based on information such as “position” and “size” of the object area in the low resolution ROI image data at time t. decide. Here, for example, the smallest rectangular area including the object area is determined as the high resolution ROI.

  As described above, since the determination of the low-resolution ROI at time t is based on the low-resolution ROI image data at time t−1, the center of the object region is not necessarily the time t when the target object is moving. The low-resolution ROI image data of −1 is not necessarily at the center. However, since the determination of the high resolution ROI at time t is based on the low resolution ROI image data at time t, which is the same time, the center of the target object may coincide with the center of the high resolution ROI image data. it can. Thus, by determining the high resolution ROI in two steps, the high resolution ROI can be determined stably regardless of the motion of the target object.

  Then, the high-resolution ROI determination unit 1223 uses two pieces of information, ROI information indicating the determined high-resolution ROI and resolution information (here, information for designating high-resolution image data) as resolution / ROI control of the camera 11. Notification to the unit 114. The resolution / ROI control unit 114 transmits the ROI information and resolution information obtained from the object tracking processing device 12 to the image output unit 112. The image output unit 112 acquires high-resolution image data at time t captured by the imaging unit 111 according to the resolution information. In the following description, both low resolution image data and high resolution image data are image data having different resolutions captured in the same imaging range. The image output unit 112 extracts the range of the high resolution ROI specified by the ROI information from the high resolution image data at the time t, and sends it to the object tracking processing device 12 via the communication path 13 as the high resolution ROI image data at the time t. To do.

  The image input unit 121 obtains high-resolution ROI image data at time t and sends it to the ROI image generation unit 123. The ROI image generation unit 123 sends the high-resolution ROI image data at time t to the image processing device 14 arranged outside the object tracking processing device 12. The image processing device 14 executes processing related to the target object using the high-resolution ROI image data at time t. The process related to the target object here is, for example, a process of reading a part number stamped on the target object or a process of determining a manipulator insertion position when the robot holds the target object.

  After sending the high-resolution ROI image data at time t, the image output unit 112 instructs the imaging control unit 113 to erase the image data at time t, and performs imaging at time t + 1 to the imaging control unit 113. Instruct. The imaging control unit 113 controls the imaging unit 111 in accordance with instructions from the image output unit 112.

  By repeating the above operation and obtaining ROI image data in the order of low resolution ROI image data at time t + 1, high resolution ROI image data at time t + 1,... ROI image data is sent continuously.

  In the case of a general global shutter type (a method in which all pixels are exposed simultaneously) CMOS image sensor, the digitized image data is on the CMOS image sensor and can be read out nondestructively. Therefore, the high-resolution ROI image data can be realized by reading out all the pixels in the area designated as the high-resolution ROI on the CMOS image sensor by the imaging unit 111. Further, the low resolution ROI image data can be realized by reading out the pixels in the area designated as the low resolution ROI on the CMOS image sensor by thinning out pixels one by one, for example.

  By the way, in this embodiment, the low resolution ROI and the high resolution ROI are substantially the same region. Therefore, most of the pixels included in the low resolution ROI image data are included in the high resolution ROI image data. Therefore, when transferring the high-resolution ROI image data at time t, the image output unit 112 transfers only pixel data not included in the low-resolution ROI image data at time t. As a result, the amount of image data flowing through the communication path 13 can be made substantially equal to the amount of high-resolution ROI image data originally required. This means that there is almost no pressure on the communication path 13 by thinning out pixels one by one and reading out the low resolution ROI image data and transferring it to the object tracking processing unit 12.

  FIG. 2 is a diagram showing an example of pixels constituting low resolution ROI image data and pixels constituting high resolution ROI image data on a CMOS image sensor. In the example of FIG. 2, the pixels constituting the low resolution ROI image data are ◯ and ●, and the pixels constituting the high resolution ROI image data are ■ and ●. First, the image output unit 112 reads out low-resolution ROI image data from the pixels ◯ and ● and transfers it to the object tracking processing device 12. Thereafter, the image output unit 112 reads the high-resolution image ROI data from the pixels {circle around (1)} and ●, and transfers it to the object tracking processing device 12. However, since ● has already been transferred, the image output unit 112 transfers only ■. In the case of FIG. 2, the overlap between the low resolution ROI image data and the high resolution ROI image data is about 65%, but the increase in the amount of data (number of pixels) due to the transfer of the low resolution ROI image data is 9% or less. .

  In this embodiment, in order to make the explanation easy to understand, after obtaining the low-resolution ROI image data at time t, the high-resolution ROI image data at time t and then the low-resolution ROI image data at time t + 1 are obtained. Although described, the process may not be sequential. After obtaining low-resolution ROI image data at time t and obtaining information such as “position” and “size” of the object region in the target extraction unit 1221, the low-resolution ROI determination unit 1222 and the high-resolution ROI determination unit 1223 You may proceed at the same time. However, the transmission of the low resolution ROI image data at time t by the image output unit 112 must be performed prior to the transmission of the high resolution ROI image data at time t. Therefore, it is necessary to make the order correct when sending the ROI information and the resolution information, or after receiving the instruction information by the resolution / ROI control unit 114.

In this embodiment, the minimum rectangular area including the object area is set as the ROI, but other methods may be used. Some examples of techniques are given below.
As a first method, a rectangular region having a predetermined size with the center of the object region as the image center is determined as the ROI. As a second method, the center of the object region is set as the image center, and the minimum rectangular region including the object region is determined as the ROI. As a third method, the center of the smallest rectangular area including the object area is set as the image center, and a rectangular area having a size obtained by multiplying the size of the smallest rectangular area including the object area by a fixed magnification is determined as the ROI. is there. As a fourth technique, the center of the rectangular area of the second or third technique is used as the image center, and the rectangular area having the minimum predetermined aspect ratio including the rectangular area is determined as the ROI.

  Next, a second embodiment of the present invention will be described. As shown in the first embodiment, the low resolution ROI determination unit 1222 is based on the low resolution ROI image data at time t−1 when determining the low resolution ROI at time t. Therefore, when the target object is moving, the low resolution ROI at time t does not necessarily include the high resolution ROI at time t. This means that when the high resolution ROI determination unit 1223 determines the high resolution ROI at time t, a part of the unknown area must be included.

  FIG. 3 is a diagram illustrating the relationship between the low resolution ROI at time t and the high resolution ROI at time t when the target object is moving. As shown in FIG. 3, when the high resolution ROI at the time t is determined based on the low resolution ROI image data at the time t when the target object is moving, a part of the high resolution ROI at the time t is There is an area that does not belong to the low-resolution ROI image data at time t. Therefore, when determining the high resolution ROI at time t, it is necessary to estimate the object region for a part of the high resolution ROI at time t.

  Therefore, if all the regions that may be included in the high-resolution ROI image data at time t are included in the region of the low-resolution ROI image data at time t, the determination of the high-resolution ROI at time t is simply clipping. It comes down to the problem. Specifically, the low resolution ROI may be determined to be larger by the maximum distance at which the target object can move on the screen during the shooting interval.

  FIG. 4 is a diagram for explaining a method of determining the low resolution ROI at time t in the present embodiment. As shown in FIG. 4, the low resolution ROI at time t is determined to be larger by the maximum distance (margin) than the high resolution ROI at time t. As a result, the high resolution ROI at time t is always included in the low resolution ROI image data at time t, and the determination of the high resolution ROI can be performed accurately and stably. The maximum distance calculation method may be determined dynamically, but may be determined in advance as system operation requirements.

  For example, when the target object moves around on a surface 1 m away in an imaging system with a left and right angle of view of 90 °, and the number of pixels in the horizontal direction is 1024 pixels, it has a resolution of about 1.5 mm per pixel. Will be. When imaging is performed at 1000 frames per second, when 20 pixels margin (maximum distance) is expected between consecutive frames, 20 [pixel] × 1.5 [mm / pixel] × 1000 [frame / second] = 3000 [ mm / sec], and it is possible to track the target object up to 3 m / s.

  Next, a third embodiment of the present invention will be described. In the above embodiment, the high resolution ROI determination unit 4222 determines the high resolution ROI at time t from the low resolution ROI image data at time t. Although the rough “position” and “size” of the object region can be understood from the low-resolution ROI image data, the detailed “position” and “size” cannot be easily calculated because the resolution is insufficient. For example, in the case of an object having a thin pointed protrusion such as a heel foot, it may be difficult to clearly determine the position and orientation of the protrusion from the low-resolution ROI image data. In such a case, high resolution ROI image data may be used in combination. Low-resolution ROI image data is used for the rough calculation of “position” and “size”, and high-resolution ROI image data is used for the detailed calculation of “position” and “size”. Specifically, the target extraction unit 1221 calculates a rough “position” and “size” of the object region from the low-resolution ROI image data at time t, and further uses the high-resolution ROI image data at time t−1. The detailed “position” and “size” are calculated.

  FIG. 5 shows an example of a procedure for obtaining the center of the image, assuming that the size of the high resolution ROI is constant. The target object (物体) is shown in FIG. The left side of FIG. 6 is a captured image at time t−1, and the right side is a captured image at time t. As shown in FIG. 6, the target object is moving from top to bottom, and there are projections that are difficult to detect in the low-resolution image data, and these projections also move.

  In the case of the present embodiment, both low-resolution ROI image data and high-resolution ROI image data are passed to the high-resolution ROI determination unit 1223. When the low-resolution ROI image data at time t is input, the high-resolution ROI determination unit 1223 calculates the center of the target object indicated by a cross in the circle 601 in FIG. Step S501). Then, the high-resolution ROI determination unit 1223 corrects the center of the target object from the obtained center of the target object and the offset information for each feature specific to the target object stored separately (step S502). Here, the feature unique to the target object is a feature that is difficult to extract from the low-resolution ROI image data but can be extracted from the high-resolution ROI image data. ◯ in 602). The offset information indicates the relative positional relationship between the center position obtained from the low-resolution ROI image data and the feature specific to the target object, and information at time t−1 is stored. Since continuous imaging is performed at a high frame rate, the relative positional relationship at time t-1 and the relative positional relationship at time t can be regarded as substantially equal. By using the offset information obtained in this way to estimate the characteristics specific to the target object, the size and center of the target object are estimated (corrected).

  On the right side of FIG. 6, a region corresponding to the high resolution ROI estimated by the above procedure is indicated by a one-dot chain line. The high resolution ROI determination unit 1223 corrects the center position of the target object, and then updates the center position of the target object obtained from the low resolution ROI image data before correction as the low resolution center position information (step S503). Then, the high resolution ROI determination unit 1223 uses the corrected center of the target object (high resolution ROI) as the output of the high resolution ROI determination unit 1223 (step S504).

  When high-resolution ROI image data is input, the high-resolution ROI determination unit 1223 updates offset information for each feature having a relative positional relationship. When the high-resolution ROI image data at time t is input, the center position of the target obtained from the low-resolution ROI image data at time t is stored in the low-resolution center position information. Therefore, the high resolution ROI determination unit 1223 estimates the feature position on the high resolution ROI image data from the low resolution center position information and the offset information for each feature (step S505). As described above, since continuous shooting is performed at a high frame rate, an actual feature exists in the vicinity of the feature position estimated here. Therefore, the high resolution ROI determination unit 1223 detects the feature position of the target object in the vicinity of the estimated position using image pattern matching or the like (step S506). The high resolution ROI determination unit 1223 calculates and updates offset information for each feature from the detected feature position and low resolution center position information (step S507).

  Specifically, the estimation is performed as follows. First, the high resolution ROI determination unit 1223 specifies the detailed position of the feature unique to the object. The center position of the abdomen of the target object (蜘蛛) at time t−1 can be estimated from the low-resolution ROI image data at time t. From here, the high-resolution ROI determination unit 1223 uses the offset information to identify a range that is likely to have object-specific features. Then, the high resolution ROI determination unit 1223 detects a characteristic specific to the target object within this range and specifies a detailed position. For the detection of the characteristic peculiar to the target object, for example, several characteristic image patterns of the heel leg are prepared, and detection is performed by pattern matching. The high-resolution ROI determination unit 1223 specifies the detailed position of the object-specific feature and then updates the offset information as that at time t. Next, the high-resolution ROI determination unit 1223 obtains the minimum rectangular area including all the toes (target-specific features) of the heel, and calculates the center and size as the center and size of the target object.

  In addition, although the example which calculates | requires the center of high resolution ROI image data was shown here, you may obtain | require another value. Any value may be used as long as it is a value (image size, orientation, etc.) constituting the high-resolution ROI or a value necessary for obtaining it.

  In this example, the high resolution ROI at time t is obtained by using the low resolution ROI image data at time t-1 and time t and the high resolution ROI image data at time t-1. An image or a high-resolution ROI image may be used.

  FIG. 7 is a diagram illustrating an example of an object tracking imaging system for comparison with the object tracking imaging system according to the embodiment of the present invention. 7, the configuration with the same reference numerals as in FIG. 1 is the same configuration as in FIG. The difference from FIG. 1 is that the camera 711 is provided with the ROI control unit 115, and the object tracking processing device 712 includes the motion prediction unit 1224 and the ROI determination unit 1225 in addition to the target extraction unit 1221 common to FIG. The point is that an ROI calculation unit 722 provided with the above is provided.

  The range captured by the imaging unit 111 is the same as that in the above-described embodiment. However, what is sent from the image output unit 112 to the object tracking processing device 712 via the communication path 13 is the ROI instructed from the ROI control unit 115. It is only a range.

  Since the ROI image data sent to the object tracking processing device 712 is already in a size necessary for the image processing device 14, it is only necessary to send it to the image processing device 14 as it is. The target extraction unit 1221 extracts a target object to be tracked from the ROI image data, and the ROI determination unit 1225 determines the ROI of the next image data based on the extracted target object, and the ROI control unit 115 Notice. Here, the ROI at time t is determined based on the image data at time t-1 (FIG. 8A). Therefore, as shown in FIG. 8B, it is difficult to accurately capture the target object when the target object is moving at a higher speed than the frame size and the frame rate. Therefore, it is necessary to predict the position of the target object at time t from the past movement of the target object. The motion prediction unit 1224 predicts the motion of the target object. The ROI determination unit 1225 determines the ROI at time t based on the prediction result from the motion prediction unit 1224. As a result, the target object can be captured as shown by the solid line frame in FIG.

  However, in the object tracking imaging system shown in FIG. 7, even if the frame rate can be increased, the configuration of the motion prediction unit 1224 is required. In addition, when the target object moves differently from the motion assumed by the motion prediction unit 1224, it is difficult to accurately and stably capture the target object. On the other hand, the embodiment of the present invention can capture the target object accurately and stably without adding a special configuration such as the motion prediction unit 1224, regardless of the motion of the target object. Become.

  The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program. It is a process to be executed.

  11: Camera, 12: Object tracking processing device, 13: Communication path, 111: Imaging unit, 112: Image output unit, 113: Imaging control unit, 114: Resolution / ROI control unit, 121: Image input unit, 122: ROI Calculation unit, 123: ROI image generation unit, 124: Image processing device, 1221: Object extraction unit, 1222: Low resolution ROI determination unit, 1223: High resolution ROI determination unit

Claims (5)

First determination means for determining a first region of interest, which is a region of interest of low-resolution image data captured at the next time point, based on image data captured at the previous time point;
Obtaining means for obtaining the low resolution image data corresponding to the first region of interest among the low resolution image data;
Second determining means for determining a second region of interest, which is a region of interest of the high-resolution image data captured at the next time point, based on the low-resolution image data corresponding to the first region of interest. An information processing apparatus comprising:   The information processing apparatus according to claim 1, wherein the first determination unit determines the first region of interest so as to include a region that may be included in the second region of interest.   The second determining unit determines the second region of interest based on the high resolution image data captured at the previous time point in addition to the low resolution image data corresponding to the first region of interest. The information processing apparatus according to claim 1 or 2. An information processing method executed by an information processing apparatus,
A first determination step of determining a first region of interest, which is a region of interest of low-resolution image data captured at the next time point, based on image data captured at the previous time point;
Obtaining the low resolution image data corresponding to the first region of interest among the low resolution image data; and
A second determination step of determining a second region of interest that is a region of interest of the high-resolution image data captured at the next time point based on the low-resolution image data corresponding to the first region of interest; An information processing method comprising: A first determination step of determining a first region of interest, which is a region of interest of low-resolution image data captured at the next time point, based on image data captured at the previous time point;
Obtaining the low resolution image data corresponding to the first region of interest among the low resolution image data; and
A second determination step of determining a second region of interest that is a region of interest of the high-resolution image data captured at the next time point based on the low-resolution image data corresponding to the first region of interest; A program that causes a computer to execute.

Images