JP2023143123A - Imaging apparatus, imaging system, control method, and program - Google Patents

Abstract

To provide an imaging apparatus, an imaging system, a control method, and a program that are capable of quickly detecting a subject regardless of the size of the subject in a capturing image and suppressing the reduction of a detection rate.SOLUTION: An imaging apparatus 100 includes an imaging unit 102 that has a pixel area 102a in which a plurality of pixels is arranged and reads out a captured image from the pixel area 102a, a DSP 106 that performs recognition processing for recognizing a subject included in the captured image, and a subject size calculation unit 115 for detecting the size of the subject. When reading out the captured image, the imaging unit 102 changes the size of a read-out area to be read out in the pixel area 102a.SELECTED DRAWING: Figure 3

Description

The present invention relates to an imaging device, an imaging system, a control method, and a program.

In recent years, object recognition using machine learning has been performed in various fields such as digital single-lens cameras, video cameras, and smartphones. Patent Document 1 discloses an imaging device that performs recognition processing to recognize a subject based on a learning model in which an arbitrary readout unit is set when reading out an imaging signal and supervised learning is performed for each readout unit. There is. In this imaging device, a subject can be recognized before one frame of image data is collected.

Patent No. 6635222

In the imaging device described in Patent Document 1, in order to improve the readout speed, an intermittent pattern is set in the readout unit or the readout is thinned out. However, such a readout configuration has a problem in that when a relatively small image of a subject is formed on the imaging surface, the detection rate of the subject decreases.

The present invention provides an imaging device, an imaging system, a control method, and a program that can quickly detect an object regardless of the size of the object in a captured image and can suppress a decrease in the detection rate. The purpose is to provide

In order to achieve the above object, an imaging device of the present invention has a pixel area in which a plurality of pixels are arranged, an imaging means for reading out a captured image from the pixel area, and a recognition process for recognizing a subject included in the captured image. and a detection means that detects the size of the subject. is characterized by being changed.

According to the present invention, regardless of the size of the subject in a captured image, it is possible to quickly detect the subject, and it is possible to suppress a decrease in the detection rate.

1 is a block diagram showing a configuration example of an imaging device according to a first embodiment; FIG. This is an example of a captured image captured by the imaging unit. FIG. 2 is a detailed block diagram of the imaging device shown in FIG. 1. FIG. 3 is a flowchart of a subject size calculation process executed by a subject size calculation unit. 3 is a flowchart of a readout area determination process executed by a readout area determination unit. FIG. 2 is a block diagram showing a configuration example of an imaging device according to a second embodiment. FIG. 7 is a block diagram showing a configuration example of an imaging device according to a third embodiment.

Hereinafter, each embodiment of the present invention will be described in detail with reference to the drawings. However, the configurations described in each embodiment below are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment.

<First embodiment>
The first embodiment will be described below with reference to FIGS. 1 to 5. FIG. 1 is a block diagram showing a configuration example of an imaging device according to a first embodiment. As shown in FIG. 1, the imaging device 100 includes an imaging lens 101, an imaging section (imaging means) 102, an A/D converter 103, a signal processing section 104, a memory 105, a DSP (Digital Signal Processor) 106, and an output section 107. , has a CPU 108.

The imaging lens 101 is composed of a lens group including a zoom lens, a focus lens, a shift lens, an aperture, and the like. The imaging unit 102 is an imaging device configured with a CCD, CMOS device, or the like that converts an optical image into an electrical signal. A/D converter 103 converts analog signals into digital signals. This A/D converter 103 is used to convert an analog signal output from the imaging section 102 into a digital signal. The signal processing unit 104 performs resizing processing such as predetermined pixel interpolation and reduction, and other color conversion processing on the data from the A/D converter 103. Output data from the A/D converter 103 is directly written into the memory 105 via the signal processing unit 104.

The DSP 106 performs visual image processing on output data from the signal processing unit 104 or data acquired from the signal processing unit 104 via the memory 105. Further, the DSP 106 stores the image-processed data in the memory 105 or outputs it to the output unit 107. Further, the DSP 106 uses output data from the signal processing unit 104 or data acquired from the signal processing unit 104 via the memory 105 as input data, and performs recognition processing to recognize a subject included in the data (captured image). (recognition process). This recognition process is performed based on a learning model that has been subjected to machine learning. As a result, the object recognition rate (estimation rate) improves each time learning is performed. Then, the DSP 106 stores the data on which the recognition process has been performed as output data in the memory 105 or outputs the data to the output unit 107. In this way, the DSP 106 has a function as an image processing means that performs image processing for visual recognition, and a function as a recognition means that performs recognition processing for recognizing a subject.

The memory 105 stores various data such as image data processed by the DSP 106 and recognition results. Memory 105 has sufficient storage capacity to store various types of data. Furthermore, the memory 105 also serves as a nonvolatile memory. The nonvolatile memory is a memory that can be electrically erased and recorded, and for example, an EEPROM or the like is used. Constants, programs, etc. for the operation of the DSP 106 and the CPU 108 are stored in the nonvolatile memory. This program includes, for example, a program for causing the CPU 108 (computer) to execute operations of each section and each means of the imaging apparatus 100. The output unit 107 outputs the image data and recognition results processed by the DSP 106 to the outside.

The CPU 108 controls the entire imaging device 100. The CPU 108 executes each process described below based on the program recorded in the memory 105 described above. Note that the memory 105 also serves as a system memory. A RAM is used as the system memory, and constants and variables for the operation of the CPU 108, programs read from the nonvolatile memory, and the like are expanded.

As described above, the imaging device 100 includes the imaging section 102. The imaging unit 102 may be of a rolling shutter type that reads out lines sequentially, or may be of a global shutter type that reads out all pixels at once. Furthermore, the imaging unit 102 can read out digital signals by setting a partial area within the pixel area 102a, which is the imaging surface, as a reading unit that is read out in the smallest unit in the pixel area 102a. Further, the DSP 106 can perform machine learning with training data for each reading unit and execute recognition processing. As a result, processing that would normally be performed after one frame of digital signals has been completed can be time-divided and processed in a cycle shorter than the imaging cycle, which can be expected to speed up recognition processing. . For example, in the case of a rolling shutter method, the reading unit may be any continuous line group or a discrete line group. In the case of the global shutter method, the readout unit is a rectangular area of a continuous pixel group, or a pattern in which these are arranged discretely.

Further, the imaging unit 102 can also read out digital signals by thinning out the reading. By performing recognition processing on the digital signals that have been thinned out and read out, it is possible to expect a faster calculation cycle. For example, the imaging unit 102 can read out only 1/4 of the number of pixels in the total pixel area 102a and execute the recognition process. Further, if the desired recognition result is still not obtained even after this 1/4 recognition process, the imaging unit 102 sequentially reads out 2/4, 3/4, and all pixels of the number of pixels in the total pixel area 102a. It is possible to increase the number of pixels. With this, it is possible to eventually reach the desired recognition result.

FIG. 2 is an example of a captured image captured by the imaging unit. As shown in FIG. 2, the imaging unit 102 has a pixel area 102a in which a plurality of pixels are arranged. are arranged vertically and horizontally, 16 each. One pixel is a square, and serves as a readout unit when reading out a captured image from the pixel area 102a (imaging process). In FIG. 2, there are two alphabets "Z" as subjects included in the captured image, and due to perspective, the "Z" closer to the imaging device 100 is larger, and the "Z" farther from the imaging device 100 is smaller. It has become. Further, in FIG. 2, the pixel area 102a includes an outlined portion and a hatched portion. The white portion is the portion to be read (hereinafter referred to as "readout area 102b"). The hatched portion is a portion that is thinned out without being read out, that is, a portion where reading is stopped in the pixel region 102a (hereinafter referred to as “thinned out region 102c”).

As shown in FIG. 2A, when 1/4 of the total pixel area 102a is set as the readout area 102b, a large subject "Z" is recognized as a subject, but a small subject "Z" is located in the thinning area 102c. occupies a large proportion of the image, and tends not to be recognized as a subject. In FIG. 2(a), the one recognized as the subject is surrounded by a thick line. The same applies to subsequent FIGS. 2(b) and 2(c). As shown in FIG. 2B, when 3/4 of the total pixel area 102a is set as the readout area 102b, both the large object "Z" and the small object "Z" are recognized as objects. Therefore, in the imaging device 100, the imaging unit 102 is configured to change the size of the readout area 102b read out in the pixel area 102a according to the subject size (size of the subject) when reading out the captured image. ing. For example, as shown in FIG. 2C, reading is performed by adjusting the number (amount) of the thinning areas 102c for a large subject "Z" and a small subject "Z". Thereby, it is possible to improve the detection rate of the small subject "Z" while appropriately maintaining the readout speed of the captured image (pixel region 102a).

FIG. 3 is a detailed block diagram of the imaging device shown in FIG. 1. Note that in FIG. 3, the imaging lens 101, A/D converter 103, signal processing section 104, and output section 107 in FIG. 1 are omitted. As shown in FIG. 3, the imaging unit 102 includes an imaging sensor 109 and an imaging control unit 110. The image sensor 109 is a two-dimensional pixel array, and reads out a subject image formed by the image pickup lens 101 as an electrical signal. Further, the image sensor 109 has two photoelectric conversion units in each pixel, and can read out an electrical signal for phase difference detection from each photoelectric conversion unit. The imaging control unit 110 controls the readout area, readout timing, etc. of analog signals accumulated in the imaging sensor 109. Further, the imaging control unit 110 controls readout based on the readout unit and readout position determined by the readout area determination unit 114 of the CPU 108.

The DSP 106 includes a visual recognition processing section 111, a recognition processing section 112, and a phase difference calculation section 113. The memory 105 stores learning data 116 and read area information 117. The visibility processing unit 111 performs demosaic processing, white balance processing, etc. on the digital signal read out by the imaging control unit 110 and subjected to predetermined conversion processing via the A/D converter 103 and the signal processing unit 104. Performs visual recognition processing. Furthermore, the visual recognition processing unit 111 outputs the image data for visual recognition obtained through the visual recognition process via the output unit 107 . The recognition processing unit 112 performs recognition processing by performing arithmetic processing based on learning data 116 recorded in the memory 105 and created by machine learning. Further, the recognition processing unit 112 can output the recognition result obtained in the recognition process via the output unit 107. This allows recognition results to be recorded and used externally. The phase difference calculation unit 113 calculates the phase difference on the image plane of the subject images read out from the two photoelectric conversion units of the imaging control unit 110.

Note that the recognition processing unit 112 includes, for example, a GPU (Graphic Processing Unit). A GPU is a processor for neural network calculations, and can process data in parallel to perform efficient calculations. For example, if a learning model trained by deep learning is used and the learning is performed multiple times, the learning process can be performed quickly using the GPU. Further, the recognition processing unit 112 may further include a TPU (Tensor Processing Unit), an NPU (Neural Processing Unit/Neural Network Processing Unit), and the like. Further, the learning data includes a learning model used when the recognition processing unit 112 executes recognition processing. The learning model is, for example, a learning model using a neural network whose parameters are adjusted by an error backpropagation method or the like. Thereby, the recognition processing unit 112 can perform, for example, deep learning in which various parameters such as feature amounts and weights (connection weighting coefficients) for learning are generated by itself. Note that machine learning is not limited to deep learning, and may be machine learning using any machine learning algorithm such as support vector machine, logistics regression, decision tree, nearest neighbor method, naive Bayes method, etc. . Furthermore, the learning model may be a model using something other than a neural network. Such a recognition processing unit 112 performs supervised learning. In this supervised learning, an image that includes the object to be recognized and does not have the thinned out region 102c is assumed to be correct data (supervised data). Further, there are a plurality of images to be used as training data, and each image has a different size, position, etc. of the subject.

The CPU 108 includes a readout area determining section 114 and a subject size calculating section 115. The subject size calculation unit 115 is a detection means that detects the subject size (size of the subject), and calculates numerical data representing the subject size based on the phase difference calculated by the phase difference calculation unit 113 (detection step ). The readout area determination unit 114 is a determining unit that determines the size and position of the readout area 102b in the image sensor 109 according to the subject size calculated by the subject size calculation unit 115. The reading area 102b is recorded in the memory 105 as reading area information 117. The readout area information 117 is information regarding the readout unit and the size (amount) of the thinning area 102c for the pixel area 102a of the image sensor 109.

Note that the imaging device 100 includes a first board on which the imaging unit 102 is configured (mounted), and a second board, which is configured separately from the first board and on which the DSP 106, CPU 108, and memory 105 are configured (mounted). It is preferable to have a substrate. Thereby, for example, the arrangement position of the imaging unit 102 in the imaging device 100 can be designed preferentially to a position suitable for imaging.

FIG. 4 is a flowchart of the subject size calculation process executed by the subject size calculation unit. A program based on the flowchart of FIG. 4 is recorded in the nonvolatile memory of the memory 105. This program is loaded into the RAM and executed by the subject size calculation unit 115.

The subject size calculation unit 115 detects the subject size (distance to the subject) using an imaging plane phase difference method. Specifically, in step S1000, the subject size calculation unit 115 obtains an image plane phase difference map (imaging plane phase difference map) from the phase difference calculation unit 113. The "image plane phase difference map" refers to a map in which the phase difference of each pixel with respect to a reference area within the pixel area 102a is calculated and the calculation result is displayed as a phase difference map.

In step S<b>1001 , the subject size calculation unit 115 acquires the readout unit and readout position currently set in the imaging control unit 110 from the imaging control unit 110 via the readout area determination unit 114 .

In step S1002, the subject size calculation unit 115 extracts an image plane phase difference submap from the image plane phase difference map. The "image plane phase difference submap" refers to a region within the pixel area 102a that corresponds to the readout unit and readout position currently set in the imaging control unit 110 and is displayed as a phase difference map. Further, the digital signal for phase difference calculation inputted to the phase difference calculation unit 113 may be a signal for each readout unit. In this case, since there is a correspondence between the current readout area and the image plane phase difference, the processes of step S1000 and step S1001 can be omitted.

In step S1003, the subject size calculation unit 115 calculates image plane phase difference data in the current readout unit by performing predetermined processing on the image plane phase difference submap. This predetermined process is not particularly limited, and includes, for example, an additive average, a weighted average, or a process in which center position data is used as a representative value.

In step S1004, the subject size calculation unit 115 transmits the image plane phase difference data calculated in step S1003 to the readout area determination unit 114 as the subject size. By using the imaging plane phase difference method as described above, it is possible to accurately detect the object size.

FIG. 5 is a flowchart of the readout area determination process executed by the readout area determination unit. A program based on the flowchart of FIG. 5 is recorded in the nonvolatile memory of the memory 105. This program is loaded into the RAM and executed by the read area determination unit 114.

In step S1100, the readout area determination unit 114 receives and acquires the subject size transmitted from the subject size calculation unit 115 in step S1004.

In step S<b>1101 , the readout area determination unit 114 acquires the readout unit and readout position currently set in the imaging control unit 110 from the imaging control unit 110 .

In step S1102, the readout area determining unit 114 determines whether the subject size acquired in step S1100 is smaller than a predetermined threshold. As a result of the determination in step S1102, if it is determined that the subject size is smaller than the predetermined threshold, the process proceeds to step S1103. On the other hand, as a result of the determination in step S1102, if it is determined that the subject size is not smaller than the predetermined threshold value, the process advances to step S1104. The predetermined threshold value may be a pixel size selected from the requirements of the minimum object size to be detected, or a pixel size of a readout unit. Alternatively, the size of the readout area 102b may be changed in steps by having a plurality of threshold values depending on the subject size and changing the readout unit or the thinning amount (the size of the thinning area 102c) in steps. In this way, changing the size of the reading area 102b includes changing at least one of the size of the reading unit and the size of the thinning area 102c. For example, when changing the size of the readout area 102b, if the size of the readout unit and the size of the thinning area 102c are changed respectively, the size can be changed finely, and the recognition rate of the object can be changed. Contribute to the improvement of Note that the predetermined threshold value is stored in the memory 105 in advance and can be changed as appropriate.

In step S1103, the readout area determination unit 114 performs a readout unit or thinning process so that the readout area 102b (data) read out from the image sensor 109 becomes dense, that is, increases the ratio occupied by the readout area 102b per unit area. Change the amount. As a result, the readout area 102b becomes, for example, on the side of the small subject "Z" surrounded by the thick line in FIG. 2(c).

In step S1104, the readout area determination unit 114 performs a readout unit or thinning process so that the readout area 102b (data) read out from the image sensor 109 becomes sparse, that is, the ratio occupied by the readout area 102b per unit area is reduced. Change the amount. As a result, the readout area 102b becomes, for example, on the side of the large subject "Z" surrounded by the thick line in FIG. 2(c). Note that in steps S1103 and S1104, the readout unit and the thinning amount are respectively selected from the readout area information 117, which is a nonvolatile memory in the memory 105.

In step S1005, the read area determining unit 114 determines the read unit and read position to be used for the next read. This determination is made based on the readout unit and readout position currently set in the imaging control unit 110, and the readout unit or thinning amount selected in step S1103 or step S1104.

In step S1106, the readout area determination unit 114 transmits the readout unit and readout position determined in step S1005 to the imaging control unit 110. Then, the imaging control unit 110 performs readout control of the imaging sensor 109 based on the readout unit and readout position transmitted in step S1106.

As described above, in the imaging device 100, the size of the readout area 102b determined by the readout area determination unit 114 is used to change the size of the readout area 102b. If the size of the subject in the captured image is smaller than a predetermined size, the readout unit or thinning amount is decreased to make the readout area 102b dense. On the other hand, if the size of the subject in the captured image is larger than a predetermined size, the readout unit or the thinning amount is increased to make the readout area 102b sparse. In this way, in the imaging device 100, the size of the readout area 102b is changed depending on the size of the subject. Thereby, regardless of the size of the subject within the captured image, the subject can be detected quickly, that is, the readout speed can be maintained. Further, it is also possible to suppress a decrease in the detection rate of the subject.

<Second embodiment>
The second embodiment will be described below with reference to FIG. 6, but the explanation will focus on the differences from the embodiments described above, and the explanation of similar matters will be omitted. This embodiment is the same as the first embodiment except for the configuration (method) for acquiring the object size. In the first embodiment, a method was presented in which the subject size was calculated based on the imaging plane phase difference detected by the imaging unit 102 and the readout was controlled. It may be data. When integrating the DSP 106 and the CPU 108 inside the sensor, it may be difficult to execute a large number of complex processes due to hardware constraints and the like. Therefore, in this embodiment, a more accurate distance image or the like is obtained from the outside and readout is controlled.

FIG. 6 is a block diagram showing a configuration example of an imaging device according to the second embodiment. As shown in FIG. 6, in this embodiment, the imaging device 100 constitutes an imaging system 500 together with an external device 400. The external device 400 has an information acquisition section (information acquisition means) 401. The information acquisition unit 401 is configured to acquire information (distance information to the subject) for the subject size calculation unit 115 to detect the size of the subject. The information acquisition unit 401 can be configured with at least one of a distance sensor and a stereo camera, for example. For example, an LRF (Laser Range Finder) or a LiDAR (Light Detection and Ranging) can be used as the distance sensor. A stereo camera having a plurality of cameras can be used. Thereby, the distance to the subject can be accurately detected. This distance information is then transmitted to the imaging device 100.

The imaging device 100 further includes an input section 200. The input unit 200 receives data such as distance images and image plane phase difference maps, which are distance information, from the information acquisition unit 401 and transfers them to the memory 105 . Then, the subject size calculation unit 115 can detect the size of the subject according to the distance information (the magnitude of the distance to the subject). For example, when the subject is far from the imaging device 100, the subject size calculation unit 115 detects the subject as being small, and when the subject is close to the imaging device 100, it detects the subject as being large. To detect. Further, the external device 400 may calculate the phase difference of the digital signal for phase difference calculation read from the imaging unit 102. In this case as well, accuracy can be expected to improve by executing complex processing externally. For example, it is conceivable that the imaging device 100 configured as shown in FIG. 6 is built into a digital still camera or a digital video camera, and the generation of the imaging plane phase difference map is executed by a processor on the camera body side. In this way, the imaging system 500 has an effective configuration, for example, when the imaging device 100 and the external device 400 are configured separately and the distance information obtained by the external device 400 is to be used in the imaging device 100. ing.

<Third embodiment>
The third embodiment will be described below with reference to FIG. 7, but the explanation will focus on the differences from the embodiments described above, and the explanation of similar matters will be omitted. This embodiment is similar to the first embodiment except for the configuration (method) for acquiring the object size. In the first embodiment, a method was presented in which the subject size was calculated based on the imaging plane phase difference detected by the imaging unit 102 and the readout was controlled. It may be calculated data. When detecting the object size based on the phase difference on the imaging plane, it is necessary to provide each pixel with a photoelectric conversion unit for detecting the phase difference. Therefore, in this embodiment, instead of using the subject size, data related to the subject size is calculated by image processing or the like, and the calculated data is treated as a simple subject size.

FIG. 7 is a block diagram showing a configuration example of an imaging device according to the third embodiment. As shown in FIG. 7, the imaging device 100 includes an image processing section 300 instead of the phase difference calculation section 113 in FIG. The image processing unit 300 performs simple image processing and rule-based determination processing for calculating the subject size on the digital signal acquired from the imaging control unit 110. For example, the image processing unit 300 can separate the subject and calculate its size using a general-purpose area division method such as the Watershed algorithm or superpixel. The image processing unit 300 can also treat the upper side of the image as distal to the imaging device 100 and the lower side of the image as proximal to the imaging device 100, based on a rule. This allows the subject size to be treated as simple data. Such a configuration contributes to maintaining the readout speed in the imaging unit 102.

Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above, and various modifications and changes can be made within the scope of the gist thereof. The present invention provides a program that implements one or more functions of each embodiment described above to a system or device via a network or a storage medium, and one or more processors of a computer in the system or device reads the program. It is also possible to implement the process by executing the process. The present invention can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

100 Imaging device 102 Imaging unit (imaging means)
102a Pixel region 102b Readout region 102c Thinning region 106 DSP (Digital Signal Processor)
108 CPUs
112 Recognition processing unit 114 Reading area determination unit 115 Subject size calculation unit

Claims (21)

an imaging means having a pixel area in which a plurality of pixels are arranged, and reading out a captured image from the pixel area;
recognition means that performs recognition processing to recognize a subject included in the captured image;
Detecting means for detecting the size of the subject,
The imaging device is characterized in that, when the imaging means reads out the captured image, the size of the readout area read out in the pixel area is changed depending on the size of the subject. Changing the size of the readout area includes changing at least one of the size of a readout unit that is read out in the pixel area in the smallest unit, and the size of a thinning area where readout is stopped in the pixel area. The imaging device according to claim 1, characterized in that: The imaging means decreases the size of the thinning area when the size of the subject is smaller than a predetermined size, and decreases the size of the thinning area when the size of the subject is larger than a predetermined size. The imaging device according to claim 2, characterized in that the size is increased. comprising determining means for determining the size of the readout area according to the size of the subject;
4. The imaging device according to claim 1, wherein the imaging device uses the size of the readout area determined by the determining device to change the size of the readout area. Imaging device. The imaging device according to any one of claims 1 to 4, wherein the imaging means changes the size of the readout area in stages. The imaging device according to any one of claims 1 to 5, wherein the detection means detects the size of the subject depending on a distance of the subject from the pixel area. . 7. The imaging apparatus according to claim 6, wherein the detection means detects the distance to the subject using an imaging plane phase difference method. 8. The recognition means recognizes the subject based on a learning model subjected to machine learning when performing the recognition process. imaging device. a first substrate on which the imaging means is configured;
The imaging device according to any one of claims 1 to 8, further comprising a second substrate configured separately from the first substrate and configured with the recognition means and the detection means. Device. An imaging device having a pixel area in which a plurality of pixels are arranged, and reading out a captured image from the pixel area, a recognition device performing recognition processing to recognize a subject included in the captured image, and detecting the size of the subject. an imaging device having a detection means for
an external device having information acquisition means for acquiring information for detecting the size of the subject by the detection means and capable of transmitting the information to the imaging device,
The imaging system is characterized in that, when the imaging means reads out the captured image, the size of the readout area read out in the pixel area is changed according to the size of the subject. 2. The information acquisition means includes at least one of a distance sensor that detects the distance to the subject, a plurality of cameras, and a stereo camera that detects the distance to the subject. 10. The imaging system according to 10. Changing the size of the readout area includes changing at least one of the size of a readout unit that is read out in the pixel area in the smallest unit, and the size of a thinning area where readout is stopped in the pixel area. The imaging system according to claim 10 or 11, characterized in that: The imaging means decreases the size of the thinning area when the size of the subject is smaller than a predetermined size, and decreases the size of the thinning area when the size of the subject is larger than a predetermined size. 13. The imaging system of claim 12, characterized in that it has an increased size. The imaging device includes determining means for determining the size of the readout area according to the size of the subject,
14. The imaging device according to claim 10, wherein the imaging device uses the size of the readout region determined by the determining device to change the size of the readout region. Imaging system. 15. The imaging system according to claim 10, wherein the imaging means changes the size of the readout area in stages. The information is distance information to the subject,
16. The imaging system according to claim 10, wherein the detection means detects the size of the subject according to the distance information. 17. The imaging system according to claim 16, wherein the information acquisition means detects the distance to the subject using an imaging plane phase difference method. According to any one of claims 10 to 17, the recognition means recognizes the subject based on a learning model subjected to machine learning when performing the recognition process. imaging system. a first substrate on which the imaging means is configured;
The imaging device according to any one of claims 10 to 18, further comprising a second substrate configured separately from the first substrate and configured with the recognition means and the detection means. system. an imaging step of having a pixel area in which a plurality of pixels are arranged, and reading out a captured image from the pixel area;
a recognition step of performing recognition processing to recognize a subject included in the captured image;
a detection step of detecting the size of the subject,
In the imaging step, when reading out the captured image, the size of the readout region read out in the pixel region is changed depending on the size of the subject detected in the detection step. Control method. A program for causing a computer to execute each means of the imaging apparatus according to claim 1.

Images