JP2019201324A - Imaging apparatus, information processing apparatus, control method of the same, and program - Google Patents

Abstract

To provide a technique enabling prevention of erroneous detection in a synthesis area for composite images that are subject to image analysis such as detecting the number of people.SOLUTION: The imaging apparatus includes a plurality of imaging units, each of which is arranged so that a part of an imaging angle of view overlaps with an adjacent imaging unit. The imaging apparatus includes: combining means for generating a combined image by performing a combining process for combining images captured by each of the plurality of imaging units; and communication means for transmitting the combined image to an information processing apparatus. The communication means transmits information on subject images that are duplicated or missing in the combined image as combined information to the information processing apparatus together with the combined image.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an imaging apparatus, an information processing apparatus, a control method thereof, and a program.

  As an imaging apparatus capable of capturing an image with a wide field of view, an imaging apparatus that realizes a wide field of view by arranging a plurality of lenses and imaging elements and synthesizing the obtained images has been proposed (see Patent Document 1). In addition, in an imaging apparatus used for monitoring or the like, a function for determining the presence / absence of a person who has passed through a setting area or a function for detecting the number of persons in the area may be implemented as an image analysis function.

JP 2006-229789

  When image analysis is performed using a composite image obtained by the imaging apparatus of Patent Document 1, the following problems occur.

  Depending on the arrangement configuration of a plurality of imaging devices, there may be a region where a double image is generated at a combined portion of an image and a region where a defect is generated depending on the position of the subject. Therefore, when such a composite image is subjected to image analysis to detect the number of people, there is a possibility that a problem may occur when the subject subject passes through the composite region of the image. Specifically, it may be impossible to detect when a part or the whole of the subject is missing, or the number of persons may be mistaken due to duplication, which is a cause of erroneous detection.

  Therefore, the present invention provides a technique for preventing erroneous detection in a composite area for a composite image that is an object of image analysis such as detection of the number of people.

The present invention for solving the above problems is an imaging apparatus including a plurality of imaging units,
Each of the plurality of imaging units is arranged so that a part of the imaging angle of view overlaps with an adjacent imaging unit,
Combining means for generating a combined image by performing a combining process for combining images captured by each of the plurality of imaging units;
Communication means for transmitting the composite image to an information processing device,
The communication means transmits information related to a subject image overlapping or missing in the composite image as composite information to the information processing apparatus together with the composite image.

  ADVANTAGE OF THE INVENTION According to this invention, the technique for making it possible to prevent the misdetection in a synthetic | combination area | region can be provided about the synthetic | combination image used as the object of image analysis, such as a person detection.

The figure which shows the structural example of the imaging device which concerns on the 1st Embodiment of invention, and the figure for demonstrating arrangement | positioning of the imaging part in the said imaging device. 2 is a diagram illustrating an example of functional configurations and hardware configurations of the imaging apparatus 100 and the client apparatus 180 according to the first embodiment of the invention. FIG. FIG. 3 is a diagram illustrating an example of a composite image according to the first embodiment of the present invention, and a diagram for explaining a relationship between a subject and an imaging field angle of each imaging unit of the imaging apparatus. 6 is a flowchart illustrating an example of an image composition process corresponding to the first embodiment of the invention. The figure for demonstrating generation | occurrence | production of the overlap defect | deletion of the to-be-photographed image which concerns on the 1st Embodiment of invention. The figure for demonstrating the image composition information which concerns on the 1st Embodiment of invention. 6 is a flowchart showing an example of processing performed by the client device 180 according to the first embodiment of the invention. The figure for demonstrating the example of a setting of the 2nd analysis area | region in the client apparatus 180 which concerns on the 1st Embodiment of invention. The figure which shows the function structural example of the imaging device 100 which concerns on the 2nd Embodiment of invention. 6 is a flowchart showing an example of the operation of the imaging apparatus 100 according to the second embodiment of the invention. The flowchart which shows an example of operation | movement of the client apparatus 180 which concerns on the 2nd Embodiment of invention. The figure for demonstrating the image composition information which concerns on the 2nd Embodiment of invention. The flowchart which shows an example of the overlap defect determination process 1 which concerns on the 2nd Embodiment of invention. The figure for demonstrating the duplication defect | deletion determination process 1 which concerns on the 2nd Embodiment of invention. The flowchart which shows an example of the overlap defect determination process 2 which concerns on the 2nd Embodiment of invention. The figure for demonstrating the duplication defect | deletion determination process 2 which concerns on the 2nd Embodiment of invention.

  Embodiments of the present invention will be described below in detail with reference to the accompanying drawings.

[Embodiment 1]
In the following embodiment, an embodiment will be described in which the invention is applied to an imaging apparatus capable of generating a wide-angle image or a wide-angle moving image obtained by combining images captured by a plurality of imaging units.

  First, FIG. 1A is a schematic diagram illustrating a configuration of the imaging apparatus 100 according to the present embodiment. FIG. 1B is a schematic diagram illustrating the arrangement of each imaging unit in the imaging apparatus 100 in a planar shape. As shown in FIG. 1B, in the present embodiment, an example in which the imaging apparatus 100 includes eight imaging units 101 to 108 is shown. Each of the imaging units 101 to 108 is an imaging unit including a lens optical system and an imaging element, and is arranged radially with respect to the center of the imaging device 100 to enable imaging of a range of 360 degrees. In the present embodiment, the range of 360 degrees is covered by the eight imaging units 101 to 108, but the number of imaging units is not limited to this, and the range that can be imaged is from 360 degrees. May be narrow. For example, the imaging range can be determined according to the installation location.

  Each imaging unit has an imaging angle of view. In FIG. 1B, these are alternately shown by a solid line and a dotted line. For example, like the imaging field angle 122 of the imaging unit 102 adjacent to the imaging field angle 121 of the imaging unit 101, the imaging field angles are respectively arranged so as to partially overlap. The imaging apparatus 100 can synthesize images captured by the imaging units and generate a wide-angle moving image having a horizontal field of view of 360 ° at the maximum.

  Next, the functional configuration and hardware configuration of the imaging apparatus 100 and the client apparatus 180 will be described with reference to FIG. FIG. 2A is a schematic diagram illustrating a functional configuration example of the imaging device 100 and the client device 180 corresponding to the embodiment of the invention. The control unit 150 can be configured by, for example, a CPU, MPU, other dedicated arithmetic circuit, and the like, and controls the entire imaging apparatus 100. The memory 160 includes a nonvolatile memory and a RAM. The nonvolatile memory stores a processing procedure (control program) of the control unit 150 and various parameters. The RAM is used as a work area for the control unit 150 and is also used as a storage area for performing image processing. The control unit 150 can include an image processing unit 151, an image composition unit 152, and a compression / decompression unit 153. The processing functions of these blocks are, for example, corresponding programs stored in the memory 160 by the CPU. It can be realized by executing. Alternatively, the control unit 150 may be realized by a dedicated arithmetic circuit.

  The imaging units 101 to 108 have a unit configuration that includes the imaging elements 131 to 138 and the lens optical systems 111 to 118, respectively. Each imaging unit is connected to and controlled by the control unit 150, and imaging that is synchronized with each other is performed. An imaging signal obtained by such synchronous imaging constitutes synthesized image data of one frame through a subsequent synthesis process. Therefore, hereinafter, “every frame” means an imaging signal obtained by synchronous imaging or image data obtained from the imaging signal. The image pickup signal transmitted from the image pickup unit 101 or the like is subjected to various image processing in the control unit 150. The imaging elements 131 to 138 are configured by a CMOS sensor or the like, and convert a subject image formed on the imaging surface into an electrical signal and output it. An imaging signal as an electrical signal output from the imaging element 131 or the like is input to the image processing unit 151 in the control unit.

  The image processing unit 151 performs image processing such as pixel interpolation processing and color conversion processing on the captured image signal, and captures captured image data (“captured image” for each image capturing unit, or “ Also referred to as “pre-combination image”). The image processing includes various correction processing such as pixel defect correction and lens correction, detection processing for adjusting black level, focus, exposure, etc., demosaic processing, white balance processing, white balance processing and gamma correction processing, Edge enhancement processing, noise suppression processing, and the like are included. Each captured image data is stored in the memory 160. Further, the captured image data for each imaging unit stored in the memory 160 is sequentially combined for each frame by the image combining unit 152 and stored in the memory 160 as wide-angle combined image data.

  The control unit 150 further compresses the composite image data by the compression / decompression unit 153 to generate compressed image data. The compression / decompression unit 153 executes still image compression and moving image compression. H.264, H.C. 265, based on standards such as MPEG or JPEG. Furthermore, image data in an arbitrary format including mp4 or avi format may be generated. The compressed image data generated by the compression / decompression unit 153 is recorded in a recording medium (not shown) or a built-in memory attached to the imaging device 100, and is transmitted from the communication unit 170 to the external information processing apparatus via the network 190. It is transmitted to a certain client device 180. The communication unit 170 is a network processing circuit, converts the compressed composite image data into a communication signal conforming to a communication protocol, and distributes the communication signal onto the network 190.

  The client device 180 is typically an information processing device such as a personal computer, and is connected to the imaging device 100 via the network 190. The control unit 181 of the client device 180 can be configured by a CPU, MPU, other dedicated arithmetic circuit, and the like, and controls the entire client device 180. In addition, the control unit 181 receives composite image data from the imaging device 100, performs decompression processing, and performs various controls by transmitting control information for controlling the imaging device 100. The image analysis unit 182 performs image analysis of the composite image data received from the imaging device 100. Further, the image analysis unit 182 may perform extension processing. In the image analysis, for example, a subject existing in a specific area in the composite image data is detected and the number of people is counted, a movement of the subject is detected, and an entry into a predetermined area is detected. The display unit 183 displays the image analysis result and the composite image data received from the imaging device 100 to the operator of the client device 180. The communication unit 184 is a network processing circuit, communicates with the imaging device 100 via the network 190, receives composite image data distributed by the imaging device 100, and can control the operation of the imaging device 100. is there.

  An example of a hardware configuration of the imaging device 100 and the client device 180 described above is illustrated in FIG. FIG. 2 is a block diagram illustrating an example of a hardware configuration mainly related to the control unit and the processing unit of the imaging apparatus 100 and the client apparatus 180.

  In FIG. 2B, the CPU 210 executes an operating system (OS), a control program, a processing program, and the like stored in a hard disk device (hereinafter referred to as HD) 215, and processing corresponding to the embodiment of the invention. Is realized. Furthermore, data transmission / reception with an external apparatus via the interface (I / F) 218 is controlled. The CPU 210 can function as the control unit 150 or 181 in FIG.

  The ROM 211 stores various data such as an application program for executing predetermined processing in addition to the basic I / O program. The RAM 212 temporarily stores various data and functions as a main memory, work area, and the like for the CPU 210. The external storage drive 213 is an external storage drive for realizing access to a recording medium, and can load a program or the like stored in the medium (recording medium) 214 into the computer system. In this embodiment, the hard disk device 215 uses an HD (hard disk) that functions as a large-capacity memory. The HD 215 stores application programs, OS, control programs, related programs, and the like. A nonvolatile storage device such as a flash (registered trademark) memory may be used instead of the hard disk. These ROM, RAM, HD, and the like can function as the memory 160 in FIG.

  The instruction input device 216 corresponds to a keyboard, a pointing device (such as a mouse), a touch panel, or the like. The output device 217 outputs a command input from the instruction input device 216, a response output of the client device 180 in response thereto, and the like. The output device 217 can include a display, a speaker, a headphone terminal, and the like. This corresponds to the display portion 183 in FIG. A system bus 219 manages the flow of data in the apparatus.

  An interface (hereinafter referred to as I / F) 218 plays a role of mediating exchange of data with an external device. Specifically, the I / F 218 can include a wireless communication module, which includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset. Well-known circuitry, including a subscriber identity module card, memory, and the like. In addition, a wired communication module for wired connection can be included. The wired communication module enables communication with other devices via one or more external ports. It can also include various software components that process the data. The external port is coupled to another device directly via Ethernet, USB, IEEE1394, or indirectly via a network. In addition, it can also be comprised as an alternative of a hardware apparatus with the software which implement | achieves a function equivalent to the above each apparatus. The interface 218 can function as the communication unit 170 or 184 in FIG.

  Next, an image composition process according to this embodiment will be described with reference to FIGS. 3 and 4. Hereinafter, a case where a wide-angle moving image is generated will be described. FIG. 3 is a diagram for explaining the relationship between the subject corresponding to the embodiment of the invention and the imaging angle of view of each imaging unit of the imaging apparatus 100. FIG. 3A illustrates an example of a composite image 300 obtained by combining the captured images captured by the imaging units 101 to 104. FIG. 3B is a diagram for describing image composition processing of each captured image captured by the imaging units 101 to 104, which is executed to obtain a composite image such as the composite image 300. 3B, dashed lines 121 to 124 correspond to the imaging field angles 121 to 124 of the imaging units 101 to 104 in FIG. As shown here, the imaging angle of view of each imaging unit is set to partially overlap each other.

  When combining images captured by the image capturing units, there is a parallax between adjacent image capturing units depending on the arrangement of the image capturing units. Therefore, synthesis is performed for each frame in an image region (image overlap region) where the imaging angle of view overlaps. It is necessary to adjust the composition position of the image. The composition position adjustment is performed so that the same subject image in the image overlap region overlaps. In the example shown in FIG. 3B, the shaded areas 301 to 303 are image overlapping areas. The image overlap area can be arbitrarily set by setting the imaging angle of view according to the subject distance when the imaging apparatus 100 is installed. Further, the image overlap area may be configured to be appropriately changed according to the subject. The imaging device 100 synthesizes the captured images from the imaging units to obtain a composite image 300. Here, four captured images are combined. However, all the captured images captured in synchronism, or in the case of the imaging apparatus 100 shown in FIG. 1, eight images are combined to form a single combined image. Alternatively, a plurality of synthesized images may be generated by synthesizing in units of subgroups of captured images. For example, a total of two composite images may be generated for every four subgroups. Furthermore, the number of composites is not limited to these.

  Next, an image composition process according to this embodiment will be described with reference to FIG. FIG. 4 is a flowchart illustrating an example of an image composition process corresponding to the embodiment of the invention. The processing corresponding to the flowchart can be realized, for example, by executing a program (stored in the memory 160 or the like) corresponding to one or more processors (CPU, MPU of the control unit 150) functioning as the control unit 150.

  First, in S401, the control unit 150 issues an imaging instruction to each of the imaging elements of the imaging units 101 to 108. Each imaging element performs an imaging operation based on an imaging instruction from the control unit 150, and an imaging signal output from each imaging unit is stored in the RAM of the memory 160. The imaging signal received from the imaging unit is subjected to predetermined image processing such as pixel interpolation processing and color conversion processing in the image processing unit 151, and is held in the memory again as captured image data.

  In subsequent step S <b> 402, the control unit 150 selects partial image data corresponding to the image overlap area from the captured image data stored in the memory 160 and supplies the partial image data to the image composition unit 152. In subsequent S403, the image composition unit 152 adjusts the composition position. The image composition unit 152 detects the same subject in the image data of each image overlap region between the two images to be combined, and combines them so that the positions of the same subject detected in the pre-combine image match. A position correction parameter is calculated. The position correction parameters include parameters for performing deformation processing such as enlargement / reduction of each captured image data and distortion correction, and parameters for performing image cutout from the captured image data after the correction processing. For the detection of the same subject, for example, it is possible to determine whether the subject is the same subject by detecting the edge by background difference or the like and extracting the contour shape of the subject, or by extracting the motion vector and determining it together with the color pixel information. . Note that the processing of S402 and S403 may be executed together by the image composition unit 152.

  In subsequent S <b> 404, the image composition unit 152 performs a composition process for combining the captured image data obtained from the image capturing units to generate composite image data. Based on the calculated correction parameter, a combination position correction process is performed on each captured image data, and combination is performed for each frame. Thereafter, an appropriate scaling process is performed to cut out to a designated angle of view, and wide-angle composite image data is generated. The generated combined image data is stored in the memory 160 together with information (image combining information) regarding image combining processing performed by the image combining unit 152. In subsequent S 405, the compression / decompression unit 153 sequentially compresses the combined image data and outputs it to the communication unit 170 as moving image data. At this time, the image composition information generated by the image composition unit 152 is added to the moving image data. Details of the image composition information will be described later. In subsequent S406, the communication unit 170 distributes the moving image data to the client device 180 via the network 190 based on a predetermined protocol.

  Next, with reference to FIG. 5, description will be given on the occurrence of overlap and loss of subject images in the image composition processing of the image composition unit 152. First, FIG. 5A is a diagram for explaining the relationship between the imaging angle of view of the imaging apparatus 100 and the subject position. Here, the relationship between the imaging unit 101 and the imaging unit 102 is shown, but this is merely omitting other imaging units to show an example, and the same relationship applies to other imaging units. Can be established. Since the image capturing units 101 and 102 of the image capturing apparatus 100 have different installation positions, the image capturing angle of view does not match the position of the optical axis center. For this reason, depending on the position of the subject, the imaging angle of view of these imaging units may not be entered. Subjects 501 to 503 are subjects having different distances from the imaging apparatus 100. The subjects 501 and 502 are subjects that enter the imaging angle of view 121 and 122, but deviate from any of the imaging angle of view at the position of the subject 503.

  FIGS. 5B and 5C show an example of image data of each imaging unit that images the subject of FIG. 5A and a combined image after combining. First, in the example of FIG. 5B, the composite image 513 is obtained by combining the image data 511 and the image data 512. In the example of FIG. 5C, the captured image 511 and the captured image 512 are combined to obtain a combined image 514. The composite image 513 and the composite image 514 have different composite positions. The composite image 513 shown in FIG. 5B is aligned with the subject 501, and the entire image of the subject 501 is well contained in the composite image. However, the subject 502 is generated as two images (multiple images) in the composite image 513.

  The composite image 514 shown in FIG. 5C is aligned with the subject 502, and the entire image of the subject 502 is well within the composite image. However, the subject 501 is synthesized in a state where a part of the image is missing, and it is difficult to determine the original subject 501.

  As described above, depending on the relationship between the position of the subject and the composition position, the subject image in the composite image is lost or overlapped. As a result, the client apparatus that has received the composite image makes a false detection when performing image analysis. Therefore, in order to prevent erroneous detection in the client device 180, the imaging apparatus 100 corresponding to the present embodiment performs distribution by adding information on the image combination position to the synthesized image as image synthesis information. Hereinafter, addition of image composition information for preventing erroneous detection of multiple images will be described.

  FIG. 6 is a diagram for explaining the image composition information corresponding to the embodiment of the invention. In the present embodiment, a case will be described in which information indicating a composite position (composite position information) is notified to the client device 180 as image composite information. A composite image 601 shown in FIG. 6A represents at least a part of a composited image distributed to the client device 180. Dashed lines 602 and 603 indicate image combining positions at the time of synthesis. Then, the image synthesis information is expressed as coordinate information of two points (AA ′) on the boundary line where the image synthesis indicated by the coupling positions 602 and 603 as shown in FIG. It can be provided to the client device 180.

  Further, as shown in FIG. 6B, the image synthesis information may be expressed as a region including an image boundary. For example, the image composition information can also be transmitted to the client device 180 as a polygon composed of a plurality of points like image composition areas 611 and 612 in the composite image 610. At this time, the image composition information is, for example, coordinate information of the points A to D, and is notified to the client device as a curve or a region connecting the coordinates. The number of coordinates is not limited to four and can be set to an arbitrary number.

  Further, as shown in FIG. 6C, the image composition information can be transmitted to the client device 180 as a combination of a plurality of regions centered on the composition position. For example, in the case of a subject having a depth from the lower side to the upper side of the image 620, a defective area may be generated in front of the aligned subject, and a multiple image may be generated behind the aligned subject. At this time, the region 621 can be notified as a multiple image generation region including overlapping subjects, and the region 622 as a defective region where there is a high possibility that the subject is defective. The region where multiple images are generated and the region where image defects occur are changed according to the distance of the subject to be aligned at the time of synthesis, and may be appropriately changed according to the installation environment and the subject being watched. .

  Next, image data and image composition information received by the client device 180 will be described. FIG. 7 is a flowchart illustrating an example of processing performed by the client device 180 corresponding to the embodiment of the invention. The processing corresponding to the flowchart can be realized, for example, when one or more processors (CPU, MPU) functioning as the control unit 181 and the image analysis unit 182 execute the corresponding program.

  First, in step S <b> 701, when the control unit 181 of the client device 180 receives moving image data via the communication unit 184, the control unit 181 performs decompression processing and restores composite image data. In subsequent S <b> 702, the control unit 181 of the client device 180 displays the composite image data obtained in S <b> 701 on the display unit 183. In subsequent S703, the control unit 181 of the client device 180 determines whether image synthesis information is added to the obtained synthesized image data. If there is no image synthesis information, the process proceeds to S706, the received synthesized image data is analyzed by the image analysis unit 182 in the client device 180, and the analysis result is displayed on the display unit 183 in the subsequent S706. Overlays the data. On the other hand, when the image composition information is added, the process proceeds to 4. In S <b> 704, if the information specifying the image composition area is included based on the image composition information extracted from the received data, the control unit 181 designates the image composition area as an analysis area (second area specific to the present embodiment). Analysis area). In addition, when the image composition information includes boundary information for image combination, the second analysis region is set based on the boundary line. The second analysis region is an analysis different from the first analysis region when a region in which analysis processing can be performed without being influenced by image synthesis in the composite image data is set as the first analysis region. Represents an area to which processing is applied.

  In subsequent S705, the control unit 181 superimposes a marker (frame display, marker display, etc.) for notifying the user of the set second analysis region on the already displayed composite image data on the display unit 183. indicate. Thereafter, in S706, the image analysis unit 182 analyzes the first analysis region, and counts the number of persons existing in the region included in the composite image, for example. In subsequent S707, the control unit 181 displays the analysis result in S706, for example, the counted number of people on the display unit 183.

  FIG. 8 is a diagram for explaining a setting example of the second analysis area in the client device 180. A broken line 801 in FIG. 8A indicates a combined position of images (boundary line of combined images). An area 802 indicates a second analysis area. The broken line 801 and the area 802 may be superimposed on the image so as to be visible to the user of the client device 180. FIG. 8B shows an example where the subject crosses the second analysis region. The illustrated subjects 811 to 813 move from the position indicated by the subject 811 to the position indicated by the subject 813 as time passes in the composite image. Here, the client device 180 sets the area 810 as the second analysis area. In the present embodiment, the analysis processing to be performed is different between the inside of the second analysis region and the first analysis region outside the second analysis region. Specifically, inside the second analysis region 810, the subject shape may be collapsed in the vicinity of the combined position of image synthesis, and therefore processing for excluding the subject shape from the determination condition is performed. For example, in the first analysis region, the subject is tracked using three types of information, ie, the shape, color (color in the outline), and motion (motion vector) of the subject. On the other hand, in the second analysis area, the subject shape may be collapsed at the boundary of image composition, so the subject shape is excluded from the determination conditions, and the control method is performed in a tracking manner using only the color and the motion vector. Can be changed.

  As described above, according to the present embodiment, the client device 180 is notified of the image composition information regarding the overlapping subject and the missing subject in the composite image, thereby causing the client device to erroneously detect image analysis. Can reduce the possibility of causing

[Embodiment 2]
Next, a method for generating image synthesis information corresponding to the second embodiment of the invention will be described. The method can be performed in addition to the first embodiment of the invention described above. First, FIG. 9 is a block diagram illustrating a functional configuration example of the imaging device 100 and the client device 180 corresponding to the present embodiment. The same configuration as that in FIG. 2A is omitted, and only the difference will be described.

  The control unit 150 of the imaging apparatus 100 corresponding to the present embodiment includes an overlap defect determination unit 901 in addition to the configuration of FIG. The overlap defect determination unit 901 compares the images before and after the synthesis and determines the overlap and loss of the subject image. Other structures are similar to those in FIG. The hardware configuration of the client device is the same as that shown in FIG.

  With reference to FIG. 10, the overlap defect determination processing in the present embodiment will be described. FIG. 10 is a flowchart illustrating an example of the overlap defect determination of the imaging apparatus 100 corresponding to the embodiment of the invention. The processing corresponding to the flowchart can be realized, for example, by executing a program (stored in the memory 160 or the like) corresponding to one or more processors (CPU, MPU of the control unit 150) functioning as the control unit 150.

  First, in step S <b> 1001, the control unit 150 selects captured image data captured by the image capturing apparatus 100 and stored in the memory 160, and the image composition unit 152 combines the captured image data through image composition processing, and composite image data. Is generated. The generated composite image data is stored in the memory 160. In subsequent S1002, the overlap defect determination unit 901 determines whether or not there is a subject overlap or defect in the composite image. Details of the determination process will be described later with reference to FIGS. If it is determined that the subject images are duplicated or missing, the process proceeds to S1003. If no overlap or loss has occurred, the process proceeds to S1004. In S1003, the image composition unit 152 generates image composition information according to the determination result in S1002. In S <b> 1004, the communication unit 170 distributes the image composition information together with the image data to the client device 180 via the network 190.

  Next, the operation on the client device 180 side will be described with reference to FIG. FIG. 11 is a flowchart showing an example of processing performed by the client device 180 corresponding to the second embodiment of the invention. The processing corresponding to the flowchart can be realized, for example, when one or more processors (CPU, MPU) functioning as the control unit 181 and the image analysis unit 182 execute the corresponding program.

  First, in step S1101, when the control unit 181 of the client device 180 receives the composite image data via the communication unit 184, the control unit 181 performs decompression processing to restore the composite image data. In subsequent S1102, the control unit 181 of the client device 180 displays the composite image data obtained in S1101 on the display unit 183. In subsequent S1103, the control unit 181 determines whether or not image synthesis information is added to the obtained synthesized image data. If there is no image synthesis information, the process proceeds to S1106, where the received synthesized image data is analyzed by the image analysis unit 182 in the client device 180, and the analysis result is displayed on the display unit 183 in the subsequent S1107. Overlays the data. On the other hand, if the image composition information is added to the received data, the process proceeds to S1104.

  In step S1104, the control unit 181 of the client device 180 performs image analysis based on the image synthesis information extracted from the received data, and identifies a subject image or an image region that is overlapped or missing in the received synthesized image. In subsequent S1105, the control unit 181 superimposes a marker (a frame display, a marker display, etc.) for notifying the user of the set overlapping or missing location on the already displayed composite image data. 183. Thereafter, in S1106, the image analysis unit 182 analyzes the composite image data, and counts the number of persons included in the composite image, for example. In subsequent S1107, the control unit 181 displays the analysis result in S1106, for example, the counted number of people on the display unit 183.

  Next, an example of the data structure of image composition information including overlap / deletion information added to composite image data in the imaging apparatus 100 will be described with reference to FIG. FIG. 12A shows an example of a composite image 1200 generated by the imaging apparatus 100. FIG. 12B shows an example of the data structure of the image composition information 1210 that the imaging apparatus 100 adds to the composite image 1200 and transmits it.

  A composite position 1201 is indicated by a broken line on the composite image 1200 in FIG. The position on the composite image 1200 of the composite position 1201 is defined by the composite position 1211 of the image composite information 1210 in FIG. As shown in the synthesis position 1211, the synthesis position can be represented by a plurality of coordinates from (x0, y0) to (xn, yn). A curve connecting the coordinates indicates a combination position (boundary line of the combined image) when the images captured by the imaging units are combined.

  The overlap / deletion flag 1212 is flag information for notifying whether or not a multiple image of a subject image or a defect has occurred. The flag information can be expressed by, for example, 2 bits, and the first bit can indicate the presence or absence of duplication, and the second bit can indicate the presence or absence of a defect. Specifically, “11” indicates that both duplication and deficiency exist. “01” represents only duplication, “10” represents only deficiency, and “00” represents that no duplication or deficiency has occurred. The duplication / deletion flag 1212 is flag information indicating the presence / absence of duplication / deletion occurring in relation to the synthesis position 1211. The presence / absence of duplication / deletion occurring at other synthesis positions of the synthesized image data is determined by the synthesis. Registered in relation to location.

  When the overlap / deletion flag 1212 is not “00”, subject information and position information for specifying a subject image that is overlapping or missing is further added to the image composition information 1210. A target 1213 is identification information for identifying a subject in which duplication or loss occurs. The identification information may be any information as long as the information can uniquely identify the same subject. In the present embodiment, alphabets such as A and B are assigned to individual subjects for identification.

  The position of each subject in the image is specified by information specifying the position and size of each subject image on the composite image data included in subject information 1214 to 1217 such as A1, A2, B1, and B2. Specifically, a subject in which overlapping / deletion of the subject image has occurred can be expressed as, for example, coordinates (x, y, r) based on information indicating the center position and size of the subject image. x and y are subject coordinates, and r is a circle with a radius r centered on the subject coordinates, indicating the approximate size of the subject. The subject coordinates are not necessarily the center of the subject. It is only necessary that the subject image can be discriminated on the client device 180 side, and any point within the contour of the subject subject can be used. Further, when the contour of the subject is extracted from the background difference information on the client device 180 side, it can be recognized if the subject coordinates transmitted from the imaging device 100 are included in the contour.

  The image composition information 1210 indicates that the subjects A1 and A2 are subject images of the same subject A. There is no particular limitation on the number of subjects, and it is preferable to prepare as many information as the number of subjects with overlapping / deletion. The subjects A1 and A2 are handled by one set of data. When two left and right images are combined, the positional relationship may be indicated as A1 on the left and A2 on the right. Alternatively, the state of the combined subject may be determined, and A1 may be assigned as the main subject and A2 may be assigned as the sub-subject.

  Further, the information on the main subject and the sub-subject may be incorporated into the coordinate information. For example, a bit p (1 bit) indicating the priority order is added to the coordinates (x, y, r) and can be expressed as (x, y, r, p). The value of p can represent a main subject at 0 and a sub-subject at 1.

  FIG. 12B shows an example of image composition information registered in relation to the composition position 1201 in FIG. 12A, and the composite image data has a plurality of composition positions. A set of information as shown in FIG. 12B is registered for each synthesis position. Further, the image composition information can be generated when duplication or loss of the subject image occurs, and can be generated in units of one frame. Further, when a plurality of composite image data is generated from the captured image data captured synchronously in the imaging units 101 to 108 in units of subgroups, image composite information is generated for each composite image data. .

  In the example of the subjects 1202 and 1203 in FIG. 12A, the subject 1202 whose entire image is known is the main subject A1, and the subject 1203 in which a part of the image is missing is sent as the sub-subject A2 to the client device. Can do. By assigning priorities in this way, it is possible to increase detection accuracy when the client device 180 that has received the image composition information 1210 performs image analysis.

  12C and 12D show examples of images displayed on the display unit 183 provided on the client device 180 side. The subjects 1221 and 1222 in FIG. 12C are in an overlapping (multiple image) relationship, and the subject 1221 is the main subject A1 and the subject 1222 is the sub-subject A2 in the received image composition information 1210. It is displayed. As described above, in the display of the client device 180, markers such as markers can be superimposed and displayed in a manner corresponding to the priority order with respect to the subject images that are overlapped or missing. At this time, the priority can be identified by the color or shape of the sign to be displayed. A subject 1231 in FIG. 12D shows a state of being missing from the image. Accordingly, at this time, the subject 1232 becomes the main subject A1.

  Hereinafter, for example, a case where the client apparatus 180 executes an image analysis function for counting the number of subjects will be described. When the client device 180 detects that the subject is doubled in the image composition information acquired from the imaging device 100, the client device 180 includes only the subject image with the highest priority as the count target, and otherwise. Subject images are excluded from the count target. For example, the sub-object 1222 in FIG. 12C is excluded from the count target. Further, the client device 180 displays information including a priority relationship with respect to the multiple images based on the image composition information from the imaging device 100. As an example, the subject is surrounded by a broken line, but the display method is not limited to this. Various expression methods can be applied, such as displaying a subject with a different color or overlaying marker displays such as arrows. A broken line 1201 is a boundary line at the time of image composition. It may be presented in a form that is visible to the user, or may be configured to be switchable between display and non-display.

  Next, with reference to FIGS. 13 and 14, a specific example of the method for determining the overlapping defect of the subject image by the overlapping defect determination unit 901 in S1002 of FIG. 10 will be described.

  FIG. 13 is a flowchart illustrating a specific example of the overlap defect determination process. In step S <b> 1301, the image composition unit 152 of the image capturing apparatus 100 sets a determination region for determining overlapping deficiency with respect to captured image data (pre-combination image) before combining, and subject images that exist within the set determination region. To extract. In subsequent S1302, the overlap defect determination unit 901 determines whether or not the subject images detected in S1301 are those of the same subject among the pre-combination images to be synthesized. Whether or not they are the same subject can be determined based on the subject shape, the subject color, and the motion vector, for example.

  In subsequent S1303, the overlap defect determination unit 901 determines whether or not subject images that are recognized as subject images of the same subject between the pre-combination images have a relationship in which overlap defect occurs. Duplication / deletion occurs according to the position and distance of the subject within the imaging angle of view. Therefore, based on whether or not the subject exists at a specific position and distance within the imaging angle of view, it can be determined whether or not the subject image overlaps or is lost after the composition. In the present embodiment, since the arrangement of the imaging unit and the imaging angle of view are known information, the position and distance of the subject within the imaging angle of view can be specified by the coordinates of each subject image on the two pre-combine images. it can. Therefore, it is possible to determine whether duplication or loss occurs after composition depending on where the subject image is located in each pre-combination image.

  The method for determining the position and distance of the subject is not limited to the above. For example, when the imaging apparatus 100 includes an imaging device that can detect the phase difference of the subject image formed on the imaging device, the subject distance can be calculated from the subject image formed on the imaging device. Further, a method of detecting the subject distance from the contrast value at which the subject image is formed may be used like a general contrast AF (autofocus) operation.

  Next, the determination process will be described more specifically with reference to FIG. FIG. 14A is a schematic view of a scene in which the imaging units 101 and 102 are imaging the subject 1401 as seen from above. It is assumed that the subject 1401 moves sequentially to the positions T1 to T3. A subject 1402 is a subject that has existed in the imaging environment since the imaging device 100 was installed. The image composition unit 152 adjusts the composition position based on the subject 1402. Dashed lines 1403 and 1404 conceptually indicate subject distances that can be combined without overlapping or missing images in the combined image of the imaging apparatus 100. The subject 1401 crosses a position farther than the subject 1402 with respect to the imaging apparatus 100. Regions 1405, 1406, and 1407 are regions that overlap each other at the imaging angle of view of the imaging units 101 and 102, respectively. Among these, the subject existing in the region 1406 has duplicate subject images in the composite image data.

  FIG. 14B shows a pre-combination image and a composite image for the positions T1 to T3 under the situation of FIG. First, images 1411, 1412, 1421, 1422, 1431, and 1432 are pre-combination images captured by the imaging units 101 and 102, respectively. Images 1413, 1423, and 1433 are synthesized images after synthesis. Areas 1414 and 1415 are determination areas set in the image 1411, and areas 1416 and 1417 are determination areas set in the image 1412. The determination areas are similarly set for the images 1421, 1422, 1431, and 1432, respectively. Here, of the determination areas, areas 1415 and 1416 correspond to first areas not included in the combined image, and areas 1414 and 1417 correspond to second areas included in the combined image. Further, although the boundary between the first area and the second area is indicated by a broken line 1441 and a broken line 1442, these serve as a connecting portion when the images are combined.

  Specifically, when the subject 1401 is at the position T1, since the subject 1401 is outside the determination region, the subject image does not overlap. When the subject 1401 is at the position T2, the overlap defect determination unit 901 searches for the subject image in the overlap defect determination area of the captured images 1421 and 1422 because it enters the determination area, specifically, the area 1414 and the area 1416. Extract a subject image of the same subject. Subsequently, the overlap defect determination unit calculates the spatial position where the subject 1401 exists from the parallax information of the two images (coordinates on the two pre-combination images), and determines whether the overlap defect occurs. In the example shown in FIG. 14B, the subject image of the subject 1401 is included in the region 1414 in the image 1421, but is included in the region 1416 in the image 1422. Here, since the region 1416 is a first region not included in the composite image data, it can be determined that no overlap occurs in the composite image data. Further, since the subject image of the subject 1401 of the image 1421 does not belong to the first region, it can be determined that no defect occurs. The subject image of the subject 1401 of the image 1422 belongs only to the first region, and is completely deleted from the composite image data. Therefore, the composite image data 1423 includes only the subject image 1424 on the image 1421 side.

  Next, when the subject 1401 is at the position T3, the overlap defect determination unit 901 determines that the subject 1401 is located so as to belong to both the first region 1415 and the second region 1414 in the image 1431. In this case, since a portion belonging to the first region 1415 in the subject image is deleted in the composite image data, it can be determined that a defect occurs. In addition, in each of the images 1431 and 1432, the subject image exists in the second regions 1414 and 1417, so that it can be determined that overlap has occurred. At this time, since the subject image on the image 1432 side belongs only to the second region 1417, no defect occurs and the entire image remains. The composite image data 1433 includes a subject image 1434 in which a part is lost and a subject image 1435 in which the entire image remains. In such a case, the image composition unit 152 can determine the subject image 1434 as the sub subject and the subject image 1435 as the main subject.

  As described above, according to the method described in relation to FIGS. 13 and 14, the determination area including the first area and the second area is set as the pre-combination image, and at least a part of the subject image is the first area. If it belongs to one region, it can be determined that a defect occurs. Further, if subject images (may be partial) extracted from each of the pre-combination images to be synthesized belong to the second region, subject images overlap after synthesis. Can be determined.

  Next, with reference to FIGS. 15 and 16, another specific example of the subject image overlap defect determination method by the overlap defect determination unit 901 in S1002 of FIG. 10 will be described.

  FIG. 15 is a flowchart illustrating a specific example of the overlap defect determination process. In step S1501, the overlap defect determination unit 901 of the imaging apparatus 100 sets determination areas for determining overlap defects for the pre-combination image and the composite image, and detects a subject image in the determination area. In subsequent S1502, the overlap defect determination unit 901 determines whether the number of detected subject images that have entered the determination area increases or decreases. This is done by comparing the number of input / output of subject images to the determination area with the number of subject images in the determination area. If the number of subject images in the determination area exceeds the number of inputs / outputs to the determination area, it is determined that overlap (multiple images) of the subject images has occurred. On the other hand, if the number of subject images in the determination area is less than the number of inputs / outputs to the determination area, it is determined that a defect has occurred.

  In subsequent S1503, the overlap defect determination unit 901 compares the pre-combination image and the composite image to determine whether there is a change in the subject image. If there is a change in the subject image, it can be determined that a defect has occurred. In S1502, since the determination is based only on the number of subject images, it is conceivable that the number of detections does not differ even if there is a partially missing subject image. Therefore, even if the number of moving body images matches in S1502, it is determined whether or not the subject image has changed before and after the synthesis, in particular whether or not the shape of the subject image has changed. It is determined that a defect has occurred. When a part of the subject image is lost, it is difficult to determine the shape of the subject image on the composite image. Therefore, it is determined whether or not the shapes of the subject image of the pre-combination image corresponding to the subject image on the composite image match. The comparison between the pre-combination image and the composite image is preferably performed between the pre-combination image and the composite image stored in the memory 160. However, when the movement of the subject image is small, the composite image and the pre-combination image after one frame are used. May be compared. Thereby, the capacity of the memory 160 can be reduced.

  In subsequent S1504, the overlap defect determination unit 901 generates coordinate information in the composite image for the subject image in which overlap or defect is detected based on the determination result. At this time, if the subject images overlap, the multiple images can be compared to determine the main subject and the sub-subject. The image synthesis unit 152 can generate image synthesis information based on the information generated in this way.

  Next, the overlap defect determination method of FIG. 15 will be described more specifically with reference to FIG. FIG. 16A is a schematic view of a scene in which the imaging units 101 and 102 are imaging the subject 1601 as seen from above. The case where the subject 1601 sequentially moves to positions T1 to T5 is shown. A subject 1602 is a subject serving as a reference for the synthesis position. Regions 1603, 1604, and 1605 are regions that overlap each other at the imaging angle of view of the imaging units 101 and 102, respectively. Among these, the subject existing in the area 1604 is duplicated in the composite image data.

  FIG. 16B shows a pre-combination image and a composite image at positions T1 to T5 under the situation of FIG. Images 1611, 1612, 1621, 1622, 1631, 1632, 1641, 1642, 1651, and 1652 are pre-combine images respectively captured by the imaging units 101 and 102. Images 1613, 1623, 1633, 1643, and 1653 are synthesized images after synthesis. Areas 1614 and 1615 are determination areas set for the image 1611, and areas 1616 and 1617 are determination areas set for the image 1612. An area 1618 is a determination area set in the composite image 1613. The determination area is similarly set for each of the other pre-combination images 1621 and 1623. Here, among the determination areas, areas 1615 and 1616 correspond to a first area not included in the combined image, and areas 1614 and 1617 correspond to a second area included in the combined image. In addition, a boundary between the first area and the second area is indicated by a broken line 1671 and a broken line 1672, which serve as a connecting portion when the images are combined. In order to distinguish from the determination area of the pre-combine image, the area 1618 is set as a third area.

  Specifically, when the subject 1601 is at the position T1, the subject image 1663 is not detected by the overlap defect determination unit 901 because there is no subject image in the third determination region 1618 of the composite image. However, the detection range of the subject image may be limited to the determination area or the entire image area. By limiting the size of the area, the processing load can be reduced.

  When the subject 1601 is at the position T2, it overlaps that the subject image 1663 is entered as the subject image in the third determination region 1618 of the composite image 1623 and that the subject image 1663 exists in the third determination region 1618. The defect determination unit 901 detects it. At this time, since the number of input / outputs of the subject in the determination area 1618 is the same as the number of subject images, it is determined that neither the target subject image 1663 is duplicated nor missing.

  Next, when the subject 1601 is at the position T3, a new subject image 1664 is generated in the third determination region 1618, whereby the number of subject images in the third determination region 1618 of the composite image 1633 is determined to be two. Is done. On the other hand, since the number of inputs / outputs to / from the third determination region 1633 does not increase or decrease, the number of detected subject images exceeds the number of inputs / outputs, and it is determined that the subject images may have overlapped.

  Here, the overlap defect determination unit 901 compares the pre-combination images 1631 and 1632 with the composite image 1633, and subject images that may have overlapped are displayed in the second determination regions 1614 and 1617 on the pre-combine image. Therefore, it is determined that the subject images of the subject 1601 overlap. Here, out of the subject image 1663 and the subject image 1664, the subject image 1664 in which a larger area is displayed as the subject image is the main subject, and the subject image 1663 partially missing is the sub-subject. When the subject is a person, an organ such as a face becomes a more characteristic part than the body. Therefore, a subject image including more important organs instead of the size of the subject image may be used as the main subject. The priority of the subject image when overlap occurs may be changed as appropriate according to the type of subject.

  Next, when the subject 1601 is at the position T4, the input / output number of subjects in the third determination area 1618 of the composite image 1643 is not changed, and the number of detected subjects is 1, so the input / output number and subject image This is consistent with the detected number. However, it is difficult to determine the subject image. Therefore, the overlap defect determination unit 901 compares the images before and after the synthesis and determines that a defect has occurred in the subject image 1663. In response to this, the image composition unit 152 distributes, to the client device, together with the image data, information indicating that a defect has occurred in the subject image 1643 as image composition information.

  Next, when the subject 1601 is at the position T5, the subject image cannot be detected within the third determination region 1618 of the composite image 1653. Therefore, the number of detected subject images is 0, and the number of input / outputs is not changed, so the number of input / outputs is greater than the number of detected subject images. In this case, it can be determined that a defect has occurred in the subject image. The overlap defect determination unit 901 compares the images before and after the synthesis and determines that a defect has occurred in the subject 1601. The image composition unit 152 distributes to the client device together with the composite image data that the subject image 1663 is deficient as image composition information.

  The client device 180 that has received the image composition information notifies the image analysis unit that the subject is missing in the region of the subject image 1663 and performs image analysis. For example, when counting the number of subjects as image analysis, the client device does not display the subject 1601 on the image data, but assumes that the subject 1601 exists based on the image composition information, the number of subjects = 1. Count. Further, a marker is displayed to notify the user that the subject image 1663 is missing on the display unit, and image data is displayed on the display unit.

  As described above, the case where the overlap defect determination is performed by the imaging apparatus has been described, but the configuration in which the overlap defect determination on the composite image is performed by the client apparatus 180 and the imaging apparatus 100 is inquired when there is a possibility that the overlap defect has occurred. It doesn't matter. The imaging apparatus 100 can also obtain the same effect by holding the pre-combination images 1611 and 1612 in the memory and performing the overlap defect determination in response to an inquiry from the client apparatus.

  As described above, it is possible to prevent erroneous detection during image analysis by notifying the client device 180 of the existence of a subject in which duplication or loss has occurred. In addition, it is possible to notify the user of the client device 180 of overlapping or missing object images.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100: imaging device, 180: information processing device

Claims (23)

An imaging device comprising a plurality of imaging units,
Each of the plurality of imaging units is arranged so that a part of the imaging angle of view overlaps with an adjacent imaging unit,
Combining means for generating a combined image by performing a combining process for combining images captured by each of the plurality of imaging units;
Communication means for transmitting the composite image to an information processing device,
The image capturing apparatus, wherein the communication unit transmits information related to a subject image overlapping or missing in the composite image as composite information to the information processing apparatus together with the composite image.   The composite information includes at least one of information for specifying a boundary of combined images and information for specifying a region including the boundary in the composite image. The imaging device described in 1.   The imaging device according to claim 2, wherein the region includes at least one of a region including overlapping subject images and a region in which a subject image is missing.   The composite information includes information on the presence or absence of duplication or loss of subject images in the composite image, and information for specifying the subject image in the composite image. The imaging apparatus according to item 1.   The imaging apparatus according to claim 4, wherein the combination information further includes priority assigned to the plurality of subject images when the information includes a plurality of subject image information for the same subject. A determination means for determining whether duplication or loss of the subject image occurs in the composite image;
The imaging apparatus according to claim 1, wherein the composite information is generated when the determination unit determines that the subject image is duplicated or missing.   The determination unit sets a determination region for an image captured by each of the plurality of imaging units, and the subject is included when the determination region of the combined image includes a subject image of the same subject. The imaging apparatus according to claim 6, wherein it is determined that image overlap occurs.   The determination area includes a first area that is not included in the synthesized image and a second area that is included in the synthesized image, and at least a part of the subject image is included in the first area. The image pickup apparatus according to claim 7, wherein the determination unit determines that the subject image has a defect in the composite image.   The determination unit sets a determination region for the composite image, counts the number of subjects entering / exiting the determination region and the number of subjects in the determination region, and counts the number of subjects input / output and the number of subjects in the determination region The imaging apparatus according to claim 6, wherein it is determined whether or not the subject images are duplicated or missing based on a difference from the number of subjects.   The imaging apparatus according to claim 9, wherein the determination unit determines that the subject images overlap when the number of subjects in the determination region is larger than the number of subjects to enter and exit. .   11. The determination unit according to claim 9, wherein the determination unit determines that the subject image is missing when the number of subjects in the determination region is smaller than the number of subjects entering and exiting. Imaging device.   The determination means compares the shape of the subject image in the determination region extracted in the composite image with the shape of the corresponding subject image extracted in the image captured by each of the plurality of imaging units, The imaging apparatus according to claim 9, wherein if there is no match, it is determined that the subject image is missing. An information processing apparatus,
Communication means for receiving the composite image and the composite information from the imaging apparatus according to any one of claims 1 to 3,
Control means for controlling a display unit for displaying the composite image;
Analyzing means for performing analysis processing on the composite image,
The control means sets an area based on the composite information in the composite image,
In the analysis processing to be performed on the composite image, the analysis unit makes the processing different between the inside of the region and the outside of the region,
The information processing apparatus, wherein the control unit controls the display unit to display an analysis result by the analysis unit.   The information processing apparatus according to claim 13, wherein the control unit controls the display unit to display a sign indicating the region so as to be superimposed on the composite image. The analysis process includes counting the number of subject images included in the composite image,
The analysis means performs the counting using at least information on the shape of the subject image inside the region, and performs the counting without using information on the shape of the subject image outside the region. The information processing apparatus according to claim 13 or 14, characterized in that: The analysis means counts the number of the subject images based on information on the shape, color, and movement of the subject images,
The information processing apparatus according to claim 13, wherein the analysis unit performs the analysis processing without using the shape information inside the region. An information processing apparatus,
Communication means for receiving the composite image and the composite information from the imaging apparatus according to any one of claims 4 to 12,
Control means for controlling a display unit for displaying the composite image;
Analyzing means for performing analysis processing on the composite image,
The control means controls the display unit to superimpose and display a sign on a subject image that is overlapped or missing in the composite image based on the composite information,
The information processing apparatus, wherein the control unit controls the display unit to display an analysis result by the analysis unit. An information processing apparatus,
Communication means for receiving the composite image and the composite information from the imaging device according to claim 5 and any one of claims 6 to 12 dependent on claim 5.
Control means for controlling a display unit for displaying the composite image;
Analyzing means for performing analysis processing on the composite image,
The control means controls the display unit to superimpose and display a marker on a subject image of a subject that is overlapped or missing in the composite image based on the composite information,
The control means controls the display unit to display an analysis result by the analysis means,
The analysis process includes counting the number of subject images included in the composite image,
The analysis unit includes only the subject image having the highest priority among the plurality of subject images based on the priority order assigned to the plurality of subject images. Processing equipment.   19. The information processing according to claim 18, wherein the control unit controls the display unit to display the sign in a manner corresponding to a priority order assigned to the plurality of subject images. apparatus. A method for controlling an imaging apparatus including a plurality of imaging units arranged so that a part of an imaging angle of view overlaps with an adjacent imaging unit,
A generating step in which a combining unit performs a combining process for combining images captured by each of the plurality of imaging units to generate a combined image;
A communication step of transmitting the composite image to an information processing apparatus;
Including
In the transmission step, information regarding a subject image that is overlapped or missing in the composite image is transmitted as composite information to the information processing apparatus together with the composite image. A method for controlling an information processing apparatus, comprising:
A receiving step in which a communication unit receives the composite image and the composite information from the imaging apparatus according to any one of claims 1 to 3.
An analysis step in which the analysis means performs an analysis process on the composite image;
The control means includes a display step of displaying the analysis result in the analysis step and the composite image on the display unit,
In the analysis step, in the analysis processing performed on the composite image, the processing is different between the inside of the region set by the control unit in the composite image based on the composite information and the outside of the region. Control method for information processing apparatus.   The program for functioning a computer as each means of the imaging device of any one of Claims 1-12.   A program for causing a computer to function as each unit of the information processing apparatus according to any one of claims 13 to 19.