JP2019115031A - Information processing apparatus, system, information processing method, and program - Google Patents

Abstract

To output an image suitable for an imaging scene or the like.SOLUTION: The information processing apparatus includes: setting means for setting a pattern for outputting a partial image obtained from an omnidirectional image on the basis of the number and positions of regions of interest obtained from the omnidirectional image which is a captured image captured using a fisheye lens; cutting means for cutting out a partial image including a region of interest from the omnidirectional image on the basis of the pattern set by the setting means; correction means for performing distortion correction processing on the partial image cut off by the cutting means; and output means for outputting the partial image subjected to distortion correction processing by the correction means in accordance with the pattern set by the setting means.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing apparatus, system, information processing method, and program.

Conventionally, a fisheye camera equipped with a fisheye lens as one of the types of surveillance cameras has penetrated. As one unit can monitor 360 °, it leads to the reduction of the number and cost. There is also a camera capable of outputting not only an omnidirectional image obtained from a fisheye camera as it is but also an image or video subjected to a plurality of types of clipping processing and distortion correction processing. In such a camera, when using various video content analysis and the like such as human body detection and moving body detection, an image after cutting processing and distortion correction processing is often used. Furthermore, such a camera can output a plurality of images subjected to a plurality of types of cutting processing and distortion correction processing, and can perform monitoring according to the imaging environment and the installation environment.
However, it is necessary to set an appropriate clipping process or distortion correction method for the imaging scene monitored by the user via the user interface. Also, if the cropping process and distortion correction method set once are used, it is not possible to cope with changes in the imaging scene.
For example, Patent Document 1 discloses a method of detecting feature amounts of a plurality of images and switching to a display device at each continuous display time of each designated image.

JP, 2015-222917, A

  However, regarding the display of a captured image obtained from a fisheye camera, a display suitable for a captured scene or the like is further required.

  An information processing apparatus according to the present invention is a pattern for outputting a partial image obtained from the omnidirectional image, based on the number and position of a notable area obtained from the omnidirectional image which is a captured image captured using a fisheye lens. And a cutting unit that cuts out a partial image including the region of interest from the omnidirectional image based on the pattern set by the setting unit, and a distortion with respect to the partial image cut by the cutting unit. The image processing apparatus further includes: a correction unit that performs a correction process; and an output unit that outputs a partial image that has been subjected to the distortion correction process by the correction unit according to the pattern set by the setting unit.

  According to the present invention, it is possible to output an image suitable for an imaging scene or the like.

It is a figure showing an example of the hardware constitutions of a camera. It is a figure which shows an example of an imaging part. It is a figure which shows an example of the software configuration of a camera. It is a figure which shows an example of a captured image. It is a figure which shows an example of the image output by an output part. It is a figure which shows an example of a captured image. It is a figure which shows an example of the image output by an output part. It is a flowchart which shows an example of the information processing of a camera. It is a flowchart which shows an example of the information processing of a camera. FIG. 6 is a diagram showing an example of a pattern 1; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 7 is a diagram showing an example of a pattern 2; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 10 is a diagram illustrating an example of a pattern 3; It is a figure which shows an example of the image in which the distortion correction process was performed. It is a figure which shows an example of the image output by an output part. It is a figure which shows an example of the hardware constitutions of the apparatus which comprises an imaging system. It is a figure which shows an example of the software configuration of the apparatus which comprises an imaging system. It is a flowchart which shows an example of the information processing of a client apparatus. It is a figure which shows an example of a captured image. It is a figure which shows an example of patterns other than the pattern 2. FIG. It is a flowchart which shows an example of the information processing of a client apparatus. FIG. 10 is a diagram illustrating an example of a pattern 3; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 7 is a diagram showing an example of a pattern 2; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 10 is a diagram illustrating an example of a pattern 3; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 10 is a diagram illustrating an example of a pattern 3; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 10 is a diagram illustrating an example of a pattern 3; FIG. 10 is a diagram illustrating an example of a pattern 3; It is a figure which shows an example of the image in which the distortion correction process was performed. FIG. 6 is a diagram showing an example of a pattern 1; It is a figure which shows an example of the image in which the distortion correction process was performed.

  Hereinafter, embodiments of the present invention will be described based on the drawings. Below, the case where a camera is used as an imaging device is demonstrated.

First Embodiment
FIG. 1 is a diagram showing an example of the hardware configuration of the camera 10. As shown in FIG. The camera 10 is a fisheye camera. The camera 10 includes an imaging unit 11, a CPU 12, a memory 13, an input unit 14, a display unit 15, and a communication unit 16 as a hardware configuration. The imaging unit 11 captures an image of a subject. Details of the imaging unit 11 are shown in FIG. 2 described later. The CPU 12 controls the entire camera 10. The memory 13 stores a program, an image captured by the imaging unit 11, a setting value, and the like. The input unit 14 inputs the user's selection operation and the like, and passes it to the CPU 12. The display unit 15 displays a screen or the like based on the control of the CPU 12. The communication unit 16 connects the camera 10 to a network and controls communication with other devices. By the CPU 12 executing processing based on the program stored in the memory 13, the software configuration of the camera 10 shown in FIG. 3 described later and the processing of the flowchart in FIG. 8 described later are realized. The camera 10 is an example of an information processing apparatus. The following embodiments can also be applied to a recording server that accumulates omnidirectional images acquired from the camera 10 via a network, or a terminal device for displaying omnidirectional images.

  FIG. 2 is a diagram showing an example of the imaging unit 11. The imaging unit 11 includes a lens 201 including a plurality of lens groups, and an imaging element 202 such as a CCD or a CMOS. In addition, the imaging unit 11 includes a correlated double sampling (CDS) circuit 203 that performs noise reduction. The imaging unit 11 also includes a gain control amplifier circuit (Automatic Gain Control: AGC) 204 that automatically performs gain control of the camera. The imaging unit 11 further includes an A / D conversion 205 that converts an analog signal into a digital signal.

  FIG. 3 is a view showing an example of the software configuration of the camera 10. As shown in FIG. The imaging control unit 101 controls the imaging unit 11 and passes the signal from the imaging unit 11 to the image generation unit 102. The image generation unit 102 generates a captured image based on the received signal. The determination unit 103 determines a captured scene from the captured image. The pattern setting unit 104 cuts a partial image from the captured image, and sets a pattern for arranging an image subjected to distortion correction on the output image. The cut position setting unit 105 sets the cut position of the captured image according to the set pattern. The cutting unit 106 cuts out the image at the cutting position set by the cutting position setting unit 105. The distortion correction processing unit 107 performs distortion correction corresponding to the image cut by the cutting unit 106, and performs processing of arranging the cut image based on the pattern set by the pattern setting unit 104 as an output image. The output unit 108 outputs the image signal that has been subjected to the distortion correction by the distortion correction processing unit 107.

The operation of each component described above will be described in detail.
First, the imaging apparatus will be described in detail with reference to FIGS. 2 and 3. An imaging element 202 converts an object image formed through a lens 201 as an imaging optical system into an electrical signal. The lens 201 may be a wide-angle lens with a large distortion and a wide angle of view or a fisheye lens. In the present embodiment, the case of a fisheye camera provided with a fisheye lens as an imaging device will be described. The CDS circuit 203 performs correlated double sampling processing or the like on the electrical signal output from the imaging element 202. The AGC amplifier 204 performs amplification processing and the like on the electrical signal output from the CDS circuit 203. The A / D conversion 205 converts the analog signal amplified by the AGC amplifier 204 into a digital signal. The imaging control unit 101 passes the luminance signal and the color signal obtained from the imaging unit 11 to the image generation unit 102. The image generation unit 102 generates an image signal such as an RGB image or a YUV image from the obtained luminance signal and color signal. The determination unit 103 determines the number of attention areas and the position of the attention areas from the image generated by the image generation unit 102. The pattern setting unit 104 sets a pattern based on the number and the position (or distribution) of the attention area in the image determined by the determination unit 103. The cutout position setting unit 105 sets a position at which the attention area in the image determined by the determination unit 103 is cut out, corresponding to the pattern set by the pattern setting unit 104. At this time, a cutting position is set such that one region of interest is not divided. The cutting unit 106 cuts a part of the image from the image generated by the image generation unit 102 using the cutting position set by the cutting position setting unit 105. The distortion correction processing unit 107 performs distortion correction processing such as various geometric transformations on the image cut off by the cutting unit 106. Then, based on the pattern set by the pattern setting unit 104, the distortion correction processing unit 107 executes a process of arranging the distortion-corrected image as an output image.

  Here, the image cut out by the cutting unit 106 and the image corrected by the distortion correction processing unit 107 will be described with reference to FIGS. 4 to 7. The omnidirectional image 30 in FIG. 4 is an example of an image obtained by imaging a direction perpendicular to the floor with a fisheye camera equipped with a fisheye lens installed on an indoor ceiling. The cutout unit 106 cuts out the area 301 and the area 302 which are similarly divided by the broken line except for the part of the area 300 which is divided by the broken line in the omnidirectional image 30 of FIG. 4. Then, FIG. 5 shows an example of an output image in which the distortion correction processing unit 107 performs distortion correction processing, and the areas 302 are vertically inverted and connected. FIG. 4 is an example in which the area 300 is regarded as an invalid area and is not regarded as an output image, but the area 301 and the area 302 including the attention area are regarded as effective areas and is regarded as an output image. Furthermore, as another example, as shown in FIG. 6, an example in which the region 400 to 402 which is the region of interest is partially cut out by the cutout unit 106 and the distortion correction processing unit 107 performs distortion correction processing can be considered. In the areas 400 to 402 of FIG. 7, images obtained by performing distortion correction processing on the images of the cut areas 400 to 402 shown in FIG. 6 are arranged. In the example of FIG. 6, the areas 400 to 402 of FIG. 6 are regarded as effective areas and used as an output image, and the other areas are regarded as invalid areas and are not used as an output image. Note that the output unit 108 may combine three partial images and output as one image, or may output these three partial images as a plurality of streams. Similarly, in the case of four or more, for example, each cut out image may be synthesized and output after being scaled according to the size of the image output by the output unit 108, or may be output as a plurality of streams. The output unit 108 outputs the image signal that has been subjected to the distortion correction processing by the distortion correction processing unit 107. The output destination of the image signal may be a display incorporated in the camera 10 or an external terminal device connected to the network via the communication unit 16.

  An example of setting a pattern and a cutout position for outputting an image in accordance with the condition of the attention area of the captured image will be described using the flowchart of FIG. In S101, the determination unit 103 acquires the image generated by the image generation unit 102. This image is an omnidirectional image before distortion correction processing. Next, in S102, the determination unit 103 extracts a region of interest from the acquired omnidirectional image before distortion correction processing. Next, in S103, the determination unit 103 determines whether there is one or more attention areas. The determination unit 103 may calculate the distribution of faces and / or persons in the image, and determine an area having the number of faces and / or persons in the image larger than the set number as the attention area, or the density degree is set. An area higher than the value may be determined as the area of interest. In addition, the determination unit 103 may calculate the staying frequency or flow line density of the face or person from the image of one sheet, or calculate the staying frequency or flow line density of the face or person from the plurality of images. May be In the case of detecting a face or a person using pattern matching, it is necessary to detect the face or the person in an image which has been subjected to distortion correction. In addition, the determination unit 103 may simplify the analysis and count the number of foreground areas of a predetermined size or more. In the present embodiment, the determination unit 103 sets an area in which a predetermined number (for example, three people) or more of persons are present in an area having a predetermined size with respect to the peripheral area in the omnidirectional image before distortion correction processing. It is determined that On the other hand, with respect to the central area in the omnidirectional image before the distortion correction processing, the determination unit 103 sets an area in which one or more persons are present in the area having the set size as a focused area. This is because, in the image before the distortion correction processing, the person present in the central portion is photographed larger than the peripheral portion. The set size of the above-mentioned area may be determined in advance as a proportion of the entire image and may be stored in the memory 13 or may be stored in the memory 13 or the like according to user operation via the input unit 14 or the like. It may be stored. If the notable area is not present on the image (No in S103), the determination unit 103 transitions to S104, and if there is one or more notable areas on the image (Yes in S103), the determination unit 103 transitions to S105. In S104, the pattern setting unit 104 sets a pattern set by the user in advance. In addition, the cut position setting unit 105 sets a cut position corresponding to the pattern. Here, the pattern set in advance by the user may be any one of pattern 1, pattern 2 and pattern 3, or may be an omnidirectional image before cutting. In the present embodiment, it is assumed that an omnidirectional image before clipping which allows the entire image to be overlooked is used as a pattern set in advance by the user, and this omnidirectional image is output. In S105, the determination unit 103 determines whether there are two or more attention areas. If the determination unit 103 determines that two or more attention areas do not exist (No in S105), the process transitions to S106. If two or more attention areas exist (Yes in S105), the determination unit 103 transitions to S107.

  In step S106, the pattern setting unit 104 sets pattern 1 to be output as one image without outputting divided images in order to correspond to one target area. In addition, the cut position setting unit 105 sets a cut position that includes one target area so as to correspond to the set pattern 1. In S107, the determination unit 103 determines whether there are three or more attention areas. The determination unit 103 transitions to S108 when three or more attention areas do not exist (No in S107), and transitions to S109 when three or more attention areas exist (Yes in S107). In step S108, the pattern setting unit 104 generates a two-screen split image (double panorama) and sets pattern 2 for output in order to correspond to the two areas of interest. Further, the cut position setting unit 105 sets a cut position such that each of the detected attention areas is not divided so as to correspond to the set pattern 2. In step S109, the pattern setting unit 104 generates a 4-screen split image and sets pattern 3 for output in order to correspond to three or more target areas. In addition, the cutting position setting unit 105 cuts out the attention area so as to correspond to the set pattern.

  An example in which the image is cut out based on the set pattern and the cut position, the distortion correction is performed, and the image is output will be described using the flowchart of FIG. 9. In S150, based on the pattern set by the pattern setting unit 104 and the position set by the cutting position setting unit 105, the cutting unit 106 cuts out a region of interest from the captured image. In the example of the pattern 1 and the cutting position set in S106, the cutting unit 106 cuts out the attention area 501 located at the upper left surrounded by one broken line from the entire image of FIG. 10 in the example of the cutting position. In addition, in the example of the pattern 2 and the cutting position set in S108, the cutting unit 106 is an area 601 represented by a semicircle except the middle invalid area surrounded by two broken lines from the entire image of FIG. Cut out a partial image from 602. That is, the cutting position is set so that the attention area is included in each of the two cut-out images. In addition, in the example of the pattern 3 and the cutting position set in S109, the cutting unit 106 cuts out the attention areas 701, 702, and 703 surrounded by three broken lines from the entire image in FIG. In S151, the distortion correction processing unit 107 performs distortion correction processing on the image cut out in S150. For example, when the pattern is pattern 1 or pattern 3, the distortion correction processing unit 107 executes distortion correction processing according to the pattern. Also, for example, when the set pattern is the pattern 2, the distortion correction processing unit 107 executes distortion correction processing according to the pattern and the cutout position. Here, the distortion correction processing is, for example, processing such as scaling of an image or geometric transformation. In S152, the output unit 108 outputs the image subjected to the distortion correction processing in S151. FIG. 11 is a view showing an example of the image of the attention area 501 of FIG. 10 cut out according to the setting of S106, distortion correction processing, and output to the client device 20 shown in FIG. 17 described later. FIG. 13 is a view showing an example in which the areas 601 and 602 in FIG. 12 are cut out according to the setting of S106, the distortion correction processing is performed, and the output. As shown in FIG. 13, in pattern 2, a two-screen split image called double panorama is output. FIG. 15 is a view showing an example in which the attention areas 701, 702, and 703 in FIG. 14 are cut out according to the setting of S109, distortion correction processing is performed, and the result is output. That is, in the pattern 3, a 4-screen divided image is output. In the case where pattern 3 is selected and there are only three areas of interest, a vacant area is generated. In this case, the output unit 108 rearranges the cut images 701, 702, and 703 as in the image of FIG. 16, and reduces the omnidirectional image, which is a captured image, as illustrated in 801 of FIG. May be output. This omnidirectional image is effective when there are five or more regions of interest. In the flowchart shown in FIG. 8, when three or more attention areas exist, pattern 3 is selected. When five or more attention areas exist, when one attention area is assigned to the divided image, an attention area that can not be output occurs. In the present embodiment, when there are five or more attention areas, adjacent attention areas may be grouped to create four groups. Also, if there are five or more areas of interest, or if the areas of interest overlap and an optimal cropping position can not be set, all directions before distortion correction so that the entire image can be viewed without outputting a partial image You may output an image.

  Each pattern in the present embodiment is an example at the time of cutting. Other patterns with different number of cuts and cut positions may be used. In addition, the change timing of each pattern may be immediately reflected on the determination result of the determination unit 103, may be a predetermined constant interval, or may be arbitrarily changed by the user. The distortion correction processing unit 107 performs distortion correction processing on the clipped image, and the output unit 108 outputs the image.

  As described above, according to the process of the present embodiment, it is possible to prevent the attention region (the object of interest) from being overlooked due to the image clipping. In addition, it is possible to output an image that has been subjected to appropriate clipping processing and distortion correction processing according to the imaging scene or the like without being set by the user. Therefore, since the pattern of the image and the distortion correction process are appropriately changed according to the imaging scene or the like, an image suitable for monitoring can be provided.

Second Embodiment
FIG. 17 is a diagram illustrating an example of a hardware configuration of an apparatus that configures an imaging system. In the imaging system, the camera 10 and the client device 20 are communicably connected via a network. The network may be wired or wireless. The hardware configuration of the camera 10 is the same as that of the first embodiment. The CPU 12 executes the processing based on the program stored in the memory 13 to realize the software configuration and the like of the camera 10 shown in FIG. 18 described later. The client device 20 includes a CPU 21, a memory 22, an input unit 23, a display unit 24, and a communication unit 25 as a hardware configuration. The CPU 21 controls the entire client device 20. The memory 22 stores a program, an image transmitted from the camera 10, setting values, and the like. The input unit 23 inputs the user's selection operation and the like, and passes it to the CPU 21. The display unit 24 displays a screen or the like based on the control of the CPU 21. The communication unit 25 connects the client device 20 to a network, and controls communication with another device (for example, the camera 10). The CPU 21 executes the processing based on the program stored in the memory 22 to realize the software configuration of the client device 20 shown in FIG. 18 described later and the processing of the flowchart shown in FIG. 19 described later. The client device 20 is an example of an information processing apparatus.

  FIG. 18 is a diagram showing an example of the software configuration of an apparatus that constitutes an imaging system. The configuration of the imaging unit 11 and the image generation unit 102 of the camera 10 is the same as that of the first embodiment and thus the description thereof is omitted. The imaging direction detection unit 121 detects the imaging direction of the imaging unit 11. For example, the imaging direction detection unit 121 may acquire the angle at which imaging is performed from the output of the gyro sensor. The determination unit 103, the pattern setting unit 104, the cutting position setting unit 105, the cutting unit 106, the distortion correction processing unit 107, and the output unit 108 of the client device 20 are the same as the configuration of the camera 10 of the first embodiment. I omit it. The communication processing unit 122 communicates with the camera 10 and receives the image generated by the image generation unit 102 and the imaging direction detected by the imaging direction detection unit 121.

Next, using the flowchart shown in FIG. 19, an example in which the imaging system including the camera 10 and the client device 20 sets and outputs the pattern and the cropping position of the image according to the conditions of the attention area of the captured image explain. The process contents of S101 to S109 in the flowchart of FIG. 19 are the same as those of the first embodiment, and the description thereof is omitted. Further, with regard to the patterns and the like of FIGS.
In S201, the determination unit 103 determines whether the imaging direction of the imaging unit 11 is horizontal based on the imaging direction of the imaging unit 11 detected by the imaging direction detection unit 121 and received via the communication processing unit 122. At this time, the imaging direction is, for example, the case where the camera 10 is installed on a wall surface, and the case where the camera 10 is installed on a ceiling to capture an image in the vertical direction. If the determination unit 103 determines that the imaging direction is the horizontal direction, the process proceeds to step S106. If the determination unit 103 determines that the imaging direction is not the horizontal direction, the process proceeds to step S108. In pattern 2, a cutting process and a distortion correction process are performed on the peripheral image excluding the central portion of the entire image. Therefore, when the imaging direction is horizontal, the pattern setting unit 104 sets pattern 1 in S106. In addition, the cut position setting unit 105 sets a cut position at which one partial image is cut so as to include two regions of interest so as to correspond to the set pattern 1. At this time, it is considered that the region of interest appears in the central portion excluding the upper and lower regions of the omnidirectional image, so the cutout position is set so as to cut out the central portion of the omnidirectional image. On the other hand, when the imaging unit 11 is not in the horizontal direction, that is, for example, is facing the vertical direction, the pattern setting unit 104 sets the pattern 2 in S108. In addition, the cutout position setting unit 105 sets the cutout position so that one of the two partial images cut out includes the region of interest so as to correspond to the set pattern 2. In S202, the determination unit 103 determines whether or not an attention area such as a specific moving object has appeared at the center of the entire image as shown in FIG. When the determination unit 103 determines that a specific moving object or the like has appeared in the central portion of the entire image (Yes in S202), the process transitions to S203, and the specific moving object does not appear in the central portion of the entire image. In the case (No in S202), the processing of the flowchart in FIG. 19 is ended. In S203, the pattern setting unit 104 temporarily sets a pattern other than the pattern 2, for example, the pattern 3 as shown in FIG. Further, the cut-out position setting unit 105 sets positions at which three partial images are cut out so as to respectively include the regions of interest in accordance with the set pattern 3. Here, the determination unit 103 may recognize a specific subject as a specific moving object by face recognition or the like, or specify when the size or speed of the moving object exceeds a predetermined threshold or a threshold set by the user. It may be recognized as a moving body of Further, the central portion of the image is, for example, within a range set from the center of the image. Pattern 2 is an example of a first pattern. The process of S203 is an example of a process of setting a pattern other than the first pattern.

  As described above, according to the processing of the present embodiment, in addition to the effects of the first embodiment, it is possible to provide an image suitable for monitoring a specific moving object.

Embodiment 3
The third embodiment will explain an example different from the first and second embodiments in setting the pattern and the cutout position corresponding to the pattern according to the condition of the attention area of the captured image. The hardware configuration and software configuration of the camera 10 and the client device 20 are the same as those of the above-described embodiment, and thus the description thereof is omitted. The processing of the flowchart of FIG. 22 described later is realized by the CPU 21 executing the processing based on the program stored in the memory 22.
Next, using the flowchart illustrated in FIG. 22, an example in which the imaging system including the camera 10 and the client device 20 sets and outputs the pattern and cutout position of the image according to the condition of the attention area of the captured image Explain. The processing content and processing flow similar to those of the first embodiment and the second embodiment are respectively S106 to S109 and S201 in the flowchart. Further, the processes of S101 to S105, S202, and S203 described in the flowcharts of the first embodiment and the second embodiment may be added to FIG.
The patterns and the like in FIGS. 23 to 28 are the same as in the first embodiment, and thus the description thereof is omitted.

  In S301, the determination unit 103 determines whether there are five or more attention areas. If the determination unit 103 determines that five or more attention areas do not exist (No in S301), the processing proceeds to S109. If five or more attention areas exist (Yes in S301), the determination unit 103 transitions to S302. In S109, in the case where the number of attention areas is four at less than five as shown in FIG. 23, the pattern setting unit 104 selects pattern 3 as shown in FIG. 24, and division is performed according to the correspondence of areas 1001 to 1004. Do. Further, although the number of attention areas is five, it may be a number that exceeds the number of divisions that can be divided by the pattern setting unit 104. In step S302, the determination unit 103 determines whether a priority area is set in the captured image. If no priority area is set in the captured image (No in S302), the determination unit 103 transitions to S303, and if a priority area is set in the captured image (Yes in S302), the process proceeds to S306. Transition. FIG. 25 is an example in the case where there are five regions of interest, and the determination unit 103 determines the positional relationship of the five regions. Since all five regions of interest can be output in pattern 2, the pattern setting unit 104 selects pattern 2 as shown in FIG. 26 in S303. At this time, the cutting position setting unit 105 sets the cutting position at a position where each of the target areas is not divided. As a result, the image cutout unit 106 cuts out the area 1101 and the area 1102 at positions where the respective target areas are not divided. Also, the distortion correction processing unit 107 performs distortion correction processing. In S304, the determination unit 103 determines whether or not the subject which is the region of interest has entered the region 1103 located at the center as shown in FIG. When the determination unit 103 intrudes into the area where the subject is located at the center (Yes in S304), the process transitions to S305, and when the object does not enter the area where the subject is located in the center (No in S304), the flowchart of FIG. End the processing of. In S305, the pattern setting unit 104 may temporarily select an omnidirectional image as an area 1104 that can include all the areas 1101 to 1103 of FIG. 27 as shown in FIG. When the subject disappears from the area 1103, the pattern setting unit 104 resets pattern 2 as shown in FIG.

In step S306, when the detection area is set in the captured image, the determination unit 103 determines whether a detection target exists in the set detection area. If there is no detection target in the detection area preset in the captured image (No in S306), the determination unit 103 transitions to S307, and detects in the detection area preset in the captured image. If the target exists (Yes in S306), the process transitions to S308.
FIG. 29 shows an example in which the priority area is set and the detection area is not set. The area 1205 of FIG. 29 is set as the priority area. The priority area is an area in which the partial image corresponding to the priority area is cut out and output regardless of whether the area is detected as the target area. However, the priority area can be set to any area in the image. Also, a plurality of priority areas may be set. FIG. 31 further shows an example of the detection area 1206 set in the image. The detection area here is an area for detecting a specific event such as a person intruding into the area or disappearance from the area. FIG. 32 shows that an intrusion of a person is detected in the detection area 1206.
In S308, the pattern setting unit 104 selects the priority area 1205 (1202) and the detection area 1206 (1207) shown in FIG. 32, and further selects an area of higher attention from the areas of interest extracted in step 102. select. Then, the pattern setting unit 104 sets pattern 3 so as to be the arrangement of the image of FIG. At this time, areas 1201 and 1203 with a large number of people are selected as areas of higher interest among the areas of interest. Then, the cut position setting unit 105 sets a cut position at which partial images of the priority area 1205, the detection area 1206, and the areas 1201 and 1203 are cut out. As a result, the image cutout unit 106 cuts out partial images from the regions 1201, 1203, 1205, and 1206. In this process, when the priority area is set in the omnidirectional image, the partial image corresponding to the priority area is prioritized regardless of whether the image cutout unit 106 detects the priority area as a target area. It is an example of processing to cut into. Further, in this process, when the detection area is set in the omnidirectional image, the image cutting unit 106 is to detect the detection area as a focus area having a lower level of attention than the other focus areas. Is also an example of processing for preferentially cutting out a partial image corresponding to a detection area. The distortion correction processing unit 107 performs distortion correction processing on the partially cut partial images.
In S307, when the detection area is not set, the priority area 1205 shown in FIG. 29 is selected, and further, the areas 1201, 1203, and 1204 with higher degree of attention are selected from the attention areas extracted in step S102. Do. Then, the pattern setting unit 104 sets pattern 3 so as to be arranged as shown in FIG. In FIG. 29, since the detection area is not set, the attention area of FIG. 29 present at the same position as the area 1207 of FIG. 32 is not selected, and the attention area 1204 is selected instead. The reason is that the number of people in the region of interest 1204 has a higher degree of attention than the number of people in the region of interest in FIG. 29 located at the same position as the region 1207 in FIG. Although there is an attention area on the left side of FIG. 29 in which the same number of people as the attention area 1204 exists, in the present embodiment, “Women” is given priority as the attribute of the person, and the attention area 1204 is It will be selected.
In addition, the cutting position setting unit 105 sets a cutting position so that partial images corresponding to the selected areas 1201, 1202 (1205), 1203 and 1204 are cut out.

  In step S309, the determination unit 103 detects two regions of interest, and determines whether the positions of the two detected regions of interest are close to each other. The determination unit 103 determines that two regions of interest are in proximity within one semicircle (excluding the central region) of the circular captured image of the fisheye camera before distortion correction. . If the determination unit 103 determines that the two regions of interest are in proximity as shown in FIG. 34, the procedure proceeds to step S106. That is, the pattern 1 is selected, and a semicircle excluding the central region is set as a cutting position. The image after distortion correction becomes a single panoramic image as shown in FIG. On the other hand, as shown in FIG. 12, when the determination unit 103 determines that the two attention areas are not close to each other, the process proceeds to step S108. As a result, a double panoramic image as shown in FIG. 13 is displayed.

As described above, according to the processing of this embodiment, in addition to the effects of the first and second embodiments, it is possible to provide an image suitable for monitoring in consideration of a specific region, an object, and a moving object.
The selection of the patterns described in the first to third embodiments can be appropriately combined.
<Other Embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium. And, it is also possible to realize the processing in which one or more processors in the computer of the system or apparatus read and execute the program. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

As mentioned above, although an example of an embodiment of the present invention was explained in full detail, the present invention is not limited to such a specific embodiment.
For example, in the embodiment described above, the camera described as the imaging device can be applied to a digital still camera or a digital video camera.
Further, part or all of the above-described software configuration may be implemented as a hardware configuration in an apparatus.
Further, as a hardware configuration, a GPU (Graphics Processing Unit) may be used instead of the CPU.

  As described above, according to each of the above-described embodiments, it is possible to output an image that has been subjected to appropriate clipping processing and distortion correction processing in real time according to an imaging scene or the like without being set by the user.

10 Camera 11 Imaging Unit 12 CPU

Claims (14)

Setting means for setting a pattern for outputting a partial image obtained from the omnidirectional image, based on the number and position of a region of interest obtained from the omnidirectional image which is a captured image captured using a fisheye lens;
A cutting unit that cuts out a partial image including the region of interest from the omnidirectional image based on the pattern set by the setting unit;
Correction means for performing distortion correction processing on the partial image cut off by the cutting means;
An output unit that outputs a partial image subjected to distortion correction processing by the correction unit according to the pattern set by the setting unit;
An information processing apparatus having   The information processing apparatus according to claim 1, wherein the setting unit further sets the pattern based on an imaging direction of an imaging unit having the fisheye lens.   When there are two areas of interest obtained from the omnidirectional image, the setting means cuts out one partial image including the two areas of interest if the positions of the two areas of interest are close, The information processing apparatus according to claim 1, wherein a pattern for cutting out two partial images each including one region of interest is set.   When there are three attention areas obtained from the omnidirectional image and one attention area thereof is located at the center of the captured image, the setting unit excludes the attention area located at the center of the captured image. The information processing apparatus according to claim 1, wherein a pattern for cutting out a partial image including a region of interest is set.   When there are three regions of interest obtained from the omnidirectional image, the setting unit arranges partial images corresponding to the three regions of interest cut out from the captured image and the captured image obtained through the fisheye lens The information processing apparatus according to claim 1, wherein a pattern for setting is set.   The information processing apparatus according to claim 1, wherein when the attention area obtained from the omnidirectional image is five or more, the setting unit sets a pattern for outputting the omnidirectional image without cutting out a partial image.   The information according to claim 1, wherein the setting unit sets a cutout position at a position where each of the regions of interest is not divided and outputs two partial images when the number of regions of interest detected from the captured image is five or more. Processing unit.   When the priority area is set in the omnidirectional image, the cutting unit preferentially cuts the partial image corresponding to the priority area regardless of whether the priority area is detected as the target area. An information processing apparatus according to claim 1.   When the detection area is set in the omnidirectional image, the cutting unit may detect the detection area even if it is detected as an attention area whose attention degree is lower than that of other attention areas. The information processing apparatus according to claim 1, wherein the partial image corresponding to is preferentially cut out.   The information processing apparatus according to any one of claims 1 to 9, wherein the information processing apparatus is an imaging apparatus.   The information processing apparatus according to any one of claims 1 to 9, wherein the information processing apparatus is a client apparatus capable of communicating with an imaging apparatus. Setting means for setting a pattern for outputting a partial image obtained from the omnidirectional image, based on the number and position of a region of interest obtained from the omnidirectional image which is a captured image captured using a fisheye lens;
A cutting unit that cuts out an image of a portion including the attention area from the omnidirectional image based on the pattern set by the setting unit;
Correction means for performing distortion correction processing on the partial image cut off by the cutting means;
Output means for outputting an image subjected to distortion correction processing by the correction means in accordance with the pattern set by the setting means;
With a system. An information processing method executed by the information processing apparatus;
A setting step of setting a pattern for outputting a partial image obtained from the omnidirectional image, based on the number and position of a region of interest obtained from the omnidirectional image which is a captured image captured using a fisheye lens;
Cutting out a partial image including the region of interest from the omnidirectional image based on the pattern set in the setting step;
A correction step of performing distortion correction processing on the partial image cut off in the cutting step;
An output step of outputting the partial image subjected to the distortion correction processing in the correction step according to the pattern set in the setting step;
Information processing method including:   A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 9.

Images