JP7150462B2 - Information processing device, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, an information processing method, and a program.

In recent years, with the diversification of types of network cameras, an omnidirectional network camera (hereinafter referred to as an "omnidirectional camera") capable of looking around 360 degrees without blind spots has become popular. Such an omnidirectional camera is installed, for example, on the ceiling of a facility, and used as a monitoring camera for monitoring subjects such as people and vehicles passing under the omnidirectional camera.
In Patent Document 1, a portion in which a moving object is concentrated is preferentially extracted from an omnidirectional image captured by an omnidirectional camera installed on the ceiling, and an area including the moving object is cut out at an aspect ratio for moving image display. points are disclosed.

JP 2014-222825 A

By the way, in an omnidirectional image captured directly downward from an omnidirectional camera installed on the ceiling, the closer the subject is to the center of the image, the larger the subject is imaged. It is characterized by increasing the amount of information that can be obtained from an angle. Moreover, such an omnidirectional image is characterized in that the pixel density changes according to the distance from the center of the image.
However, in the technique described in Patent Literature 1, display priority is given without considering the position of the moving object in the image. Therefore, when displaying the clipped image, it may not be possible to obtain angular information about the moving object, or the moving object may not be confirmed due to poor resolution.
Accordingly, an object of the present invention is to appropriately give display priority when displaying an image of an attention area in an omnidirectional image.

In order to solve the above problems, one aspect of an information processing apparatus according to the present invention includes: first acquisition means for acquiring position information indicating the position of a region of interest in an omnidirectional image; and pixel density information of the omnidirectional image. , the position information acquired by the first acquisition means, and the pixel density information acquired by the second acquisition means, the display priority of the attention area and setting means for setting a distance from the center of the omnidirectional image within a predetermined range, the setting means determines that the attention area is within a predetermined range when the attention area is within a predetermined range. The priority of the display is set higher than the case where it exists outside, and the predetermined area is set closer to the center of the omnidirectional image as the moving speed of the moving object included in the attention area is faster .

ADVANTAGE OF THE INVENTION According to this invention, the priority of a display at the time of displaying the image of the attention area in an omnidirectional image can be assign|provided appropriately.

1 is a configuration example of a network camera system; It is a block diagram which shows the structural example of an imaging device. (A) is an example of an omnidirectional image. (B) is an example of four-screen display of an omnidirectional image. (A) is an example of an omnidirectional image of a vehicle. (B) is an example of an omnidirectional image of a person. 8 is a flowchart showing priority setting processing; 4 is a flowchart showing moving object detection processing; It is an example of calculation of the distance from the center of the moving object. It is an example of a display target range. (A) is an example of a “high” priority region when the central pixel density is sparse. (B) is an example of a “high” priority area when the center pixel density is high. (A) This is an example of position determination with a priority of “high”. (B) is an example of pixel density determination with priority "high". (A) This is an example of position determination with a priority of "middle". (B) is an example of pixel density determination with priority "medium". This is an example of position determination not to be displayed. 4 is a flowchart showing display control processing; It is a display example of attention areas to which display priority is assigned. 9 is a flowchart showing display area change processing; It is a modified example of how to use the display screen. It is a flow chart which shows the modification of a priority setting process.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings.
The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to the embodiment of

FIG. 1 is a block diagram showing a configuration example of a network camera system 1000 according to this embodiment.
A network camera system 1000 according to the present embodiment is a monitoring system that provides a user who monitors a monitored area with an image of the monitored area. This network camera system 1000 detects an attention area from an image of a monitored area, assigns display priority to the detected attention area, and presents the image of the attention area to the user according to the assigned priority. can be done. Here, the attention area can be an area including a moving object such as a human body or a vehicle body.

A network camera system 1000 includes an imaging device 100 and a client device 200 . The imaging device 100 and the client device 200 are connected by a network 300 so as to be able to communicate with each other.
The network 300 is composed of, for example, a plurality of routers, switches, cables, etc. conforming to the Ethernet (registered trademark) communication standard. Note that the network 300 may have any communication standard, scale, or configuration as long as the network 300 can communicate between the camera 100 and the client device 200 . The network 300 may be implemented by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or a combination thereof.

The imaging device 100 is a network camera (hereinafter simply referred to as “camera”) that captures an image of a predetermined imaging range, and can deliver a captured image or a part thereof to the client device 200 via the network 300 . In this embodiment, the camera 100 is an omnidirectional camera that has a fisheye lens and can acquire an omnidirectional image (fisheye image) as a captured image. The camera 100 is installed, for example, on the ceiling of a facility, and can capture an image of an object such as a person or vehicle passing under the camera 100 . Note that the camera 100 may be a camera that captures one image (still image), or may be a camera that captures a video including one or more images.
The client device 200 can be configured by a terminal device such as a personal computer (PC), a smart phone, or a tablet PC. The client device 200 is a display device that presents images distributed from the camera 100 to the user.

Imaging device 100 includes CPU 111 , ROM 112 , RAM 113 , imaging unit 114 , and communication I/F 115 .
The CPU 111 comprehensively controls operations in the imaging apparatus 100 . The ROM 112 is a non-volatile memory that stores control programs and the like necessary for the CPU 111 to execute processing. Note that the program may be stored in an external memory (not shown) or a removable storage medium. A RAM 113 functions as a main memory, a work area, and the like for the CPU 111 . The CPU 111 loads necessary programs and the like from the ROM 112 to the RAM 113 when executing processing, and executes various functional operations by executing the programs and the like.
The imaging unit 114 includes a lens unit and an imaging device that constitute an imaging optical system. Communication I/F 115 is an interface for communicating with an external device.

The client device 200 includes a CPU 211 , a ROM 212 , a RAM 213 , a display section 214 , an operation section 215 and a communication I/F 216 . The CPU 211 , ROM 212 , RAM 213 and communication I/F 216 are the same as the CPU 111 , ROM 112 , RAM 113 and communication I/F 115 of the imaging device 100 .
A display unit 214 includes a monitor such as a liquid crystal display (LCD) and displays images delivered from the camera 100 . An operation unit 215 includes a pointing device such as a keyboard and a mouse, and can accept instruction input from the user.

FIG. 2 is a block diagram showing a configuration example of the camera 100. As shown in FIG.
The camera 100 includes an imaging unit 101 , an image processing unit 102 , an exposure control unit 103 , an optical control unit 104 , a motion vector detection unit 105 , a display information processing unit 106 and an image output unit 107 .
The imaging unit 101 includes a lens group 101a, an optical filter 101b, an imaging element 101c, a gain control amplifier circuit (AGC) 101d, and an A/D converter 101e. The imaging unit 101 corresponds to the imaging unit 114 in FIG.
The lens group 101a is an imaging optical system for condensing incident light from a subject onto the imaging element 101c. The lens group 101a includes a fisheye lens. Also, the lens group 101a may include a focus lens for focusing on a subject, a zoom lens for adjusting the angle of view, and the like.

Light entering the camera 100 through the lens group 101a passes through the optical filter 101b and enters the imaging element 101c. The optical filter 101b can be, for example, an infrared cut filter (IRCF) or the like.
The imaging element 101c includes a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), or the like. The imaging device 101c converts the subject image formed via the lens group 101a and the optical filter 101b into an electrical signal (analog signal) and outputs the electrical signal. The AGC 101d performs amplification processing and the like on the electrical signal output from the image sensor 101c, and automatically performs gain control of the camera 100. FIG. The A/D converter 101e converts the analog signal amplified by the AGC 101d into a digital signal.

The image processing unit 102 performs predetermined processing on the digital signal output from the imaging unit 101, and outputs a luminance signal and a color signal for each pixel. The image processing unit 102 can also generate parameters for camera control. Here, each parameter for camera control includes parameters used in aperture control, focus control, white balance control for color adjustment, and the like.
Based on the brightness information output from the image processing unit 102, the exposure control unit 103 controls the diaphragm and the AGC 101d so as to adjust the captured image to desired brightness. The optical control unit 104 performs a focusing operation. In the focusing operation, high-frequency components are extracted from the image signal generated by the image processing unit 102, and the value of the high-frequency components is used as focus information (focus evaluation value). It is an action to control. The optical control unit 104 can also insert and remove the optical filter 101b.

A motion vector detection unit 105 detects a moving object from the captured image (omnidirectional image) output from the image processing unit 102 . Specifically, the motion vector detection unit 105 detects a moving object by taking the difference between the current frame and the previous frame. Then, the motion vector detection unit 105 acquires the motion vector amount of the moving object and calculates the moving direction and moving speed of the moving object. The motion vector detection unit 105 can also calculate the size of the detected moving object. Here, the moving body includes a human body and a vehicle body as described above. Furthermore, the motion vector detection unit 105 can detect a region including the detected moving object as a region of interest, and acquire position information indicating the position of the region of interest in the omnidirectional image.

The display information processing unit 106 assigns display priority to the attention area including the moving object detected by the motion vector detection unit 105 . At this time, when the user designates an area to be cut out and displayed from the omnidirectional image, the display information processing unit 106 sets the display priority of the attention area in consideration of the display priority of the designated area. can also
The image output unit 107 displays an image to be displayed on the client device 200 based on the omnidirectional image generated by the image processing unit 102, the information on the attention area and the designated area, and the priority set by the display information processing unit 106. to output Specifically, the image output unit 107 cuts out the image of the attention area from the omnidirectional image, and distributes the cut-out image to the client device 200 in order of display priority, thereby displaying the image on the client device 200. control.

Some or all of the functions of the constituent elements of the camera 100 shown in FIG. 2 can be implemented by the CPU 111 of the camera 100 executing a program. However, at least some of the elements of the camera 100 shown in FIG. 2 may operate as dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 111 .

The client device 200 can display images distributed from the camera 100 . In this embodiment, the client device 200 divides one screen into four display areas and performs four-screen display in which four images are displayed in each display area. In this four-screen display, up to four images of attention areas cut out from an omnidirectional image or images of specified areas specified by the user can be displayed.

FIG. 3A is an example of an omnidirectional image 400 captured by the camera 100. FIG. A human body 601 , a vehicle body 602 , and a tree 603 are imaged in a circular omnidirectional image 400 . For example, if the user has specified specified areas A1 to A4 for this omnidirectional image 400, and the clipped image of the specified areas A1 to A4 is delivered from the camera 100, the client device 200 will transfer these specified areas A1 to A4. Receive an A4 image and display it on the display screen. An example of the display screen at this time is shown in FIG.

As shown in FIG. 3B, the display screen 500 is divided into four display areas 501-504. , a clipped image of the specified area A2 is displayed. In addition, the clipped image of the specified area A3 is displayed in the third display area 503, and the clipped image of the specified area A4 is displayed in the fourth display area 504. FIG. In this way, when the user designates four designated areas, the display of the four display areas 501 to 504 is a user designated area display that displays a clipped image of the designated area designated by the user.
On the other hand, when the camera 100 detects an attention area including a moving object from the omnidirectional image and the clipped image of the attention area is delivered from the camera 100, the client device 200 displays the image of the attention area received from the camera 100. display on the screen. For example, when the camera 100 detects four attention areas, the display of the four display areas 501 to 504 is an attention area display that displays a clipped image of each attention area.

By the way, as a feature of an omnidirectional image captured by a ceiling-mounted omnidirectional camera, the subject becomes smaller as it moves away from the center of the image, and more angular information is obtained. For example, when the subject is a vehicle, as shown in FIG. 4A, it is easy to obtain license plate information for a vehicle body 602a that is far from the center, and only the vehicle body is visible for a vehicle body 602b that is close to the center, so license plate information is acquired. Can not. Further, when the subject is a person, as shown in FIG. 4B, it is easy to obtain face information of a human body 601a far from the center, and only the head of a human body 601b close to the center is captured. can not get.

When performing four-screen display on the client device 200, the number of display areas is limited to four. Therefore, even if a moving body such as a vehicle body or a human body is present, the user may not be able to check the license plate or face even if the area including the moving body is displayed as an attention area on the display screen.
Another feature of the omnidirectional image is that the pixel density changes according to the distance from the center of the image. Therefore, depending on the position of the attention area in the image, even if the clipped image of the attention area is displayed on the display screen, the moving object may not be confirmed due to the inferior resolution.

Therefore, in the present embodiment, the camera 100 preferentially displays the moving object on the display screen of the client device 200 when it is present at an appropriate position where necessary information such as license plate information and face information can be obtained. Specifically, camera 100 acquires position information indicating the position of a region of interest in an omnidirectional image and pixel density information of the omnidirectional image, and based on the acquired position information and pixel density information, determines the region of interest. set the display priority of More specifically, when the distance from the center of the image of the attention area to the attention area is within a predetermined range and the pixel density at the position where the attention area exists is equal to or greater than a predetermined value, the camera 100 gives priority to the display of the attention area. Set to high.

In this way, the camera 100 has a function of acquiring position information of a region of interest in an omnidirectional image, a function of acquiring pixel density information of the omnidirectional image, and a function of setting display priority for the region of interest. have. In the present embodiment, a case will be described in which the camera 100 operates as an information processing device that sets display priority of attention areas based on position information and pixel density information. However, the client device 200 may operate as the information processing device, or a general PC or other equipment may operate as the information processing device. Moreover, the camera 100 may be configured to include only a part of the functions of the information processing apparatus. In this case, the client device 200 and other devices connected to the camera 100 via the network 300 have the remaining functions of the information processing device.

FIG. 5 is a flowchart showing a priority setting process procedure executed by the camera 100. FIG.
The processing in FIG. 5 is started at the timing when the motion vector detection unit 105 of the camera 100 acquires the omnidirectional image from the image processing unit 102 . However, the start timing of the processing in FIG. 5 is not limited to the above timing. The camera 100 can realize each process shown in FIG. 5 by having the CPU 111 read and execute necessary programs. Hereafter, the letter S shall mean a step in the flow chart.

First, in S1, the motion vector detection unit 105 performs moving object detection processing for detecting a moving object from an omnidirectional image.
FIG. 6 is a flow chart showing the moving object detection process executed in S1.
In S11, the motion vector detection unit 105 acquires a background difference image, which is the difference between the current frame and the previous frame, and proceeds to S12. In S12, the motion vector detection unit 105 determines the presence or absence of a moving object based on the background difference image. If it is determined that the moving object does not exist, it determines that the attention area has not been detected, and the processing of FIG. 6 is performed as it is. exit. In this case, since there is no region of interest to which priority is to be assigned, the processing in FIG. 5 is terminated as it is. On the other hand, when the motion vector detection unit 105 determines that there is a moving object, the process proceeds to S13.

In S13, the motion vector detection unit 105 detects the shape of the moving object. At this time, the motion vector detection unit 105 may perform recognition processing to determine whether the moving object is a human body or a vehicle body from the shape of the detected moving object. Also, the motion vector detection unit 105 may calculate the size of the moving object from the shape of the detected moving object.
In S14, the motion vector detection unit 105 acquires the motion vector of the moving object. In S15, the motion vector detection unit 105 calculates the moving direction of the moving object from the motion vector acquired in S14. In S16, the motion vector detection unit 105 calculates the moving speed of the moving object from the motion vector acquired in S14.

Returning to FIG. 5, in S2, the display information processing unit 106 detects the distance L from the center of the image of the moving object 610 detected in S1, as shown in FIG. The distance L from the image center of the moving object 610 is the distance from the center of the omnidirectional image of the attention area including the moving object 610 . The position of the moving object 610 (the position of the region of interest) can be the center or the center of gravity of the moving object 610 . If the moving object 610 is the vehicle body and the recognition target is the license plate, the position of the moving object 610 may be the center of the license plate. Further, when the moving body 610 is a human body and the recognition target is a person's face, the position of the moving body 610 may be the center of the face.

In S3, the display information processing unit 106 determines whether or not the distance L detected in S2 is within a predetermined range. Specifically, the display information processing unit 106 determines whether or not L2≦L≦L1. That is, the display information processing unit 106 determines whether or not the attention area including the moving object is positioned within a predetermined area inside the circle C1 with the radius L1 and outside the circle C2 with the radius L2 shown in FIG. . Here, the circles C1 and C2 are concentric circles around the center of the omnidirectional image. Then, if the attention area is located within the predetermined area, it is determined that necessary information can be acquired from the moving object included in the attention area, and the process proceeds to S4. , the amount of information that can be acquired from the moving object included in the attention area is determined to be small, and the process proceeds to S7.

In S4, the display information processing unit 106 acquires pixel density information of the position where the attention area exists, and determines whether or not the pixel density is equal to or greater than a predetermined value.
Depending on the characteristics of the lens of the camera 100, the pixel density may become denser as it approaches the periphery of the image, or it may become less dense as it approaches the periphery of the image, but it is applicable to both cases. . FIG. 9A shows a case where the pixel density becomes denser as it approaches the periphery of the image. In this case, the outside of the circle C3 with the radius L3 is the area where the pixel density is equal to or higher than the predetermined value. Also, FIG. 9B shows a case where the pixel density becomes sparse toward the periphery of the image. In this case, the inside of the circle C3 with the radius L3 is the area where the pixel density is equal to or higher than a predetermined value. Become. Therefore, by comparing the distance L from the image center of the attention area to the radius L3 of the circle C3, it is possible to determine whether or not the attention area exists at a position where the pixel density is equal to or higher than a predetermined value.
Note that the radius L3 in FIG. 9A and the radius L3 in FIG. 9B may be the same or different. It is also possible to use resolution number information instead of pixel density information.

If the pixel density at the position where the attention area exists is equal to or greater than a predetermined value, the display information processing unit 106 determines that the position where the attention area exists has good resolution and is suitable for acquiring information on a moving object. Then, the process proceeds to S5 to set the display priority of the attention area including the moving object to "high".
On the other hand, if the pixel density is less than the predetermined value, the display information processing unit 106 determines that the necessary information regarding the moving object can be obtained from the image of the attention area, but the resolution is somewhat inferior, and proceeds to S6. , the priority of the display of this attention area is set to "medium".
In S<b>7 , the display information processing unit 106 sets the display priority so that the attention area for which the priority is set is excluded from the display (no priority is given).

That is, as shown in FIG. 10A, a moving object (human body) 601 exists within a predetermined region inside the circle C1 and outside the circle C2, and as shown in FIG. If it exists outside C3, the display priority of the attention area including this moving object 601 is high. In this way, when the distance from the center of the image of the attention area including the moving object 601 is within a predetermined range and the pixel density at the position where the attention area exists is equal to or greater than a predetermined value, the display priority of the attention area is is high.
Note that in FIGS. 10A and 10B, the position of the moving object 601 (the position of the attention area) is the center of the face. The same applies to FIGS. 11A, 11B and 12, which will be described later.

On the other hand, as shown in FIG. 11(A), a moving body (human body) 601 exists within a predetermined area inside the circle C1 and outside the circle C2, but as shown in FIG. 11(B) In addition, when the moving object 601 exists inside the circle C3, the display priority is medium. That is, when the distance from the center of the image of the attention area including the moving object 601 is within a predetermined range and the pixel density of the position where the attention area exists is less than a predetermined value, the display priority of the attention area is medium. becomes.

Then, as shown in FIG. 12, when a moving body (human body) 601 is inside the circle C2, that is, when the attention area including the moving body 601 exists outside the predetermined area, the attention area including the moving body 601 is displayed. Excluded.
Note that, in the present embodiment, a case has been described in which when an attention area exists outside a predetermined area, the attention area is excluded from the display target. However, when the attention area is outside the predetermined area and the pixel density at the position of the attention area is equal to or greater than a predetermined value, the display priority is set to, for example, "low", although the priority is low. can be set to

FIG. 13 is a flow chart showing a display control processing procedure executed by the camera 100. As shown in FIG.
The process of FIG. 13 is a process of performing display control for displaying the image of the attention area on the client device 200 based on the priority of the display of the attention area, and is started after the priority setting process of FIG. 5 ends. The camera 100 can realize each process shown in FIG. 13 by having the CPU 111 read out and execute necessary programs.
First, in S<b>21 , the image output unit 107 determines whether or not the attention area is preferentially displayed according to the user's instruction. The user can operate the operation unit 215 of the client device 200 to give an instruction in advance as to whether or not to preferentially display the attention area detected by the camera 100 on the display unit 214 . If the image output unit 107 determines that the attention area is preferentially displayed, the process proceeds to S22, and if it determines that the attention area is not preferentially displayed, the process of FIG. 13 is terminated. do.

In S<b>22 , the image output unit 107 determines whether or not there is a display area used for displaying an image other than the attention area on the display screen of the client device 200 . In this embodiment, the image output unit 107 determines whether or not there is a display area used for displaying the specified area specified by the user on the display screen of the client device 200 . Then, when there is no display area used for displaying the specified area, that is, when the entire display area in which an image can be displayed on the display screen can be used for displaying the attention area to which priority is assigned. , S23. On the other hand, if there is a display area used for displaying the specified area, it is determined that the usage of the display area needs to be changed, and the process proceeds to S25.

In S<b>23 , the image output unit 107 compares the number of prioritized regions of interest with the number of display regions that can be used for displaying the regions of interest on the display screen. Then, when the number of attention areas exceeds the number of usable display areas, the process proceeds to S25, and when the number of attention areas is equal to or less than the number of usable display areas, the process proceeds to S24. In S24, the image output unit 107 can display all the attention areas to which the priority is given on the display screen, and therefore outputs all the images of the attention areas to the client device 200 in order from the attention area with the highest priority. . In this manner, the image output unit 107 performs display control for displaying the image of the attention area on the client device 200 . A display example on the client device 200 in this case will be described with reference to FIGS. 14A and 14B.

As shown in FIG. 14A, when a human body 601 and a vehicle body 602 are imaged as moving bodies in an omnidirectional image 400, the camera 100 divides an area including the human body 601 and an area including the vehicle body 602 into regions of interest. Detect as In addition, the camera 100 sets display priority for these attention areas. At this time, it is assumed that the camera 100 sets the display priority of the attention area including the vehicle body 602 to high and the display priority of the attention area including the human body 601 to medium. In this case, the camera 100 outputs the image of the attention area including the vehicle body 602 and the image of the attention area including the human body 601 to the client device 200 in this order.
At this time, all four display areas 501 to 504 that can be displayed on the display screen 500 of the client device 200 can be used for displaying the image of the attention area. Therefore, as shown in FIG. 14B, for example, the client device 200 displays the image of the attention area including the vehicle body 602 in the first display area 501, and displays the image of the attention area including the human body 601 in the second display area. Displayed in area 502 . It should be noted that which image of which priority is to be displayed in which display area may be set in advance, or may be specified by the user.

In S25 of FIG. 13, display area change processing for changing the usage of the display area is executed.
FIG. 15 is a flow chart showing the display area change processing procedure executed in S25.
First, in S251, the image output unit 107 determines the number of images of attention areas to be displayed on the client device 200, that is, the number of display areas required for displaying attention areas (required number of display areas). Here, the required number of display areas for the attention area may be a number designated by the user or a predetermined number.

Next, in S252, the image output unit 107 compares the required number of display areas for the attention area determined in S251 with the number of unused display areas that can be used for displaying the image of the attention area. Then, when the number of usable display areas is insufficient, the display area used for displaying the image other than the attention area, for example, the image of the specified area specified by the user, is used for displaying the attention area. Select the display area to use.
Specifically, as shown in FIG. 16, when user-specified area display is performed in four display areas 501 to 504 and the number of required display areas for the attention area is determined to be two, four display areas are displayed. Two of 501 to 504 are replaced with display areas for displaying the attention area. FIG. 16 shows a case where the two display areas 503 and 504 are selected as display areas for displaying the attention area, and attention area display is performed, but the replacement display area is not limited to the above. For example, priority is given to display areas, a display area displaying an image without a moving object is replaced, a display area displaying an image with the least change is replaced, and so on. be able to.

In S253, the image output unit 107 selects a specific object to be preferentially displayed in order to set the priority in more detail with respect to the attention areas for which the same priority was set in the priority setting process of FIG. Determine the type (priority target object). For example, if it is desired to check the license plate preferentially, the vehicle body is the priority target object. The priority target object may be specified by the user, or may be a priority target object for a moving object that has a high presence rate in the captured image, or a priority target object for a moving object that stays for a long time in the captured image.

In S254, the image output unit 107 determines whether or not the attention area is included in the specified area specified by the user. Here, if there are a plurality of attention areas, it is determined whether or not all attention areas are included in the specified area. Then, if the attention area is included in the designated area, it is determined that there is no need to display the image of the attention area again, and the process of FIG. 15 is terminated. That is, the image output unit 107 does not output the image of the attention area to the client device 200 .

On the other hand, when the attention area is not included in the designated area, it is necessary to display the image of the attention area on the display screen. Therefore, in that case, the processes after S255 are executed.
In S255, the image output unit 107 determines whether or not the attention area includes a priority target object. When the image output unit 107 determines that the priority target object is included in the attention area, the process proceeds to S256. .
In S256, the image output unit 107 determines the priority of the attention area determined to include the priority target object, and when the priority is high, the process proceeds to S257. On the other hand, if the priority is not high, such as when the priority is medium, the process proceeds to S258.

In S<b>257 , the image output unit 107 outputs to the client device 200 an image of the attention area that includes the priority target object and is determined to have a high priority. The image of the attention area is displayed in the display area for displaying the attention area on the display screen.
In S<b>258 , the image output unit 107 determines whether or not there is an empty display area for displaying the attention area on the display screen of the client device 200 . Then, if the image output unit 107 determines that there is space in the display area, the process proceeds to S257, and if it determines that there is no space, the process of FIG. 15 is terminated. In other words, when the priority target object is not included, or when the priority of the attention area is medium, the attention area is displayed only when there is a free space in the display area.

As described above, the camera 100 according to this embodiment detects a moving object from an omnidirectional image, and detects an area including the moving object as an attention area. Then, the camera 100 acquires position information indicating the position of the attention area in the omnidirectional image and pixel density information of the omnidirectional image, and determines display of the attention area based on the acquired position information and pixel density information. Set priority. Then, the camera 100 performs display control to display the image of the attention area on the client device 200, which is a display device, in order of the set display priority.
Specifically, the camera 100 gives priority to display over a case where the attention area exists outside the predetermined area when the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image. Set high. In addition, the camera 100 sets a higher display priority when the attention area exists at a position where the pixel density of the omnidirectional image is equal to or higher than a predetermined value than when it exists at a position where the pixel density is less than the predetermined value.

As described above, in the present embodiment, in an omnidirectional camera that captures an omnidirectional image, the position of the attention area including the moving object and the pixel density of the position of the attention area including the moving object are considered. display priority can be set. As a result, when the attention area exists at a position where necessary information such as license plate information and face information can be appropriately acquired, the image of the attention area can be preferentially displayed on the display device. Therefore, it is possible to prevent the image of the attention area from being displayed on the display device even though the necessary information cannot be acquired.

In addition, the camera 100 determines whether or not there is a specific object to be preferentially displayed in the attention area, and sets a higher display priority when the specific object exists than when the specific object does not exist. can also
As a result, when multiple types of moving objects such as a human body and a vehicle body are detected from the omnidirectional image, high priority can be given to the attention area including the moving objects to be preferentially displayed. Therefore, it is possible to appropriately present the information that the user preferentially wants to check.

Furthermore, when there is a specified area in the omnidirectional image specified by the user, the camera 100 can set the display priority of the specified area and the attention area. That is, when there is a designated area designated by the user, the image of the attention area is displayed with the designated area as the top priority, or the display area for displaying the designated area on the display screen is changed to the display area for displaying the attention area. can be replaced with
The camera 100 determines the number of images of the attention area to be displayed on the display screen of the client device 200, and compares it with the number of unused display areas available for displaying the attention area on the display screen. Then, when the number of usable display areas is insufficient, the image of the attention area is displayed from the display area used for displaying the image other than the attention area among the plurality of display areas on the display screen. Select the display area to use for Then, display control is performed to display the image of the attention area in the selected display area.
As a result, the image of the attention area can be appropriately displayed on the display screen within the determined number of display areas.

(Modification)
In the above embodiment, the camera 100 detects a region of interest without considering the distortion rate of the image, and sets the display priority of the region of interest. priority may be set. When the distortion of the image is significant, it is difficult to obtain necessary information such as license plate information and face information from the image of the attention area. Therefore, the camera 100 may acquire the distortion rate of the image, and if the distortion rate of the image is equal to or greater than the threshold, the moving object detection process may not be performed. In other words, when the distortion rate is less than the threshold, moving object detection processing may be performed to set the display priority of the attention area.
Further, in the above embodiment, the case where the predetermined areas (thresholds L1 and L2) used for determining the position of the attention area and the predetermined value (threshold L3) used for determining the pixel density of the attention area are fixed will be described. did. However, the threshold may be variable according to the moving speed of the moving object.
These will be described in detail below.

FIG. 17 is a flow chart showing a modification of the priority setting process executed by the camera 100. FIG. The start timing of the processing in FIG. 17 is the same as the start timing of the processing in FIG. The camera 100 can realize each process shown in FIG. 17 by having the CPU 111 read out and execute necessary programs.
In S31, the camera 100 acquires the distortion rate of the image and determines whether or not the distortion rate is equal to or greater than a threshold. Then, if the distortion rate is equal to or greater than the threshold, the process of FIG. 17 is terminated as it is so as not to set the display priority so as not to be displayed, and if the distortion rate is less than the threshold, S32 transition to In S32, the motion vector detection unit 105 performs moving object detection processing for detecting a moving object from the omnidirectional image. The moving object detection process in S32 is the same as the process in S1 in FIG. 5 (the process in FIG. 6).

In S33, the display information processing unit 106 acquires the moving speed of the moving object detected in S32 and the gain on the image, and changes the predetermined area used for determining the position of the attention area according to the acquired information. That is, the display information processing unit 106 changes the thresholds L1 and L2 of the distance from the center of the image to the moving object.
Also, in S34, the display information processing unit 106 changes the predetermined value used for determining the pixel density at the position of the attention area according to information such as the moving speed of the moving object and the gain on the image detected in S32. That is, the display information processing unit 106 changes the pixel density threshold L3.

For example, in S33 and S34, the display information processing unit 106 changes the predetermined area used for determining the position of the attention area to a position closer to the center of the omnidirectional image as the moving speed of the moving object increases. Further, the display information processing unit 106 sets the predetermined value used for determining the pixel density at the position of the attention area to a larger value as the moving speed of the moving object increases. As a result, the faster the moving speed of the moving object is, the higher the priority of display is when the moving object is present at a position closer to the center of the image, or when the moving object is present at a position with a higher pixel density. .
If the moving speed of the moving object is fast, it is assumed that the image of the moving object will be blurred. Therefore, by changing the threshold value as described above, when the moving speed of the moving object is fast, when the license plate or the face is larger and exists in a position where it can be imaged more clearly, it is preferentially displayed on the display screen. can be displayed on

In addition, when the gain of the attention area is equal to or greater than a first threshold value set in advance, the display information processing unit 106 determines that the predetermined area used for determining the position of the attention area is larger than the case where the gain is less than the first threshold value. to a position closer to the center of the omnidirectional image. Further, when the gain is equal to or greater than the first threshold, the display information processing unit 106 sets the predetermined value used for determining the pixel density at the position of the attention area to a larger value than when the gain is less than the first threshold. set. As a result, when the gain on the image is equal to or greater than the first threshold value, when the moving object exists at a position closer to the center of the image, or when the moving object exists at a position with a higher pixel density, the display will have a higher priority.
It is assumed that the higher the gain, the greater the influence of noise. Therefore, by changing the threshold value as described above, when the gain is equal to or greater than the first threshold value, the moving object is preferentially displayed on the display screen when it is larger and exists in a position where it can be imaged more clearly. can be displayed on
If the gain is equal to or greater than the second threshold, which is even larger than the first threshold, it is considered difficult to confirm the license plate or face. may be omitted.

Note that the above threshold can be changed according to the size of the moving object. For example, the display information processing unit 106 changes the predetermined area used for determining the position of the attention area to a position closer to the center of the omnidirectional image as the size of the moving object increases. Further, the display information processing unit 106 sets the predetermined value used for determining the pixel density at the position of the attention area to a larger value as the size of the moving object increases. As a result, the greater the size of the moving object, the higher the priority of display when the moving object exists at a position closer to the center of the image, or when the moving object exists at a position with a higher pixel density. .
Furthermore, the above threshold may be changed in consideration of the time zone in which the omnidirectional image was captured, or may be changed in consideration of the illumination range and intensity distribution.

In S35, the display information processing unit 106 sets the display priority of the attention area. The processing of this S35 is the same as the processing of S2 to S7 in FIG. However, in S3 and S4, determination using the threshold values set in S33 and S34 is performed.
Note that the display information processing unit 106 may set the display priority in consideration of the moving direction of the moving object. For example, when the moving direction of the moving object is toward the center of the omnidirectional image, the display information processing unit 106 gives higher display priority than when the moving direction of the moving object is moving away from the center of the omnidirectional image. can be set.

If the moving object is a human body, it is assumed that the purpose is to confirm the face. If the direction of movement of the moving object is from the periphery to the center of the image, by setting a high display priority, the rear view of the human body moving away from the center of the image toward the periphery can be displayed. It is possible to prevent the captured image from being preferentially displayed on the display screen. Here, when the moving object is a human body and the moving direction of the human body is the direction away from the center of the omnidirectional image, display priority is given so that the attention area including the human body is excluded from the display target. You can choose not to do it.
Also, if the moving object is a vehicle body, and if the direction of movement of the moving object is the direction from the periphery to the center of the image, the display priority is set high so that the license plate in front of the vehicle body can be displayed preferentially. can be displayed on the display screen.

As described above, by changing the display priority setting conditions according to the movement speed, size, movement direction, and gain on the image of the moving object, when the moving object exists in a position more suitable for display, , can be preferentially displayed on the display screen. Therefore, confirmation of the moving object and acquisition of information are facilitated.

In the above-described embodiment, the case where the method of dividing the display screen of the client device 200 is fixed has been described, but the method of dividing the display screen may be variable. For example, the number of divisions of the display screen may be changed or the shape of the display area may be changed according to the size and shape of the moving object included in the attention area. For example, if the shape of the moving object included in the attention area is horizontally elongated, the display is changed from 4-split display to upper and lower 2-split display. The display may be changed to a left and right split display.
Further, in the above embodiment, the client device 200 has described a case where images are displayed in a plurality of display areas on one display screen, but the image display method is not limited to split display. For example, the client device 200 may have a configuration in which a plurality of display screens (display unit 214) are provided and an image is displayed on each display screen.

Furthermore, in the above embodiment, the camera 100 cuts out the image of the attention area from the omnidirectional image and displays it on the display screen of the client device 200, but the method of displaying the image of the attention area is not limited to the above. . For example, the camera 100 may cut out an image of an attention area from a panoramic image obtained by developing an omnidirectional image in the circumferential direction, and display the image on the display screen of the client device 200 .
Further, in the above embodiment, the case where the attention area is an area including a moving object such as a human body or a vehicle body has been described, but the attention area may be an area including an object of interest, and the object of attention is a moving body. is not limited to

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

DESCRIPTION OF SYMBOLS 100... Network camera 101... Imaging part 102... Image processing part 105... Motion vector detection part 106... Display information processing part 107... Image output part 200... Client apparatus 300... Network 1000... Network camera system

Claims (16)

a first acquisition means for acquiring position information indicating the position of the attention area in the omnidirectional image;
a second acquiring means for acquiring pixel density information of the omnidirectional image;
setting means for setting the display priority of the attention area based on the position information acquired by the first acquisition means and the pixel density information acquired by the second acquisition means ;
The setting means
When the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image, the display priority is set higher than when the attention area exists outside the predetermined area. and setting the predetermined area closer to the center of the omnidirectional image as the moving speed of the moving object included in the attention area is faster . a first acquisition means for acquiring position information indicating the position of the attention area in the omnidirectional image;
a second acquiring means for acquiring pixel density information of the omnidirectional image;
setting means for setting the display priority of the attention area based on the position information acquired by the first acquisition means and the pixel density information acquired by the second acquisition means;
The setting means
When the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image, the display priority is set higher than when the attention area exists outside the predetermined area. and setting the predetermined area closer to the center of the omnidirectional image as the size of the object included in the attention area is larger . a first acquisition means for acquiring position information indicating the position of the attention area in the omnidirectional image;
a second acquiring means for acquiring pixel density information of the omnidirectional image;
setting means for setting the display priority of the attention area based on the position information acquired by the first acquisition means and the pixel density information acquired by the second acquisition means;
The setting means
When the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image, the display priority is set higher than when the attention area exists outside the predetermined area. and
When the gain of the attention area is equal to or greater than the first threshold, the predetermined area is set at a position closer to the center of the omnidirectional image than when the gain is less than the first threshold. information processing equipment. The setting means
3. When the gain of the attention area is equal to or greater than a second threshold larger than the first threshold, the display priority is set such that the attention area is excluded from display. The information processing device according to . a first acquisition means for acquiring position information indicating the position of the attention area in the omnidirectional image;
a second acquiring means for acquiring pixel density information of the omnidirectional image;
setting means for setting the display priority of the attention area based on the position information acquired by the first acquisition means and the pixel density information acquired by the second acquisition means;
The setting means
when the attention area exists at a position where the pixel density of the omnidirectional image is equal to or higher than a predetermined value, the display priority is set higher than when the attention area exists at a position where the pixel density is less than the predetermined value ; An information processing apparatus , wherein the predetermined value is set to a larger value as a moving speed of a moving object included in the attention area is faster . a first acquisition means for acquiring position information indicating the position of the attention area in the omnidirectional image;
a second acquiring means for acquiring pixel density information of the omnidirectional image;
setting means for setting the display priority of the attention area based on the position information acquired by the first acquisition means and the pixel density information acquired by the second acquisition means;
The setting means
when the attention area exists at a position where the pixel density of the omnidirectional image is equal to or higher than a predetermined value, the display priority is set higher than when the attention area exists at a position where the pixel density is less than the predetermined value; An information processing apparatus, wherein the predetermined value is set to a larger value as the size of an object included in the attention area is larger. a first acquisition means for acquiring position information indicating the position of the attention area in the omnidirectional image;
a second acquiring means for acquiring pixel density information of the omnidirectional image;
setting means for setting the display priority of the attention area based on the position information acquired by the first acquisition means and the pixel density information acquired by the second acquisition means;
The setting means
when the attention area exists at a position where the pixel density of the omnidirectional image is equal to or higher than a predetermined value, the display priority is set higher than when the attention area exists at a position where the pixel density is less than the predetermined value; The information processing apparatus, wherein when the gain of the attention area is equal to or greater than a first threshold, the predetermined value is set to a larger value than when the gain is less than the first threshold. The setting means
8. The method according to any one of claims 1 to 7 , wherein when the distortion rate of the attention area is equal to or greater than a threshold, the display priority is set so that the attention area is excluded from display. Information processing equipment. Further comprising detection means for detecting the attention area from the omnidirectional image,
9. The setting unit according to any one of claims 1 to 8 , wherein the setting unit sets a display priority between the specified area in the omnidirectional image specified by the user and the attention area detected by the detection unit. 1. The information processing apparatus according to 1. 10. An information processing apparatus according to claim 9 , wherein said detection means detects an area including a moving body as said attention area. 11. The apparatus according to any one of claims 1 to 10 , further comprising display control means for causing a display device to display the image of the attention area in order of priority of display set by the setting means. Information processing equipment. determining means for determining the number of images of the attention area to be displayed on the display device;
A display area used for displaying an image other than the attention area among a plurality of display areas capable of displaying images on the display device according to the number of images of the attention area determined by the determining means. a selection means for selecting a display area to be used for displaying the image of the attention area from,
12. The information processing apparatus according to claim 11 , wherein said display control means displays an image of said attention area in said display area selected by said selection means. obtaining position information indicating the position of the region of interest in the omnidirectional image;
obtaining pixel density information of the omnidirectional image;
a setting step of setting the display priority of the attention area based on the position information and the pixel density information ;
In the setting step,
When the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image, the display priority is set higher than when the attention area exists outside the predetermined area. and setting the predetermined area closer to the center of the omnidirectional image as the movement speed of the moving object included in the attention area is faster . obtaining position information indicating the position of the region of interest in the omnidirectional image;
obtaining pixel density information of the omnidirectional image;
a setting step of setting the display priority of the attention area based on the position information and the pixel density information;
In the setting step,
When the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image, the display priority is set higher than when the attention area exists outside the predetermined area. and setting the predetermined area closer to the center of the omnidirectional image as the size of the object included in the attention area is larger. obtaining position information indicating the position of the region of interest in the omnidirectional image;
obtaining pixel density information of the omnidirectional image;
a setting step of setting the display priority of the attention area based on the position information and the pixel density information;
In the setting step,
When the attention area exists within a predetermined area within a predetermined distance from the center of the omnidirectional image, the display priority is set higher than when the attention area exists outside the predetermined area. and setting the predetermined region to a position closer to the center of the omnidirectional image when the gain of the attention region is equal to or greater than the first threshold than when the gain is less than the first threshold. Information processing method characterized by: A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 12 .

Images