JP6996538B2 - Image processing equipment, image processing methods, and image processing systems - Google Patents

Description

The present disclosure relates to an image processing apparatus, an image processing method, and an image processing system.

Conventionally, various techniques have been developed for cutting out an area of an object such as a person to be detected from a captured image.

For example, Patent Document 1 describes a technique of detecting a moving body from an image taken by a fisheye lens camera and cutting out a region of a circumscribed quadrangle of each detected moving body. Further, Patent Document 2 describes a technique for extracting a partial region from each frame image based on the position information of the partial region extracted in the immediately preceding frame image and the physical feature amount analyzed from the current frame image. Are listed. Further, Patent Document 3 describes a technique of detecting a moving body having the largest size, existence time, or moving speed among moving bodies extracted from video data, and cutting out a region including the detected moving body.

Japanese Unexamined Patent Publication No. 2001-333422 Japanese Unexamined Patent Publication No. 2004-334587 Japanese Unexamined Patent Publication No. 2014-222825

However, in the techniques described in Patent Documents 1 to 3, the position of the cutout region may be restricted. For example, in the technique described in Patent Document 3, once an object defined as a detection target continues to be located at the same place, the same place continues to be set as a cutout area for a long time.

Therefore, in the present disclosure, a new and improved image processing device, an image processing method, and an image processing system capable of adaptively determining a cutout region according to the length of time that the object to be detected is stopped are available. To propose.

According to the present disclosure, at a first time point of a moving image, a first object is selected as a gaze target from one or more detected objects based on a predetermined criterion, and the first time point is described. A control unit for excluding the first object from the gaze target when it is detected that the first object is stopped for a predetermined time or longer at a second time point after that. , An image processing device is provided.

Further, according to the present disclosure, at the first time point of the moving image, the first object is selected as the gaze target from one or more detected objects based on a predetermined criterion. When it is detected that the first object is stopped for a predetermined time or more at a second time point after the first time point, the first object is excluded from the gaze target. An image processing method is provided.

Further, according to the present disclosure, at the first time point of the moving image, the first object is selected as the gaze target from one or more detected objects based on a predetermined criterion, and the first object is described. A control unit that excludes the first object from the gaze target when it is detected that the first object is stopped for a predetermined time or more at a second time point after the first gaze, and the gaze. A cutout area determination unit that determines a cutout area so as to include a detection position of a target object, and a cutout image generation unit that generates a cutout image by cutting out the cutout area determined by the cutout area determination unit. An image processing system including a storage unit for storing the cut-out image is provided.

As described above, according to the present disclosure, the cutout region can be adaptively determined according to the length of time that the object to be detected is stopped. The effects described herein are not necessarily limited, and may be any of the effects described in the present disclosure.

It is explanatory drawing which showed the structural example of the image processing system by embodiment of this disclosure. It is explanatory drawing which showed an example of the reduced image 32 generated by a camera 10. It is explanatory drawing which showed an example of the plurality of cropping images 50 generated from the frame image 30. It is a functional block diagram which showed the structure of the camera 10 by the same embodiment. It is explanatory drawing which showed the relationship between the frame image 30 and the cropping area 40. It is a functional block diagram which showed the structure of the monitoring terminal 22 by the same embodiment. It is explanatory drawing which showed the display example of the evaluation standard setting screen by the same embodiment. It is a functional block diagram which showed the structure of the area setting part 104 by the same embodiment. It is a flowchart which showed the operation by the same embodiment. It is a flowchart which showed a part of the operation of the cropping image generation processing by the same embodiment. It is a flowchart which showed a part of the operation of the cropping image generation processing by the same embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. In the present specification and the drawings, components having substantially the same functional configuration are designated by the same reference numerals, so that duplicate description will be omitted.

Further, in the present specification and the drawings, a plurality of components having substantially the same functional configuration may be distinguished by adding different alphabets after the same reference numerals. For example, a plurality of configurations having substantially the same functional configuration are distinguished as necessary, such as the video cropping unit 106a and the video cropping unit 106b. However, if it is not necessary to particularly distinguish each of the plurality of components having substantially the same functional configuration, only the same reference numerals are given. For example, when it is not necessary to distinguish between the image cropping unit 106a and the image cropping unit 106b, the image cropping unit 106 is simply referred to as the image cropping unit 106.

In addition, the "mode for carrying out the invention" will be described in accordance with the order of items shown below.
1. 1. Basic configuration of image processing system 2. Detailed description of the embodiment 3. Modification example

<< 1. Basic configuration of image processing system >>
The present disclosure may be implemented in a variety of forms, as described in detail in "2. Detailed Description of Embodiments" as an example. First, the basic configuration of the image processing system according to the present embodiment will be described with reference to FIG.

As shown in FIG. 1, the image processing system according to the present embodiment includes a camera 10, a storage 20, a monitoring terminal 22, and a communication network 24.

<1-1. Camera 10>
The camera 10 is an example of the image processing device in the present disclosure. The camera 10 is a device for taking a moving image of an external environment. The camera 10 may be installed in a place where there is a lot of traffic of people or cars, a place to be monitored, or the like. For example, the camera 10 may be installed on a road, station, airport, commercial building, amusement park, park, parking lot, or restricted area.

Further, the camera 10 can generate another image using the captured frame image, and can transmit the generated other image to another device via the communication network 24 described later. Here, the frame image is, for example, an image having an upper limit resolution that can be captured by the camera 10. As an example, the frame image may be a 4K image.

For example, the camera 10 generates another image with a small amount of data based on the frame image. The reason for this is that since the frame image has a large amount of data, it is not desirable to transmit the frame image itself to another device, for example, because it takes a long time to transmit.

Here, another example of the image generated by the camera 10 is a reduced image which is an image in which the resolution of the frame image is simply lowered, and a cropped image which is an image in which the region to be gazed is cropped. The reduced image may be, for example, a full HD image.

FIG. 2 is an explanatory diagram showing an example of a reduced image (reduced image 32). The reduced image 32 includes all the areas included in the frame image. On the other hand, as shown in FIG. 2, in the reduced image 32, the area to be gazed at, for example, a human face, can be very small, so that it can be difficult to visually recognize. The region 40 shown in FIG. 2 is a region corresponding to a cropping region described later. Normally, the cropping region is set in the frame image, but in FIG. 2, for convenience of explanation, the region corresponding to the cropping region in the reduced image 32 is described as the region 40.

Further, FIG. 3 is an explanatory diagram showing an example of a plurality of cropping images (set 52 of cropping images 50) generated from one frame image. The cropping image 50 has the same resolution as the frame image, but as shown in FIG. 3, each cropping image 50 includes only a part of the frame image. Therefore, the camera 10 according to the present embodiment basically generates one reduced image and one or more cropped images from one frame image. According to this generation example, the user can confirm the whole view taken by the camera 10, and the user can confirm the area to be gazed at with high resolution. Then, the total amount of data can be suppressed as compared with the frame image.

Here, the internal configuration of the camera 10 will be described with reference to FIG. As shown in FIG. 4, the camera 10 includes a shooting unit 100, an image reduction unit 102, an area setting unit 104, a plurality of image cropping units 106, and a communication unit 108. Although FIG. 4 shows an example in which four image cropping portions 106 are provided, the present invention is not limited to such an example, and any number of one or more may be provided.

[1-1-1. Shooting unit 100]
The photographing unit 100 has a function of acquiring a frame image by forming an external image on an image pickup element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor) through a lens. For example, the photographing unit 100 acquires a frame image by photographing an external image at a predetermined frame rate.

[1-1-2. Image reduction unit 102]
The image reduction unit 102 generates a reduced image by reducing the frame image acquired by the shooting unit 100 to a predetermined size.

[1-1-3. Area setting unit 104]
The area setting unit 104 sets a cropping area, which is a region from which a cropping image is generated, in the frame image acquired by the photographing unit 100. For example, the area setting unit 104 sets the cropping area by the number of the image cropping units 106 provided in the camera 10 from the frame image acquired by the shooting unit 100.

FIG. 5 is an explanatory diagram showing an example of setting a cropping area by the area setting unit 104. In FIG. 5, the width of the cropping region 40 is described as “crop_width”, and the length of the vertical width of the cropping region 40 is described as “crop_height”.

As shown in FIG. 5, the area setting unit 104 detects an object to be detected such as a person 300 from the frame image 30, and sets a cropping area 40 based on the object detection position 302.

[1-1-4. Video cropping unit 106]
The image cropping unit 106 is an example of the cropped image generation unit in the present disclosure. The video cropping unit 106 generates a cropping image by cutting out the cropping area set by the area setting unit 104 from the frame image acquired by the photographing unit 100.

For example, the example shown in FIG. 3 shows four cropping images 50 generated by each of the four video cropping units 106. As shown in FIG. 3, for example, the video cropping unit 106a generates a cropping image 50a from the cropping region corresponding to the region 40a shown in FIG. 2 set by the region setting unit 104. Further, the video cropping unit 106b generates a cropping image 50b from the cropping region corresponding to the region 40b shown in FIG. 2 set by the region setting unit 104.

[1-1-5. Communication unit 108]
The communication unit 108 is an example of the acquisition unit according to the present embodiment. The communication unit 108 transmits and receives various types of information to and from the device connected to the communication network 24 via the communication network 24 described later. For example, the communication unit 108 transmits the reduced image acquired by the video reducing unit 102 and the plurality of cropped images generated by the plurality of video cropping units 106 to the storage 20. Further, the communication unit 108 receives the detection designation information set by the user for selecting the cropping target from the monitoring terminal 22.

The detection designation information may be stored in the camera 10 from the beginning instead of being received from the monitoring terminal 22. In the following, an example in which the detection designation information is received from the monitoring terminal 22 will be mainly described.

<1-2. Storage 20>
The storage 20 is a storage device for storing a reduced image and a cropping image received from the camera 10. For example, the storage 20 stores the identification information of the camera 10, the shooting date and time, the received reduced image, and the received cropping image in association with each other. The storage 20 may be installed in, for example, a data center or a monitoring center where a watchman works.

<1-3. Monitoring terminal 22>
The monitoring terminal 22 is an information processing terminal for displaying a reduced image and a cropping image generated by the camera 10. The monitoring terminal 22 may be installed, for example, in a monitoring center and used by observers.

Here, the configuration of the monitoring terminal 22 will be described in detail. FIG. 6 is a functional block diagram showing the configuration of the monitoring terminal 22 according to the present embodiment. As shown in FIG. 6, the monitoring terminal 22 has a control unit 220, a communication unit 222, a display unit 224, and an input unit 226.

[1-3-1. Control unit 220]
The control unit 220 uses hardware such as a CPU (Central Processing Unit), a RAM (Random Access Memory), and a ROM (Read Only Memory) built in the monitoring terminal 22 to generally operate the monitoring terminal 22. Control.

[1-3-2. Communication unit 222]
The communication unit 222 transmits and receives various types of information to and from the device connected to the communication network 24 via the communication network 24 described later. For example, the communication unit 222 receives the reduced image and the cropping image stored in the storage 20 from the storage 20. The communication unit 222 can also directly receive the reduced image generated by the camera 10 and a plurality of cropping images from the camera 10.

Further, the communication unit 222 transmits, according to the control of the control unit 220, the detection designation information for selecting the object to be cropped, which is input by the user on the evaluation standard setting screen described later, to the camera 10.

[1-3-3. Display unit 224]
The display unit 224 is composed of, for example, a display such as a liquid crystal display (LCD: Liquid Crystal Display) or an OLED (Organic Light Emitting Display). The display unit 224 displays, for example, a monitoring screen including a reduced image or a cropping image received from the storage 20.

Further, the display unit 224 displays the evaluation standard setting screen according to the control of the control unit 220. This evaluation standard setting screen is a screen for the user to set (or change) the detection designation information used for determining the object to be cropped from the frame image captured by the camera 10. For example, the detection designation information is set to include one or more evaluation items selected on the evaluation standard setting screen and the evaluation criteria of the evaluation item.

Here, a display example of the evaluation standard setting screen (evaluation standard setting screen 60) will be described with reference to FIG. 7. As shown in FIG. 7, the evaluation standard setting screen 60 includes a setting field 600 for a plurality of evaluation items such as an object size setting field 600a and an object speed setting field 600b. As shown in FIG. 7, the plurality of evaluation items include, for example, the size of the object, the speed of the object, the staying time of the object, the aspect ratio of the object, the estimated time until the object goes out of the screen, and the object and the home position. Includes distance, object tracking time, object downtime, and so on. Here, the aspect ratio of an object is an evaluation item used for distinguishing the type of an object to be detected, for example, distinguishing between a person and an automobile. One of the merits of using the aspect ratio is that the type of object can be identified with a small amount of calculation.

Further, the predicted time until the object goes out of the screen is, for example, the time when the moving speed of the object is calculated based on the change in the position in the past frame, and the time is predicted based on the calculated moving speed. This "estimated time until the object goes out of the screen" is an evaluation item used when, for example, even an object moving at high speed wants to be photographed at least once. The home position is an example of the monitored area in the present disclosure. This home position is determined, for example, for each detection frame of an object. As an example, the detection frame may be determined for each place the user wishes to monitor, for example, a street, a building entrance, an exclusion zone, and the like. Further, the object detection frame may be set in association with each of the plurality of image cropping units 106 included in the camera 10.

For example, the example shown in FIG. 7 shows that the speed of the object, the aspect ratio of the object, the distance to the home position, and the stop time of the object are selected as evaluation items by the user. Further, in FIG. 7, the speed of the object is faster than the specified value, the aspect ratio of the object is smaller than the specified value, the distance to the home position is longer than the specified value, or the stop time of the object is specified. It is shown that the evaluation criteria are specified so that the object shorter than the value has a higher evaluation.

[1-3-4. Input unit 226]
The input unit 226 includes an input device such as a mouse, a keyboard, a touch panel, or a microphone. The input unit 226 accepts various inputs by the user to the monitoring terminal 22. For example, the input unit 226 accepts the input of the detection designation information to the evaluation standard setting screen displayed on the display unit 224.

<1-4. Communication network 24>
The communication network 24 is a wired or wireless transmission path for information transmitted from a device connected to the communication network 24. For example, the communication network 24 may include a public line network such as a telephone line network, the Internet, or a satellite communication network, and various LANs (Local Area Network) including Ethernet (registered trademark), WAN (Wide Area Network), and the like. .. Further, the communication network 24 may include a dedicated line network such as IP-VPN (Internet Protocol-Virtual Private Network).

The image processing system according to the present embodiment is not limited to the above-described configuration. For example, the storage 20 and the monitoring terminal 22 may be integrally configured. Alternatively, the image processing system may not include the storage 20 or the monitoring terminal 22.

In the image processing system described above, the camera 10 according to the present embodiment can automatically select the optimum cropping target based on the detection designation information designated by the user.

<< 2. Detailed description of the embodiment >>
<2-1. Configuration>
The configuration of the image processing system according to the present embodiment has been described above. Next, the configuration of the camera 10 according to the present embodiment will be described in detail.

The feature of the configuration of the camera 10 according to the present embodiment is particularly related to the configuration of the area setting unit 104.
Hereinafter, the configuration of the area setting unit 104 will be described in more detail with reference to FIG.

As shown in FIG. 8, the area setting unit 104 includes an object detection unit 120, a cropping area determination unit 122, and an overlap determination unit 124.

[2-1-1. Object detection unit 120]
(2-1-1-1. Detection example 1)
The object detection unit 120 is an object to be cropped from the detection frame in the frame image (hereinafter referred to as the current frame image) acquired by the photographing unit 100 based on the detection designation information received from the monitoring terminal 22, for example. Is detected. For example, the object detection unit 120 detects, among the plurality of objects included in the detection frame, the object having the highest value evaluated by the evaluation criteria of the evaluation items included in the received detection designation information as the cropping target.

When the received detection designation information includes a plurality of evaluation items, the object detection unit 120 includes each of the plurality of evaluation items included in the detection designation information among the plurality of objects included in the detection frame. It is also possible to detect an object to be cropped based on a combination of evaluation values based on evaluation criteria. For example, in the above case, the object detection unit 120 first evaluates each of the plurality of objects included in the detection frame according to the evaluation criteria for each of the plurality of evaluation items indicated by the detection designation information, and then each of them. Calculate the total evaluation value of the evaluation items. Then, the object detection unit 120 detects the object having the highest total evaluation value of each evaluation item as the cropping target.

(2-1-1-2. Detection example 2)
Further, when the received detection designation information includes the evaluation item “object stop time”, the object detection unit 120 determines the length of time that the current object to be detected is estimated to be stopped and the said time. It is determined whether or not to switch the object to be detected based on the comparison with the stop upper limit time included in the detection designation information.

For example, when the length of time estimated that the current object to be detected is stopped is longer than the upper limit time for stopping, the object detection unit 120 determines to switch the object to be detected to another object. do. Further, when the length of time estimated that the current object to be detected is stopped is equal to or less than the stop upper limit time, the object detection unit 120 sets the object to be detected to be the same as the previous frame image. Judge to. In addition, the object detection unit 120 may presume that the object is stopped when, for example, the amount of change in the detection position of the object to be detected between continuous frame images is within a predetermined range. It is possible.

When the received detection designation information includes the evaluation item "object tracking time", the object detection unit 120 has the duration of detection of the current object to be detected and the tracking included in the detection designation information. Based on the comparison with the upper limit time, it is determined whether or not to switch the object to be detected. The tracking upper limit time is an upper limit value of the time for maintaining the object to be cropped on the same object.

For example, when the duration of detection of the current object to be detected exceeds the tracking upper limit time, the object detection unit 120 determines to switch the object to be detected to another object. Further, when the duration of detection of the current object to be detected is equal to or less than the tracking upper limit time, the object detection unit 120 determines that the object to be detected is the same as the previous frame image.

Further, when the received detection designation information includes the evaluation item "distance between the object and the home position", the object detection unit 120 sets the home position preset in the moving image and the current detection target. Based on the comparison between the distance to the object and the monitoring upper limit distance included in the detection designation information, it is determined whether or not to switch the object to be detected.

For example, when the distance between the home position and the current object to be detected exceeds the monitoring upper limit distance, the object detection unit 120 sets the object to be detected to, for example, the object located closest to the home position. Decide to switch to another object. Further, when the distance between the home position and the current object to be detected is within the monitoring upper limit distance, the object detection unit 120 determines that the object to be detected is the same as the previous frame image. do.

(2-1-1-3. Detection example 3)
Further, when it is determined by the overlap determination unit 124 described later that the cropping regions overlap, the object detection unit 120 switches any of the determined cropping regions to another object to be cropped. To judge.

For example, the object detection unit 120 overlaps when the "detection target switching time" included in the received detection designation information elapses from the time when the overlap determination unit 124 determines that the cropping areas overlap. It is determined to switch one of the cropping target objects in the cropping area to another object.

As a modification, of the cropping regions determined by the overlap determination unit 124 to overlap the cropping regions, a cropping region for switching the object to be cropped to another object (hereinafter referred to as a cropping target switching region) is used. The object detection unit 120 may make a determination according to the received detection designation information. For example, the object detection unit 120 may determine, among the overlapping cropping regions, a cropping region including a cropping target having a lower value of the evaluation item included in the received detection designation information as the cropping target switching region. Further, the object detection unit 120 may determine, among the overlapping cropping regions, the cropping region that continues to detect (track) the object to be cropped from an earlier time as the cropping target switching region.

(2-1-1-4. Detection example 4)
The object detection unit 120 can detect a predetermined number of objects from the current frame image, for example, a number within the number of the image cropping units 106.

(2-1-1-5. Detection example 5)
Further, when the object of the cropping target (tracking target) does not exist in the detection frame, the object detection unit 120 can detect none of the objects.

[2-1-2. Cropping area determination unit 122]
(2-1-2-1. Determination example 1)
The cropping region determination unit 122 is an example of the cropping region determination unit in the present disclosure. The cropping area determination unit 122 determines the cropping area in the current frame image so as to include the detection position of the object detected by the object detection unit 120. For example, the cropping area determination unit 122 determines the cropping area in the current frame image so that the detection position of the object detected by the object detection unit 120 is at the center of the cropping area.

The shape and size of the cropping region in the frame image are basically the same for all frame images. Further, the size of the cropping area is basically set to a predetermined size.

(2-1-2-2. Determination example 2)
If no object is detected by the object detection unit 120, the cropping area determination unit 122 may decide not to output any area. Alternatively, in the above case, the cropping area determination unit 122 may determine the area including the home position as the cropping area.

[2-1-3. Overlap determination unit 124]
When the cropping area is determined by the cropping area determination unit 122, the overlap determination unit 124 determines whether or not the cropping area overlaps with another cropping area based on the overlap determination condition included in the received detection designation information. Is determined. For example, when the ratio of the area of the overlapping region to the area of the cropping region is equal to or more than a predetermined threshold value, the overlap determination unit 124 determines that the cropping region overlaps with another cropping region. Further, when the distance from the center of the cropping region to the other cropping region is equal to or less than a predetermined threshold value, the overlap determination unit 124 determines that the cropping region overlaps with the other cropping region.

<2-2. Operation>
The configuration according to the present embodiment has been described above. Subsequently, the operation according to the present embodiment will be described with reference to FIGS. 9 to 11. Here, an operation example in a scene in which the camera 10 has four image cropping units 106 and one reduced image and four cropped images are generated from one frame image will be described. It should be noted that this operation is repeatedly executed at predetermined frame rates.

[2-2-1. Overall operation]
FIG. 9 is a flowchart showing an operation example according to the present embodiment. As shown in FIG. 9, first, when a predetermined shooting timing arrives, the shooting unit 100 of the camera 10 acquires a frame image by shooting an external image (S101).

Subsequently, the image reduction unit 102 generates a reduced image by reducing the frame image (hereinafter referred to as the current frame image) acquired in S101 to a predetermined size (S103).

After that, the camera 10 repeats the "cropping image generation process" described later by the number of video cropping units 106, that is, four times (S105 to S111).

After that, the communication unit 108 transmits the reduced image generated in S103 and the four cropped images generated in S107 to the storage 20 (S113).

[2-2-2. Cropping image generation process]
Here, with reference to FIGS. 10 to 11, the operation of the “cropping image generation process” in S107 will be described in detail. As shown in FIG. 10, first, the object detection unit 120 of the camera 10 detects an object set as a cropping target at a position I, that is, an object being tracked (S151).

Subsequently, the cropping region determination unit 122 determines the cropping region in the current frame image so that the detection position of the object detected in S151 is at the center of the cropping region (S153).

Here, with reference to FIG. 11, the operation after S153 will be described. As shown in FIG. 11, after S153, the object detection unit 120 determines whether or not the total tracking time (from the start of tracking) of the objects detected in S151 has passed the tracking upper limit time (S161). ). When the total tracking time of the objects has passed the tracking upper limit time (S161: Yes), the object detection unit 120 switches the cropping target at the I position to another object (S163). After that, the camera 10 operates S153 again.

On the other hand, when the tracking time of the object does not elapse the upper limit of tracking time (S161: No), the object detection unit 120 then receives the object detected in S151 (from the time when the stop is detected). It is determined whether or not the total stop time has passed the upper limit time of stop (S165). When the total stop time of the object has passed the upper limit time of stop (S165: Yes), the object detection unit 120 operates S163.

On the other hand, when the total stop time of the object does not elapse the upper limit time of stop (S165: No), the overlap determination unit 124 has the cropping region determined in S153 as another cropping region in the same frame image. It is determined whether or not it overlaps with (S167). When the corresponding cropping region overlaps with another cropping region (S167: Yes), the object detection unit 120 operates S163.

On the other hand, when the corresponding cropping area does not overlap with another cropping area (S167: No), the video cropping unit 106 generates a cropping image by cutting out the corresponding cropping area from the current frame image. (S169).

<2-3. Effect>
[2-3-1. Effect 1]
As described above, for example, as described with reference to FIGS. 4, 8 to 11, the camera 10 according to the present embodiment detects an object based on an evaluation item selected by the user from a plurality of evaluation items. Then, the cropping area in the current frame image is determined so as to include the detection position of the detected object. Therefore, the optimum cropping target can be automatically selected from the plurality of objects included in the frame image. Further, the length of time for cropping the object to be cropped can be similarly optimized.

[2-3-2. Effect 2]
Further, when the time estimated that the object to be cropped is stopped exceeds the stop upper limit time, the camera 10 switches the object to be cropped to another object, and the object after the switching is the center. The cropping area is determined so as to be. Therefore, even if the object once defined as the cropping target continues to be located at the same place, the object to be cropped is changed to another object after the stop condition time elapses. Therefore, it is possible to prevent the same place from being continuously set in the cutout area for a long time.

[2-3-3. Effect 3]
Further, since the method of determining the cropping region by the cropping region determination unit 122 is a simple method, the camera 10 can generate a cropping image in real time.

[2-3-4. Effect 4]
Further, according to the present embodiment, it is possible to generate a reduced image and a cropping image by the camera 10 alone. Therefore, since the camera 10 does not need to transmit the frame image to another device such as a server for generating the reduced image and the cropped image, the communication amount can be reduced.

<< 3. Modification example >>
Although the preferred embodiments of the present disclosure have been described in detail with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that anyone with ordinary knowledge in the field of technology to which this disclosure belongs can come up with various modifications or amendments within the scope of the technical ideas set forth in the claims. , These are also naturally understood to belong to the technical scope of the present disclosure.

<3-1. Modification 1>
In the above-described embodiment, the example in which the image processing apparatus according to the present disclosure is the camera 10 has been described, but the present invention is not limited to such an example. For example, when the monitoring terminal 22 (control unit 220) has all of the above-mentioned image reduction unit 102, area setting unit 104, and a plurality of image cropping units 106 instead of the camera 10, according to the present disclosure. The image processing device may be the monitoring terminal 22.

Further, a separately provided server (not shown) can communicate with the camera 10 via, for example, a communication network 24, and this server replaces the camera 10 with the above-mentioned image reduction unit 102 and area setting unit 104. , And, when all of the plurality of video cropping units 106 are provided, the image processing apparatus according to the present disclosure may be a server. Further, the server and the storage 20 may be integrally configured.

<3-2. Modification 2>
Further, in the above description, an example in which the tracking upper limit time of the object and the stop upper limit time of the object are values included in the detection designation information, that is, fixed values has been described, but the camera 10 is not limited to such an example. , Tracking limit time, or stop limit time may be determined dynamically. For example, the camera 10 may dynamically determine the tracking upper limit time or the stop upper limit time for each detection frame. As an example, the camera 10 may increase the tracking upper limit time as the number of untracked objects decreases for each detection frame. Further, the camera 10 may have a longer tracking upper limit time when the detection frame includes, for example, a monitored area.

<3-3. Modification 3>
Further, according to the above-described embodiment, for example, the hardware such as the CPU, ROM, and RAM can exhibit the same functions as the above-mentioned image reduction unit 102, area setting unit 104, and image cropping unit 106. Computer programs can also be provided. Also provided is a recording medium on which the computer program is recorded.

The following configurations also belong to the technical scope of the present disclosure.
(1)
An object detection unit that detects the first object from the first frame image included in the moving image, and
A cutout area determination unit that determines a cutout area of the first frame image so as to include a detection position of the first object, and a cutout area determination unit.
Equipped with
The cutout region determination unit is based on the length of time that the first object is presumed to be stopped in the second frame image after the first frame image. An image processing device that determines the cropped area of a frame image.
(2)
The cutout area determination unit determines the cutout area of the second frame image based on whether or not the length of time estimated that the first object is stopped is larger than the stop upper limit time. , The image processing apparatus according to (1) above.
(3)
When the second frame image is a frame image when the time at which the first object is presumed to be stopped exceeds or is after the stop upper limit time, the object detection unit may perform the above-mentioned. The second object is detected from the second frame image,
The image processing apparatus according to (2) above, wherein the cutout area determination unit determines a cutout area of the second frame image so as to include a detection position of the second object.
(4)
When the second frame image is a frame image when the time at which the first object is presumed to be stopped is equal to or less than the stop upper limit time, the cutout area determination unit is the first. The image processing apparatus according to (2) or (3) above, wherein a cutout region of the second frame image is determined so as to include a detection position of one object.
(5)
The length of time that the first object is presumed to be stopped is the length of time that the amount of change in the detection position of the first object between consecutive frame images is within a predetermined range. The image processing apparatus according to any one of (2) to (4) above.
(6)
The image processing device further includes an acquisition unit that acquires the length of the stop upper limit time specified by the user.
The cutout area determination unit is based on whether or not the length of time presumed that the first object is stopped is equal to or longer than the stop upper limit time acquired by the acquisition unit. The image processing apparatus according to (5) above, which determines the cutout area of the frame image of.
(7)
The image processing apparatus according to any one of (2) to (6) above, wherein the shape and size of the cut-out region of the second frame image are the same as the cut-out region of the first frame image.
(8)
The image processing device further includes an acquisition unit that acquires target information selected by the user from a plurality of target information for determining a region to be cut out from the moving image.
The section according to any one of (2) to (5) above, wherein the cutout area determination unit further determines a cutout area of the second frame image based on the target information acquired by the acquisition unit. Image processing equipment.
(9)
The plurality of target information includes a tracking upper limit time, which is an upper limit of the time for keeping the object to be cut out to be the same corresponding to the cutout area.
When the second frame image is a frame image when the duration of detection of the first object by the object detection unit exceeds or after the tracking upper limit time acquired by the acquisition unit, the frame image is described. The object detection unit detects the second object from the second frame image and determines the second object.
The image processing apparatus according to (8) above, wherein the cutout area determination unit determines a cutout area of the second frame image so as to include a detection position of the second object.
(10)
The object detection unit further detects a predetermined number of objects from the second frame image.
The cut-out area determination unit determines the predetermined number of cut-out areas from the second frame image based on the detection positions of the predetermined number of objects, according to the above (8). Image processing device.
(11)
The plurality of target information includes an overlap determination condition for determining overlap of a plurality of cutout regions.
When it is determined that the first cut-out area and the second cut-out area partially overlap among the predetermined number of cut-out areas based on the overlap determination condition acquired by the acquisition unit, the object detection unit Detects a third object different from the object detected in association with the second cutout region from the second frame image.
The image processing apparatus according to (10), wherein the cutout area determining unit changes the position of the second cutout area so as to include the detection position of the third object.
(12)
The monitoring target area is preset in the moving image, and the monitoring target area is set in advance.
The plurality of target information includes a monitoring upper limit distance which is an upper limit value of the distance between the object to be cut out corresponding to the cutting area and the monitoring target area.
When the second frame image is a frame image when the distance between the first object and the monitored area exceeds the monitoring upper limit distance acquired by the acquisition unit, or after that, the frame image is described. The object detection unit detects the second object from the second frame image and determines the second object.
The image processing apparatus according to (8) above, wherein the cutout area determination unit determines a cutout area of the second frame image so as to include a detection position of the second object.
(13)
The image processing apparatus according to (12), wherein the second object is an object located closest to the monitored area among a plurality of objects included in the second frame image.
(14)
The plurality of target information includes the area size of the object to be detected, the moving speed of the object to be detected, the residence time of the object to be detected, the aspect ratio of the object to be detected, or the object to be detected is the moving image of the moving image. The image processing apparatus according to any one of (9) to (13) above, which includes an estimated time to move out of the region.
(15)
The image processing apparatus further includes a cut-out image generation unit that generates a cut-out image by cutting out the cut-out area determined by the cut-out area determination unit from the second frame image (1) to (14). The image processing apparatus according to any one of the above items.
(16)
Detecting the first object from the first frame image included in the moving image,
Determining the cutout area of the first frame image so as to include the detection position of the first object, and
In the second frame image after the first frame image, the cutout region of the second frame image is determined based on the length of time estimated that the first object is stopped. An image processing method that comprises doing.
(17)
An object detection unit that detects the first object from the first frame image included in the moving image, and
A cutout area determination unit that determines a cutout area of the first frame image so as to include a detection position of the first object, and a cutout area determination unit.
A cut-out image generation unit that generates a cut-out image by cutting out the cut-out area determined by the cut-out area determination unit from the first frame image.
A storage unit that stores the generated cut-out image,
Equipped with
The cutout region determination unit is based on the length of time that the first object is presumed to be stopped in the second frame image after the first frame image. An image processing system that determines the cropped area of a frame image.

10 Camera 20 Storage 22 Monitoring terminal 24 Communication network 100 Imaging unit 102 Image reduction unit 104 Area setting unit 106 Video cropping unit 108 Communication unit 120 Object detection unit 122 Cropping area determination unit 124 Overlap determination unit 220 Control unit 222 Communication unit 224 Display unit 226 Input section

Claims (18)

At the first time point of the moving image, the first object is selected as the gaze target from one or more detected objects based on a predetermined criterion.
When it is detected that the first object is stopped for a predetermined time or more at a second time point after the first time point, the gaze target is said to be the same from the first object. It has a control unit that switches to a second object that is different from the first object.
The control unit
An image processing device that determines the first predetermined time based on the number of one or more detected objects in the moving image that are not selected as the gaze target . When the control unit detects that the first object is stopped for the first predetermined time or more at the second time point, the gaze target is the second object from the first object. The image processing apparatus according to claim 1, wherein the second object is switched to the second object selected from the two or more detected objects at the time point. The control unit detects that the first object is stopped when the amount of change in the detection position of the first object between the frame images of the moving image is within a predetermined range. Item 2. The image processing apparatus according to Item 1. When the control unit detects that the first object is continuously selected as the gaze target for a second predetermined time or longer at the second time point, the gaze target is selected as the first gaze target. The image processing apparatus according to any one of claims 1 to 3, which switches from an object to the second object. The control unit determines the first predetermined time or the second predetermined time based on the number of one or more detected objects in the moving image that are not selected as the gaze target. The image processing apparatus according to claim 4. The control unit prolongs the first predetermined time or the second predetermined time as the number of objects not selected as the gaze target among the one or more detected objects in the moving image is smaller. The image processing apparatus according to claim 5. The image according to any one of claims 4 to 6, wherein the control unit lengthens the first predetermined time or the second predetermined time when a specific region is included in the moving image. Processing equipment. When the control unit detects that the distance between the first object and the specific region in the moving image exceeds a predetermined distance at the second time point, the first one. The image processing apparatus according to any one of claims 1 to 7, wherein the object is excluded from the gaze target. The control unit detects that the distance between the first object and the specific region exceeds the predetermined distance at the second time point, so that the first object is said to be the first object. The image according to claim 8, wherein when excluded from the gaze target, the object located closest to the specific region among the one or more detected objects at the second time point is selected as the gaze target. Processing device. The image processing apparatus according to any one of claims 1 to 9, wherein the control unit selects a specific region in the moving image as the gaze target when there is no detected object in the moving image. .. The image processing apparatus according to any one of claims 1 to 10, wherein the control unit selects a predetermined number of objects among one or more detected objects in the moving image as the gaze target. The predetermined criteria are the size, moving speed, staying time, aspect ratio, tracking time, stop time of the detected object in the moving image, and the detected object in the moving image is outside the area of the moving image. The image processing apparatus according to any one of claims 1 to 11, which comprises at least one of the estimated times until movement. The image processing according to any one of claims 1 to 12, wherein the image processing apparatus includes a cutting area determination unit that determines a cutting area of the moving image so as to include a detection position of the object to be gazed. Device. When it is determined at the second time point that the cutout area corresponding to the first object partially overlaps with the cutout area corresponding to the third object, the control unit is selected as the gaze target. 13. The image processing apparatus according to claim 13 , wherein either the first object or the third object is switched to the second object different from the first object or the third object. .. The image processing apparatus according to claim 14, wherein among the first object and the third object, an object that has been continuously selected as the gaze target for a longer period of time is switched to the second object. The image processing apparatus according to any one of claims 13 to 15, further comprising a cut-out image generation unit that generates a cut-out image by cutting out the cut-out area determined by the cut-out area determination unit. .. At the first time point of the moving image, the first object is selected as the gaze target from one or more detected objects based on a predetermined criterion.
When it is detected that the first object is stopped for a predetermined time or more at a second time point after the first time point, the gaze target is said to be the same from the first object. Switching to a second object that is different from the first object,
An image processing method comprising determining the first predetermined time based on the number of objects not selected as the gaze target among one or more detected objects in the moving image . At the first time point of the moving image, the first object is selected as the gaze target from one or more detected objects based on a predetermined criterion.
When it is detected that the first object is stopped for a predetermined time or more at a second time point after the first time point, the gaze target is said to be the same from the first object. A control unit that switches to a second object that is different from the first object,
A cutting area determination unit that determines a cutting area so as to include the detection position of the object to be gazed, and a cutting area determining unit.
A cut-out image generation unit that generates a cut-out image by cutting out the cut-out area determined by the cut-out area determination unit.
A storage unit that stores the generated cut-out image,
Equipped with
The control unit
An image processing system that determines the first predetermined time based on the number of one or more detected objects in the moving image that are not selected as the gaze target .

Images