JP6580213B1 - Image processing apparatus, photographing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To automatically determine an appropriate zoom magnification in a count of people performed while controlling a zoom magnification and a photographing direction of a photographing apparatus. The present invention is characterized by counting the number of persons from an image captured by dividing a predetermined area into a plurality of imaging areas, and detecting means for detecting a person included in the image; A counting unit that counts the number of persons; a control unit that controls a shooting operation of the shooting unit; and a determination unit that determines a zoom magnification of the shooting unit. The counting unit changes the zoom magnification stepwise. The determination means counts the number of persons for each of the plurality of images taken, and the determination means determines and controls the zoom magnification of the photographing means based on the zoom magnification at the time of shooting the image with the maximum counted number of persons. The means is characterized in that control is performed so as to photograph another photographing region using the determined zoom magnification. [Selection] Figure 9

Description

  The present invention relates to an image processing technique, and more particularly to a technique for counting the number of persons included in an image taken by a photographing apparatus.

  Conventionally, there has been a need to count the number of persons existing in a certain area at a station platform or a retail store. In response to such needs, there is a person counting technique for detecting a person appearing in an image by analyzing the image taken by the photographing apparatus and counting the number of persons.

  On the other hand, there is a similar need to count the number of people in a place where a large number of people gather in a wider area such as a stadium or a concert hall. However, when shooting is performed with the zoom magnification of the imaging device being lowered so that the largest possible area is within the angle of view, the size (number of pixels) of the person included in the captured image may be reduced.

  Here, in order to detect a human body in a general human body detection technique, it is necessary that the human body shown in the image has a certain size (number of pixels) or more. However, when counting people in a wide area, there may be a case where people are distributed in the depth direction as viewed from the photographing apparatus with a size smaller than the size capable of detecting a human body.

  In order to deal with such problems, Patent Document 1 discloses a method using a preset function of a network camera. In Patent Document 1, the network camera is panned, tilted, and zoomed (hereinafter referred to as PTZ) and shot while traveling around a wide area, thereby counting the number of people in the entire area while maintaining a detectable human body size. .

JP 2017-204838 A

  However, in Patent Document 1, it is necessary for the user to set in advance a zoom magnification for each shooting location that is a preset position, and an appropriate zoom magnification is set in consideration of the size of the human body that changes in the depth direction. The work was cumbersome for the user.

  As described above, an object of the present invention is to automatically determine an appropriate zoom magnification in counting the number of people while controlling the zoom magnification and the photographing direction of the photographing apparatus.

  In order to solve the above problems, for example, an image processing apparatus of the present invention has the following configuration. That is, an image processing apparatus that counts the number of persons from an image captured by dividing a predetermined area into a plurality of imaging areas by an imaging unit, the detection unit detecting a person included in the captured image; A counting unit that counts the number of persons detected by the detecting unit; a control unit that controls a photographing operation of the photographing unit; and a determining unit that determines a zoom magnification of the photographing unit. The number of persons is counted for each of a plurality of images captured by changing the zoom magnification in stages, and the determining means sets the zoom magnification at the time of capturing an image with the maximum number of persons counted by the counting means. A zoom magnification of the photographing unit is determined based on the control unit, and the control unit performs control to photograph another photographing region using the determined zoom magnification. And butterflies.

  According to the present invention, it is possible to automatically determine an appropriate zoom magnification in counting the number of people while controlling the zoom magnification and shooting direction of the photographing apparatus.

It is a schematic diagram which shows a system configuration. It is the schematic which shows the hardware constitutions of an image processing apparatus and an imaging device. It is an external view of an imaging device. It is a functional block diagram of an imaging device. It is a functional block diagram of an image processing apparatus. It is a schematic diagram for demonstrating the process which determines zoom magnification. It is a schematic diagram for demonstrating the process which determines zoom magnification. It is a schematic diagram for demonstrating the process which determines zoom magnification. It is a flowchart of the process which determines zoom magnification. It is a schematic diagram for demonstrating the process of a people count. It is a flowchart of a process of counting people. It is a schematic diagram for demonstrating the process which determines zoom magnification. It is a flowchart of the process which determines zoom magnification. It is a functional block diagram of an imaging device and a client device having one or more functions of an image processing device.

  Embodiments according to the present invention will be described below with reference to the accompanying drawings. In addition, the structure shown in the following embodiment is only an example, and is not limited to the illustrated structure.

(Embodiment 1)
FIG. 1 is a schematic diagram showing a system configuration according to the present embodiment. The image processing apparatus 100 is an apparatus that executes image processing to be described later. The image processing apparatus 100 is, for example, a personal computer in which a program for realizing an image processing function described later is installed.

  The display device 101 is connected to the image processing device 100 and includes a display that displays an image based on image data distributed from the imaging device 102.

  The imaging apparatus 102 is an apparatus that captures an image. For example, the imaging apparatus 102 is a network camera that can transmit image data of an image via a network. Note that the photographing apparatus 102 according to the present embodiment can perform PTZ (Pan Tilt Zoom) control. The recording device 103 records the image data transmitted from the imaging device 102.

  Further, the image processing apparatus 100, the imaging apparatus 102, and the recording apparatus 103 communicate with each other via the network 104. The network 104 includes a plurality of routers, switches, cables, and the like that satisfy a communication standard such as ETHERNET (registered trademark). In the present embodiment, any communication standard, scale, and configuration may be used as long as communication between the image processing apparatus 100, the imaging apparatus 102, and the recording apparatus 103 can be performed. For example, the network 104 may be configured by the Internet, a wired LAN (LOCAL AREA NETWORK), a wireless LAN (WIRELESS LAN), a WAN (WIDE AREA NETWORK), or the like.

  Next, the hardware configuration of the image processing apparatus 100 and the photographing apparatus 102 will be described with reference to FIG. FIG. 2 is a schematic diagram illustrating a hardware configuration of the image processing apparatus 100 and the imaging apparatus 102.

  First, the hardware configuration of the image processing apparatus 100 will be described. A CPU (CENTRAL PROCESSING UNIT) 200 controls the image processing apparatus 100.

  An I / F (Interface) 201 communicates with the imaging apparatus 102 via the network 104 according to TCP / IP, HTTP, or the like. Further, the I / F 201 receives image data of an image captured by the image capturing apparatus 102 and transmits various commands for controlling the image capturing apparatus 102.

  A RAM (RANDOM ACCESS MEMORY) 202 temporarily stores a computer program executed by the CPU 200. The RAM 202 temporarily stores data (commands and image data) acquired from the outside via the I / F 201. The RAM 202 provides a work area used when the CPU 200 executes image processing to be described later. The RAM 202 functions as a frame memory or a buffer memory, for example.

  CPU 200 executes a computer program stored in RAM 202. In addition to the CPU, a processor such as a DSP (DIGITAL SIGNAL PROCESSOR) may be used.

  An HDD (HARD DISK DRIVE) 203 stores an operating system program and image data. Computer programs and data for executing flowcharts shown in FIGS. 9, 11, and 13 to be described later are stored in the HDD 203, and are appropriately loaded into the RAM 202 and executed by the CPU 200 under the control of the CPU 200. In addition to the HDD, other storage media such as a flash memory may be used.

  Note that the housing of the image processing apparatus 100 may be integrated with the display apparatus 101. A UI (USER INTERFACE) 204 is an input device such as a keyboard or a mouse.

  In the following description, an example in which the CPU 200 executes image processing will be described. However, at least a part of the processing of the CPU 200 may be performed by dedicated hardware. For example, processing for displaying GUI (GRAPHICAL USER INTERFACE) and image data on the display device 101 may be executed by a GPU (GRAPHICS PROCESSING UNIT). Further, the process of reading the program code from the HDD 203 and expanding it into the RAM 202 may be executed by a DMA (DIRECT MEMORY ACCESS) functioning as a transfer device.

  Next, the hardware configuration of the photographing apparatus 102 will be described. A CPU 210 is a central processing unit that performs overall control of the photographing apparatus 102.

  The I / F 211 communicates with the image processing apparatus 100 via the network 104 according to TCP / IP, HTTP, or the like. The I / F 211 transmits image data and the like of the image to the image processing apparatus 100 and receives various commands from the image processing apparatus 100 for controlling the photographing apparatus 102.

  The RAM 212 temporarily stores a computer program executed by the CPU 210. The RAM 212 temporarily stores data (commands, parameters related to imaging conditions, etc.) acquired from the outside via the I / F 211. The RAM 212 provides a work area used when the CPU 210 executes processing. The RAM 212 functions as, for example, a frame memory or a buffer memory.

  The ROM 213 stores a program for the CPU 210 to control the photographing apparatus 102 and the like. In addition to the ROM 213, a secondary storage device equivalent to the HDD 203 may be provided.

  The photographing device 214 is a photographing unit that photographs an image provided in the photographing apparatus 102. Details of the photographing device 214 will be described later with reference to FIGS.

  Next, the photographing apparatus 102 according to the present embodiment will be described with reference to FIGS. 3 and 4. FIG. 3 is an external view of the photographing apparatus 102. FIG. 4 is a functional block diagram of a photographing function executed by the photographing device 214 of the photographing apparatus 102.

  The direction in which the optical axis of the lens 302 faces is the photographing direction of the photographing apparatus 102, and the light beam that has passed through the lens 302 forms an image on the image sensor of the photographing unit 305.

  The lens driving unit 310 includes a driving system that drives the lens 302, and changes the focal length of the lens 302. The lens driving unit 310 is controlled by the pan / tilt / zoom control unit 308.

  The pan driving unit 300 includes a mechanical driving system that performs a pan operation and a motor as a driving source, and performs rotational driving for rotating the photographing direction of the photographing apparatus 102 in the pan direction 303. The pan driving unit 300 is controlled by a pan / tilt / zoom control unit 308.

  The tilt drive unit 301 is configured by a mechanical drive that performs a tilt operation and a motor as a drive source, and performs a rotational drive for rotating the photographing direction of the photographing apparatus 102 in the tilt direction 304. The tilt driving unit 301 is controlled by a pan / tilt / zoom control unit 308.

  The imaging unit 305 includes an imaging element (not shown) such as a CCD (charge coupled device) sensor or a CMOS (complementary metal oxide semiconductor) sensor. The photographing unit 305 photoelectrically converts the subject image formed through the lens 302 to generate an electrical signal.

  The image processing unit 306 performs processing for converting the electrical signal photoelectrically converted by the photographing unit 305 into a digital signal, compression encoding processing, and the like, and generates image data.

  The pan / tilt zoom control unit 308 controls the pan driving unit 300, the tilt driving unit 301, and the lens driving unit 310 based on the instruction transmitted from the system control unit 307.

  The communication unit 309 communicates with the image processing apparatus 100 via the I / F 211. For example, the communication unit 309 transmits the generated image data to the image processing apparatus 100. The communication unit 309 receives a command for controlling the photographing apparatus 102 from the image processing apparatus 100 and transmits the command to the system control unit 307.

  The system control unit 307 controls the entire photographing apparatus 102 according to processing executed by the CPU 210, and performs, for example, the following processing. That is, the system control unit 307 analyzes a command for controlling the photographing apparatus 102 transmitted from the communication unit 309, and performs processing according to the command. In addition, the system control unit 307 instructs the pan / tilt / zoom control unit 308 to perform pan / tilt / zoom operations.

  In addition, when the system control unit 307 sends the image data generated by the image processing unit 306 to the image processing apparatus 100, the system control unit 307 also adds the shooting information when the image data is acquired to the image data. Note that the shooting information in the present embodiment is obtained from the pan / tilt / zoom control unit 308 or the like, and the pan angle and tilt angle of the shooting apparatus 102, and the zoom magnification of the shooting apparatus 102 when an image is shot by the shooting apparatus 102. , Including information indicating.

  The pan angle is an angle of the photographing direction (optical axis) in the pan direction 303 of the photographing apparatus 102 when one of the driving ends of the pan driving unit 300 is set to 0 °. The tilt angle is an angle of the photographing direction (optical axis) in the tilt direction 304 of the photographing apparatus 102 when one of the driving ends of the tilt driving unit 301 is set to 0 °. Note that the zoom magnification of the photographing apparatus 102 when an image is photographed by the photographing apparatus 102 is calculated from the focal length of the lens 302.

  Next, with reference to FIG. 5, in the process of counting the number of persons from the image captured by dividing the count area into a plurality of imaging areas by the imaging apparatus 102, the zoom magnification by the image processing apparatus 100 according to the present embodiment. The process of determining the will be described. Note that the count area represents an area to be counted, and the imaging area represents an area that the imaging apparatus 102 captures.

  FIG. 5 is a functional block diagram of the image processing apparatus 100 according to the present embodiment. Each functional block shown in FIG. 5 is a hardware module, but may be realized by a software module. Further, the software module and the hardware module may be realized in cooperation. In the following description, each function of the image processing apparatus 100 is assumed to be executed by the CPU 200 of the image processing apparatus 100.

  As shown in FIG. 5, the image processing apparatus 100 includes a communication unit 501, a detection unit 502, a count unit 503, a command management unit 504, a determination unit 505, a display unit 506, a storage unit 507, and an operation. A reception unit 508.

  The communication unit 501 receives an image photographed by the photographing apparatus 102 or transmits a command for controlling the photographing apparatus 102 via the I / F 201.

  The detection unit 502 detects a person included in the image received by the communication unit 501. In addition, the detection part 502 in this embodiment shall detect the human body contained in an image using a collation pattern (dictionary).

  At this time, it is not necessary to dictionary the features of the whole body as a matching pattern. For example, a collation pattern may be prepared for a part of the human body such as the upper body, the lower body, the head, the face, and the foot, and the detection unit 502 may consider that a human body has been detected when a part of the human body is detected from the image. . Note that the detection unit 502 only needs to have a function of detecting a human body from an image, and may be a known moving object detection or skin color detection, and is not limited to pattern matching processing.

  In this embodiment, the minimum size (number of pixels) of a person that can be detected by the detection unit 502 is assumed to be S pixels for convenience.

  The count unit 503 counts the number of persons detected by the detection unit 502.

  The command management unit 504 generates commands for controlling panning, tilting, zooming, and the like of the photographing apparatus 102 and controlling photographing operations such as control for photographing an image. The command management unit 504 generates a command for acquiring the shooting state of the shooting apparatus 102. The command generated by the command management unit 504 is sent to the photographing apparatus 102 via the communication unit 501.

  The determining unit 505 changes the zoom magnification in stages, and the count unit 503 counts the number of persons photographed by the photographing apparatus 102 in steps, and the zoom magnification at the time of photographing an image with the largest number of people. Based on the above, the zoom magnification of the photographing apparatus 102 is determined. Note that the determination unit 505 in the present embodiment determines the zoom magnification at the time of shooting an image in which the number of people counted by the counting unit 503 is the maximum. Note that the zoom magnification determined by the determination unit 505 corresponds to the size of a shooting area that is an area shot by the shooting apparatus 102.

  The command management unit 504 generates a command for controlling the imaging apparatus 102 based on the zoom magnification determined by the determination unit 505, and the count unit 503 divides the count area into a plurality of imaging areas from the captured images. Perform a people count. Detailed description of the processing of the command management unit 504 and the determination unit 505 at this time will be described later.

  In addition, although the count area | region which is an area | region made into count object is set as the area | region set by the user via the operation reception part 508 in this embodiment, it is not restricted to this. For example, the count area may be a preset area.

  The display unit 506 generates data corresponding to the screen to be displayed on the display device 101 and causes the display device 101 to display the data.

  The storage unit 507 stores the image received by the communication unit 501, the detection result of the detection unit 502, and the like. In addition, the storage unit 507 stores a shooting state of the shooting device 102 when an image processed by the detection unit 502 is shot.

  The operation receiving unit 508 receives an operation performed by the user via the UI 204. For example, the display unit 506 causes the display device 101 to display a panoramic image obtained by repeating pan / tilt / zoom control and photographing within a range that the photographing device 102 can photograph. Then, the operation reception unit 508 may receive position information of the count area set by the user with respect to the panoramic image displayed on the display device 101 by the display unit 506.

  Next, processing for determining a zoom magnification in a certain shooting direction by the image processing apparatus 100 according to the present embodiment will be described with reference to FIGS.

  FIG. 6A is a schematic diagram illustrating a state in which the photographing device 102 is looking down at the persons gathering in the count area 600 that is the target of the number of people. An area 601 indicates an imaging area in which the imaging direction of the imaging apparatus 102 is imaged toward the point 602.

  In FIG. 6B, the shooting direction of the shooting apparatus 102 is pointed at the point 602, and the zoom ratio of the shooting apparatus 102 is changed stepwise from the wide end to the tele end while the shooting apparatus 102 has different zoom magnifications. It is a schematic diagram of the image | photographed image. Note that a human body indicated by diagonal lines in each image indicates a human body detected by the detection unit 502.

  The human body 604 shown in the wide-end image 603 is not detected because it is less than the S pixel that is the minimum size detectable by the detection unit 502.

  In the image 605 photographed at a zoom magnification that is a predetermined percentage higher than the zoom magnification at the time of photographing the image 603, 25% of the human bodies 606 in the image are not detected because they are less than S pixels. On the other hand, in the image 605, 75% of human bodies 607 are detected because they exceed S pixels, and the number of detected persons is 18.

  In the tele end image 608, all human bodies 609 are detected because they exceed S pixels, and the number of detected persons is eight.

  As described above, if the zoom magnification of the photographing apparatus is extremely increased toward the telephoto end in counting the number of people for a wider area, the angle of view is narrowed although the accuracy of human body detection is increased. In this case, the number of shots when the count area is shot while performing PTZ control increases, and the number of times of analysis processing by the detection unit 502 and the count unit 503 increases, resulting in an increase in processing time.

  On the other hand, if the zoom magnification is too low toward the wide end, the angle of view increases, but the size of the person appearing in the image decreases and the accuracy of human body detection decreases, so the accuracy of counting people decreases.

  From the above, when carrying out the process of counting the number of people in a wider area, the zoom magnification is not set too high, and the zoom magnification is not set too low, that is, the reduction in counting accuracy and the increase in processing time are suppressed. It is necessary to determine a zoom magnification that satisfies both. Next, with reference to FIG. 7, a process in which the determination unit 505 according to the present embodiment determines the zoom magnification of the imaging device 102 will be described.

  FIG. 7 is a schematic diagram illustrating an example of the transition of the number of detected human bodies detected by the detection unit 502 when the zoom magnification of the photographing apparatus 102 is changed stepwise from the wide end to the tele end. FIG. 7A is a schematic diagram illustrating an example of the transition of the actual number of persons actually present in the image when the zoom magnification of the photographing apparatus 102 is changed stepwise from the wide end to the tele end. As shown in FIG. 7A, as the zoom magnification of the photographing apparatus 102 is increased, the actual number of human bodies appearing in the photographed image decreases.

  FIG. 7B shows a human body that exceeds the size (S pixel) that the detection unit 502 can detect a human body out of the actual number of people in the image when the zoom magnification of the photographing apparatus 102 is changed in stages from the wide end to the tele end. It is a schematic diagram which shows an example of transition of the ratio of. As shown in FIG. 7B, all human bodies do not satisfy the detectable size at the wide end. However, when a certain zoom magnification 701 is exceeded, the proportion of human bodies that satisfy the detectable size increases. After a certain zoom magnification 702, all human bodies appearing in the image satisfy the detectable size.

  FIG. 7C is a schematic diagram illustrating an example of the transition of the number of human bodies detected in the image by the detection unit 502 when the zoom magnification of the photographing apparatus 102 is changed stepwise from the wide end to the tele end. In FIG. 7C, a broken line 703 represents the actual number of people in the image captured by the image capturing apparatus 102 illustrated in FIG. Of the actual number of persons, only the human body satisfying the detectable size is detected by the detecting unit 502, and therefore the number of detected human bodies when the zoom magnification is moved from the wide end to the tele end is as shown by a solid line in FIG. It is expressed in In the determination unit 505 according to the present embodiment, the counting unit 503 counts the number of persons photographed by the photographing apparatus 102 in which the zoom magnification is changed in stages, and each of the images having different zoom magnifications at the time of photographing. Determines the zoom magnification at which the image with the maximum is taken.

  Next, with reference to FIG. 8 and FIG. 9, the image processing apparatus 100 according to the present embodiment in the process of counting the number of persons from the image captured by dividing the count area into a plurality of imaging areas by the imaging apparatus 102. A process for determining the zoom magnification according to will be described. FIG. 8 is a diagram for explaining processing for determining the zoom magnification. FIG. 9 is a flowchart of a process of determining the zoom magnification by the image processing apparatus 100 according to the present embodiment in the process of counting the number of persons from an image captured by dividing the count area into a plurality of imaging areas by the imaging apparatus 102. It is.

  Before entering the flow shown in FIG. 9, it is assumed that the zoom magnification of the photographing apparatus 101 is appropriately controlled according to the control by the command management unit 504 and further faces the target photographing direction. However, in the following description, it is assumed that the zoom magnification of the photographing apparatus 101 is controlled to the wide end. Details of the control by the command management unit 504 will be described later with reference to FIGS. 10 and 11.

  In the present embodiment, the processing of the flowchart shown in FIG. 9 is executed by each functional block of FIG. 5 realized by the CPU 200 executing a computer program stored in the HDD 203 of the image processing apparatus 100. explain. However, a part of the processing of the flowchart shown in FIG. 9 may be executed by dedicated hardware.

  In step S901, the communication unit 501 receives an image captured by the imaging apparatus 102, and the detection unit 502 detects a human body included in the image.

  Next, in step S902, the count unit 503 counts the number of human bodies detected by the detection unit 502.

  In step S <b> 903, the storage unit 507 stores the zoom magnification when the image processed by the detection unit 502 is captured and the number of people counted by the count unit 503 in the image in association with each other.

  Next, in step S904, based on the number of people stored in the storage unit 507 in step S903, if the number of counted people is not reduced compared to the number of people counted last time, the process proceeds to step S905, where the number is decreasing. Transits to step S907. If the number of people counted last time is not stored in the storage unit 507, that is, if the process performed in step S904 in the flowchart is the first time, the process proceeds to step S905.

  In step S905, the command management unit 504 generates a command for performing control to capture an image after increasing the zoom magnification of the imaging apparatus 102 to the tele end side by a predetermined ratio, and the command is transmitted via the communication unit 501. Is transmitted to the photographing apparatus 102. Note that the predetermined ratio, which is the degree to which the zoom magnification is changed, may be set in advance or may be set as appropriate by the user.

  In step S906, the command management unit 504 changes the zoom magnification of the photographing apparatus 102 by a predetermined ratio, thereby eliminating the difference in position between the photographing area before the change and the photographing area after the change. A command for controlling the photographing apparatus 102 to photograph the image is generated.

  Here, the process of step S906 will be described with reference to FIG. In FIG. 8A, an area 801 is an imaging area in which the lower left end of the count area 600 is imaged with the zoom magnification of the imaging apparatus 102 set to the wide end. In step S905, the photographing area of the photographing apparatus 102 is changed from the area 801 to the area 802 by changing the zoom magnification so as to increase toward the telephoto end by a predetermined ratio. In this case, a difference in position occurs between the lower left end point of the region 801 and the lower left end point of the region 802. In the command management unit 504 of the imaging apparatus 102 according to the present embodiment, the imaging area of the imaging apparatus 102 moves from the area 802 to the area 803 (shown in FIG. 8B) so as to eliminate this difference in position, and further the image. Generate a command to control shooting. The command generated at this time is sent to the photographing apparatus 102 via the communication unit 501.

  As described above, the processes in steps S901 to S906 are repeated until the number of people counted by the counting unit 503 decreases.

  In step S907, the determination unit 505 determines a zoom magnification that maximizes the number of human bodies detected from data in which the zoom magnification and the number of people stored in the storage unit 507 are associated. If there are a plurality of zoom magnifications that maximize the number of detected human bodies in step S907, the determination unit 904 may determine the zoom magnification on the widest end side among the plurality of zoom magnifications. .

  In step S908, the command management unit 504 controls the zoom magnification of the photographing apparatus 102 so as to achieve the determined zoom magnification, and further controls to eliminate the difference in the position of the photographing region caused by changing the zoom magnification. Is generated. At this time, as described with reference to FIG. 8, the command management unit 504 determines the difference in position that occurs between the lower left endpoint of the shooting area before the zoom magnification change and the lower left endpoint of the shooting area after the change. A command for controlling the photographing direction of the photographing apparatus 102 is generated so as to be eliminated. The command generated at this time is transmitted to the photographing apparatus 102 via the communication unit 501. Note that an area 1001 in FIG. 10A indicates the imaging area of the imaging apparatus 102 controlled by the command management unit 504 in step S908.

  Through the above processing, the image processing apparatus 100 according to the present embodiment determines a zoom magnification suitable for counting the number of persons in the process of counting the number of persons from an image captured by dividing the count area into a plurality of imaging areas. .

  In step S904, when the zoom magnification of the photographing apparatus 102 has not reached the tele end, the process may proceed to step S905, and when the zoom magnification has reached the tele end, the process may proceed to step S907. However, as shown in FIG. 7C, since the number of detected human bodies is considered to decrease after reaching the maximum value, the above-described embodiment is used to specify the zoom magnification that maximizes the number of detected human bodies. According to the process of step S904 according to the above, it is possible to further reduce the processing time.

  Next, referring to FIG. 10 and FIG. 11, based on the zoom magnification determined by the image processing apparatus 100 according to the first embodiment, the number of people is counted from the images shot by dividing the count area into a plurality of shooting areas. Processing to be executed will be described.

  FIG. 10 is a schematic diagram for explaining a process of counting the number of people from an image shot by dividing the count area into a plurality of shooting areas. Note that the count area 600 in this embodiment is set by the user, and the position information of the set count area 600 is stored in the storage unit 507. The position information of the count area 600 is determined by the pan angle and tilt angle at each vertex of the count area 600.

  An area 801 illustrated in FIG. 10A is an imaging area for imaging the lower left portion of the count area 600 in a state where the zoom magnification of the imaging apparatus 102 is set to the wide end. At this time, the command management unit 504 according to the present embodiment allows the imaging device 102 to match the lower left endpoint at each vertex of the count area 600 and the lower left endpoint at each vertex of the imaging area of the imaging device 102. A command for controlling the shooting direction is generated. Then, based on the command transmitted via the communication unit 501, the shooting direction of the shooting apparatus 102 becomes the center of the area 801.

  An imaging area 1001 is an imaging area of the imaging apparatus 102 controlled by executing the control described in the flowchart shown in FIG. 9 with respect to the imaging direction of the imaging apparatus 102 controlled by the command management unit 504.

  Next, the command management unit 504 controls the pan angle so that the shooting direction of the shooting apparatus 102 moves to the right with respect to the shooting region 1001, and then generates a command for performing control for shooting an image, and the communication unit 501. To the photographing apparatus 102. At this time, the command management unit 504 generates a command for performing control for capturing an image after controlling to move to the capturing region 1002 adjacent to the capturing region 1001 in which the number of people has already been counted. Then, the number of people count process is performed again on the image obtained by shooting the shooting area 1002, and the number of people is recorded.

  The process of counting the number of persons in the image captured by moving the imaging direction of the imaging apparatus 102 to the right is repeated until the imaging area of the imaging apparatus 102 reaches the right end of the count area 600. In the present embodiment, a side connecting the lower left end point and the lower right end point at each vertex of the imaging region (hereinafter referred to as a lower side), and an upper right end point and a lower right end point at each vertex of the count region 600 If there is a point that intersects the connecting edge (hereinafter, the right edge), it is assumed that the right edge has been reached. In the case of FIG. 10B, the process is repeated until there is a region 1003 where there is a point where the lower side of the photographing region of the photographing device 102 and the right end side of the count region 600 intersect.

  In this manner, the command management unit 504 generates a command for controlling to shoot another shooting area using the zoom magnification determined by the determination unit 505. In the case of FIG. 10B, using the zoom magnification determined by the determination unit 505, a command for controlling to capture the image capturing areas 1002 and 1003 is generated.

  Next, as illustrated in FIG. 10C, the command management unit 504 does not change the zoom magnification of the image capturing device 102, and the image capturing region of the image capturing device 102 is positioned at the left end in the count region 600 and above the region 1001. Generate a command to control the movement of The generated command is transmitted to the photographing apparatus 102 via the communication unit 501.

  At this time, the command management unit 504 according to the present embodiment shoots the photographing apparatus 102 so that the lower left end point at each vertex of the photographing region, the left end side of the count region 600, and the upper side of the region 1001 intersect. Generate commands to control direction. Note that the left end side of the count region 600 is a side connecting the upper left end point and the lower left end point at each vertex of the count region 600. The upper side of the region 1001 is a side connecting the upper left end point and the upper right end point at each vertex of the region 1001. The imaging area of the imaging apparatus 100 controlled in this way is an area 1004.

  Note that the command management unit 504 in the present embodiment does not change the zoom magnification of the image capturing apparatus 102, but the image capturing area may be moved after controlling the zoom magnification to the wide end. However, since the shooting area is controlled from the lower part of the count area, that is, the area where the human body is considered to be the largest, the zoom is not always necessary when shooting the upper part, that is, the back side in the depth direction. There is no need to return the magnification to the wide end. Therefore, the command management unit 504 in the present embodiment generates a command for controlling the shooting area from the right end of the count area to the upper stage of the left end without changing the zoom magnification of the shooting apparatus 102. By doing so, when executing the control described in the flowchart shown in FIG. 9, it is possible to shorten the processing time for determining the zoom magnification at the time of photographing an image with the maximum number of human bodies detected.

  In addition, an area 1005 in FIG. 10C executes the control described in the flowchart shown in FIG. 9 with respect to the shooting direction of the shooting apparatus 102 controlled by the command management unit 504, thereby controlling the shooting apparatus 102 that is controlled. This is a shooting area.

  Next, the command management unit 504 generates a control command for shooting an image by moving the shooting direction of the shooting apparatus 102 to the right with respect to the shooting area 1005. Then, the process of counting the number of people in the photographed image is repeated until the photographing area of the photographing apparatus 102 reaches the right end of the count area 600, similarly to the process performed at the bottom of the count area 600. In the case of FIG. 10D, the process is repeated until the imaging area of the imaging apparatus 102 reaches the area 1008.

  As described above, by repeating the process of counting the number of people while controlling the photographing area of the photographing apparatus 102 from the left end to the right end of the count area 600, until the photographing area reaches the upper end and the right end of the count area 600, Realize wide-area people count processing. In the present embodiment, the position of the side connecting the upper right end point and the upper left end point at each vertex of the shooting region of the shooting apparatus 102 is the side connecting the upper right end point and the upper left end point at each vertex of the count region 600. If it is above the position, it is assumed that the upper end has been reached.

  Next, with reference to the flowchart of FIG. 11, processing for counting the number of people from an image captured by dividing the count area into a plurality of imaging areas according to the present embodiment described with reference to FIG. 10 will be described.

  In the present embodiment, the processing of the flowchart shown in FIG. 11 is executed by each functional block of FIG. 5 realized by the CPU 200 executing the computer program stored in the HDD 203 of the image processing apparatus 100. explain. However, part of the processing of the flowchart shown in FIG. 11 may be executed by dedicated hardware.

  In step S1101, the command management unit 504 generates a command for controlling pan / tilt so that the zoom magnification of the photographing apparatus 101 is set to the wide end and the photographing region is moved to the lower left end of the count region. The generated command is transmitted to the photographing apparatus 102 via the communication unit 501.

  Next, in step S1102, the imaging region of the imaging device 102 is controlled to achieve the zoom magnification determined by the determination unit 505 by executing the control described in the flowchart illustrated in FIG.

  Next, in step S1103, the command management unit 504 performs control to capture an image after controlling the imaging direction of the imaging apparatus 102 to move rightward so that the counted area and the imaging area are adjacent to each other. Generate a command. The command generated at this time is transmitted to the imaging apparatus 102 via the communication unit 501.

  Next, in step S1104, the detection unit 502 detects a human body in the image photographed by the photographing apparatus 102 controlled in step S1104.

  Next, in step S1105, the count unit 503 counts the number of human bodies detected by the detection unit 502. At this time, the storage unit 507 stores the number of human bodies counted by the counting unit 503.

  Next, in step S1106, if the current shooting area of the shooting apparatus 102 has not reached the right end of the count area, the process proceeds to step S1103. If the shooting area has reached the right end of the count area, the process proceeds to step S1107. To do.

  In this way, the processing from step S1103 to step S1106 is repeated until the shooting area of the shooting apparatus 102 reaches the right end of the count area.

  In step S1107, if the current shooting area of the shooting apparatus 102 has not reached the upper end of the count area, the process proceeds to step S1108. If the shooting area of the shooting apparatus 102 has reached the upper end, the process shifts to step S1109. .

  In step S1108, the command management unit 504 moves the shooting direction so that the shooting area is positioned above the counted area and at the left end of the count area without changing the zoom magnification of the shooting apparatus 102. Generate a command to control. The command generated by the command management unit 504 is transmitted to the photographing apparatus 102 via the communication unit 501.

  As described above, the processing from step S1102 to step S1108 is executed until the upper end of the count area is reached.

  In step S1109, the count unit 503 calculates the total number of people counted in the captured image.

  In the present embodiment, the shooting order for shooting a plurality of areas is from the specific end (the front left end) to the end located on the diagonal (the back right end). Although the shooting area is moved in the raster direction, the present invention is not limited to this. For example, the imaging region of the imaging device 102 may be moved from the end on the right side on the near side to the left and finally moved to the end on the left side on the back side.

  In this embodiment, every time the image capturing apparatus 102 captures an image, the counting unit 502 counts the number of people. The images captured in a plurality of shooting directions are combined, and the number of people is counted with respect to the combined panoramic image. You may make it do.

As described above, according to the present embodiment, it is possible to automatically determine an appropriate zoom magnification in counting the number of people for a wider area. (Embodiment 2)
In the image processing apparatus 100 according to the first embodiment, for each of a plurality of images having different zoom magnifications at the time of shooting, the zoom magnification at the time of shooting an image with the largest number of counted persons is determined, and according to the determined zoom magnification. The number of people in the counting area was counted. However, even if the zoom magnification at the time of shooting an image with the largest number of people is counted, the human body size is small in the upper part of the image (the back side in the depth direction), so that it is detected by the detection unit 502. There may be no person.

  Therefore, in the image processing apparatus 100 according to the second embodiment, after specifying the zoom magnification that maximizes the number of people according to the processing according to the first embodiment shown in FIG. 9, the zoom magnification that further increases the zoom magnification by a predetermined ratio is determined. To do. Then, according to the determined zoom magnification, by controlling the imaging area of the imaging apparatus 102 with respect to the count area, the accuracy of human detection by the detection unit 502 is further improved, and as a result, the accuracy of counting people is further improved. be able to.

  Hereinafter, processing of the image processing apparatus 100 according to the second embodiment will be described. Note that functional configurations and processes that are the same as or equivalent to those of the first embodiment are denoted by the same reference numerals, and redundant descriptions are omitted as appropriate.

  Hereinafter, a zoom magnification determination process in a certain shooting direction by the image processing apparatus 100 according to the second embodiment will be described.

  FIG. 12 shows the position and size of the person detected by the detection unit 502 in the image taken at the zoom magnification with the maximum number of people specified by the processing of the flowchart shown in FIG. 9 according to the first embodiment. It is the shown schematic diagram.

  As shown in FIG. 12, when performing the counting of people for a wide area, the size of the person detected in the lower part of the image (front side with respect to the depth direction) is large and approaches the upper part of the image (back side with respect to the depth direction). As a result, the size of the detected person may become smaller. If the human body size that can be detected by the detection unit 502 is S pixels at this time, there is a possibility that there is a human body that is not detected by the detection unit 502 because the human body size is smaller than S pixels in the upper part of the image.

  Therefore, the determination unit 505 in the present embodiment executes the control described with reference to the flowchart shown in FIG. 9 to determine a zoom magnification that increases the zoom magnification at the time of shooting an image with the largest number of people by a predetermined ratio. Hereinafter, processing according to the present embodiment will be described.

  Human bodies 1201, 1202, and 1203 in FIG. 12A are human bodies detected by the detection unit 502. Note that the human body sizes of the human bodies 1201, 1202, and 1203 detected by the detection unit 502 are larger than the detectable size S, but are smaller than the threshold T (S + d). An area 1204 that includes only these human bodies is actually shown in the image, but there is a possibility that there is a human body that has not been detected because it is smaller than the detectable size S of the detection unit 502. .

  Note that the threshold T according to the present embodiment is a value obtained by adding a predetermined value d pixels to S pixels that are detectable sizes. The threshold value T may be set as appropriate by the user, or may be a preset value.

  A human body 1205 in FIG. 12B shows a human body that exceeds the threshold T for the first time when size comparison is performed sequentially from the top of the image.

  As shown in FIG. 12C, the determination unit 505 in the present embodiment determines the zoom magnification so that the upper end of the area 1206, which is an area obtained by excluding the area 1204 from the entire image area, and the upper end of the shooting area overlap. To do. In FIG. 12C, a shooting area that is shot at the zoom magnification determined at this time is an area 1207.

  Thus, based on the size of the human body included in the image, the zoom magnification is determined by increasing the zoom magnification of the photographing apparatus 102 that has captured the image with the largest number of people by a predetermined ratio. By controlling the photographing apparatus 102 according to the zoom magnification determined at this time, the accuracy of human detection by the detection unit 502 can be improved, and as a result, the accuracy of counting people can be improved.

  Note that there are cases in which there are human bodies that deviate from the angle of view, such as human bodies 1201, 1202, and 1203, by increasing the zoom magnification by a predetermined ratio. However, by executing the control of the flowcharts described with reference to FIGS. 10 and 11 and circulating the imaging area of the imaging apparatus 102 in the count area, it is possible to detect a human body that is out of the angle of view as a result.

  Next, a process in which the determination unit 505 according to the second embodiment determines the zoom magnification will be described with reference to the flowchart of FIG.

  As described with reference to FIGS. 10 and 11 in the first embodiment, before entering this flow, the zoom magnification of the image capturing apparatus 101 is appropriately controlled according to the control by the command management unit 504, and the image capturing apparatus 101 is further controlled. Is oriented in the desired shooting direction.

  In the present embodiment, the processing of the flowchart shown in FIG. 13 is executed by each functional block of FIG. 5 realized by the CPU 200 executing the computer program stored in the HDD 203 of the image processing apparatus 100. explain. However, a part of the processing of the flowchart shown in FIG. 13 may be executed by dedicated hardware.

  In step S901, the detection unit 502 detects a human body in an image captured by the imaging device 102. In the present embodiment, the storage unit 507 stores the human body size detected by the detection unit 502 and the position of the human body in the image.

  Since the processing from step S902 to step S906 is the same as that of the first embodiment, the description thereof is omitted.

  Next, in step S1301, the determination unit 505 determines an image in which the number of persons counted by the count unit 503 is maximized and a zoom magnification of the image capturing apparatus 102 when the image is captured.

  Next, in step S1302, the determination unit 505 determines the highest position in the image based on the position of the human body detected by the detection unit 502 and stored in the storage unit 507 in the image determined in step S1301. Pay attention to the human body.

  In step S1303, the determination unit 505 compares the size of the human body noted by the determination unit 505 in step S1302 with a preset threshold value.

  Next, in step S1304, when the focused human body size is equal to or smaller than the threshold, the process proceeds to step S1305. When the focused human body size exceeds the threshold, the process proceeds to step S1306.

  In step S1305, the determination unit 505 pays attention to the human body positioned at the top of the screen other than the human body that is the comparison target in step S1303. In step S1303, the determination unit 505 again compares the noticed human body size with a preset threshold value.

  In this manner, the processes in steps S1303 to S1305 are repeated, and the determination unit 505 identifies a person having a human body size that exceeds the set threshold value.

  In step S <b> 1306, the command management unit 504 determines the zoom magnification at which the human body that has exceeded the threshold value for the first time in step S <b> 1304 is positioned at the upper end of the imaging area of the imaging apparatus 102. At this time, the determination unit 505 determines that the imaging device 102 has an upper end of a circumscribed rectangle (not shown) indicating the position of the human body that exceeds the threshold from the zoom magnification specified in step S1301 and the upper end of the imaging region. It is assumed that the zoom magnification of is derived by calculation. Then, the determination unit 505 determines the derived zoom magnification as a zoom magnification to be used when photographing another imaging region.

  Since the processes in step S908 and steps S901 to S903 are the same as those in the first embodiment, description thereof is omitted.

  As described above, the determination unit 505 according to the present embodiment increases the zoom magnification of the image capturing apparatus 102 that has captured the image by a predetermined ratio based on the size of the human body in the image with the largest number of persons in a certain image capturing direction. Determine the zoom magnification.

  When performing the process of counting the number of people in the count area 600 as shown in FIG. 10 according to the zoom magnification determined at this time, the process of the flowchart shown in FIG. 13 according to the present embodiment is performed in step S1102 in the flowchart shown in FIG. I do. By controlling the photographing apparatus 102 according to the zoom magnification determined by the determination unit 505 according to the present embodiment, the accuracy of human body detection by the detection unit 502 is improved, and as a result, the accuracy of counting people can be improved. .

  Note that the determining unit 505 according to the present embodiment determines the size of the human body and the threshold value in the image with the maximum number of persons when determining the zoom ratio that is a predetermined ratio higher than the zoom ratio when the image with the maximum number of persons is captured. Although determined in consideration, it is not limited to this. For example, the determination unit 505 may use a predetermined ratio, which is the degree to which the zoom magnification is changed, without considering the human body size and the threshold, or a ratio appropriately set by the user May be used.

(Modification)
Although the processing according to the first and second embodiments described above has been described as being performed by the image processing apparatus 100, the present invention is not limited to this. The imaging apparatus 102 may have one or more functions of the image processing apparatus 100 according to the first embodiment or the second embodiment. That is, the processing according to the first or second embodiment is executed by the CPU 200 of the image processing apparatus 100, but may be realized by the cooperation of the CPU 210 of the photographing apparatus 102 and the CPU of the client apparatus.

  For example, the imaging device 102 may include the detection unit 502, the count unit 503, the command management unit 504, the determination unit 505, and the storage unit 507 illustrated in FIG. At this time, the display unit 506 and the operation receiving unit 508 are included in the client device 1400. The client apparatus 1400 is a personal computer having a hardware configuration equivalent to that of the image processing apparatus 100. This case will be described in more detail with reference to FIG. In addition, the same code | symbol shall be attached | subjected to the component and process which are the same or equivalent to Embodiment 1 and Embodiment 2, and the overlapping description is abbreviate | omitted suitably.

  FIG. 14 is an example of a functional block diagram of the photographing apparatus 102 and the client apparatus 1400 that have some functions of the image processing apparatus 100. The imaging apparatus 102 and the client apparatus 1400 communicate via the network 104.

  The imaging unit 305 includes an imaging element such as a CCD sensor or a CMOS sensor, and generates an electrical signal by photoelectrically converting a subject image formed through the lens 302 of the imaging apparatus 102.

  The image processing unit 306 performs processing for converting the electrical signal generated by the photographing unit 305 into a digital signal, compression encoding processing, and the like, and generates image data.

  The detection unit 502 detects a person in the image generated by the image processing unit 306.

  The command management unit 504 generates commands for controlling panning, tilting, zooming, and the like of the photographing apparatus 102 and controlling photographing operations such as control for photographing an image. The command generated by the command management unit 504 is analyzed by the system control unit 307, and the pan / tilt / zoom control unit 300 controls the photographing apparatus 102.

  The storage unit 507 stores the image generated by the image processing unit 306, the detection result of the detection unit 502, and the like. In addition, the storage unit 507 stores a shooting state of the shooting device 102 when an image processed by the detection unit 502 is shot.

  The operation reception unit 508 receives an operation performed by the user. For example, based on the command generated by the command management unit 504, the photographing apparatus 102 is controlled to repeat pan / tilt / zoom control and photographing with respect to a range in which the photographing apparatus 102 can photograph. The captured image is transmitted to the display unit 506 via the communication unit 309 and the communication unit 501, and the display unit 506 generates a panoramic image based on the transmitted image and causes the display device 101 to display the panoramic image.

  The operation reception unit 508 receives position information of the count area set by the user with respect to the panoramic image displayed on the display device 101 by the display unit 506. Information received by the operation receiving unit 508 at this time is sent to the storage unit 507 via the communication unit 501 and the communication unit 309.

  The functions of the count unit 503 and the determination unit 505 are the same as those in the first and second embodiments, and thus description thereof is omitted.

  The count result of the number of people in the count area counted by the count unit 503 is sent to the display unit 506 via the communication unit 309 and the communication unit 501, and the display unit 506 displays the count result on the display device 101. May be.

  As described above, the imaging apparatus 102 may have one or more functions of the image processing apparatus 100 according to the first embodiment or the second embodiment.

(Other embodiments)
The present invention can also be realized by a process in which one or more processors read and execute a program that realizes one or more functions of the above-described embodiment. The program may be supplied to a system or apparatus having a processor via a network or a storage medium. The present invention can also be realized by a circuit (for example, ASIC) that realizes one or more functions of the above-described embodiments.

501 Communication unit 502 Detection unit 503 Count unit 504 Command management unit 505 Determination unit 506 Display unit 507 Storage unit 508 Operation reception unit

Claims (14)

An image processing apparatus that counts the number of persons from an image captured by dividing a predetermined area into a plurality of imaging areas by an imaging unit,
Detecting means for detecting a person included in the photographed image;
Counting means for counting the number of persons detected by the detecting means;
Control means for controlling the photographing operation of the photographing means;
Determining means for determining a zoom magnification of the photographing means;
The counting means counts the number of persons for each of a plurality of images taken by changing the zoom magnification in stages,
The determining means determines a zoom magnification of the photographing means based on a zoom magnification at the time of photographing an image in which the number of persons counted by the counting means is maximized,
The image processing apparatus according to claim 1, wherein the control unit performs control so as to photograph another photographing region using the determined zoom magnification.   2. The determination unit according to claim 1, wherein the determination unit determines a zoom magnification at the time of shooting an image with the maximum number of people counted by the counting unit as a zoom magnification used when shooting the other shooting region. The image processing apparatus described.   When there are a plurality of images having the maximum number of persons counted by the counting unit, the determining unit determines the zoom magnification at the widest end side among the plurality of zoom magnifications when the plurality of images are captured. The image processing apparatus according to claim 2, wherein the image processing apparatus determines the zoom magnification used when shooting the shooting area.   The determining means determines a zoom magnification, which is a predetermined ratio higher than the zoom magnification at the time of shooting an image with the maximum number of people counted by the counting means, as a zoom magnification used when shooting the other shooting area. The image processing apparatus according to claim 1.   The determination unit is configured such that the person located at the top of the image is located at the upper end of the shooting area among the persons whose size of the person included in the image having the maximum number of people counted by the counting unit exceeds the threshold. The image processing apparatus according to claim 4, wherein an appropriate zoom magnification is determined as a zoom magnification used when photographing the other photographing region.   The control means controls the predetermined area to be photographed by moving the imaging area in a raster direction from a specific end in the predetermined area to an end located on a diagonal line. The image processing apparatus according to claim 1, wherein the image processing is performed separately. An imaging device that counts the number of persons from an image captured by dividing a predetermined area into a plurality of imaging areas,
Photographing means for photographing an image;
Detecting means for detecting a person included in the image photographed by the photographing means;
Counting means for counting the number of persons detected by the detecting means;
Control means for controlling the photographing operation of the photographing means;
Determining means for determining a zoom magnification of the photographing means;
The counting means counts the number of persons for each of a plurality of images taken by changing the zoom magnification in stages,
The determining means determines a zoom magnification of the photographing means based on a zoom magnification at the time of photographing an image in which the number of persons counted by the counting means is maximized,
The said control means is controlled to image | photograph another imaging area | region using the determined said zoom magnification. An image processing method for counting a number of persons from an image captured by dividing a predetermined area into a plurality of shooting areas by a shooting process,
A detection step of detecting a person included in the photographed image;
A counting step of counting the number of persons detected in the detection step;
A control step for controlling the photographing operation of the photographing step;
Determining a zoom magnification of the photographing step,
The counting step counts the number of people for each of a plurality of images taken by changing the zoom magnification in stages.
The determining step determines a zoom magnification of the photographing step based on a zoom magnification at the time of photographing an image in which the number of persons counted in the counting step is maximized,
The image processing method according to claim 1, wherein the control step performs control so as to image another imaging region using the determined zoom magnification.   9. The determination step according to claim 8, wherein the zoom magnification at the time of shooting the image with the maximum number of people counted in the counting step is determined as a zoom magnification used when shooting the other shooting area. The image processing method as described.   In the determination step, when there are a plurality of images with the maximum number of people counted in the counting step, the zoom magnification on the widest end side is set to the other zoom magnification when the plurality of images are captured. The image processing method according to claim 9, wherein the image processing method is determined as a zoom magnification used when photographing the photographing region.   In the determining step, a zoom magnification obtained by increasing the zoom magnification at the time of photographing an image with the maximum number of people counted in the counting step by a predetermined ratio is determined as a zoom magnification used when photographing the other photographing region. The image processing method according to claim 8.   In the determination step, among the persons whose size of the person included in the image having the maximum number of people counted in the counting step exceeds the threshold value, the person positioned at the top in the image is positioned at the upper end of the shooting area. The image processing method according to claim 11, wherein the zoom magnification is determined as a zoom magnification used when photographing the other photographing area.   The control step controls the predetermined area to be photographed by moving the imaging area in a raster direction from a specific end in the predetermined area to an end located on a diagonal line. The image processing method according to claim 8, wherein the image is shot separately.   A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.

Images