JP6736348B2 - Image processing apparatus, image processing method and program - Google Patents

Description

The present invention relates to an image processing device, an image processing method, and a program.

Conventionally, an object detection device that detects a human body or the like from an image is known. The object detection device is used, for example, to measure the number of spectators in a stadium or the number of visitors to an amusement park. Patent Document 1 discloses a technique for measuring the number of specific objects in an image regardless of the degree of congestion.

JP, 2014-6586, A

However, in the object detection, erroneous detection or non-detection may occur. When erroneous detection occurs, there is a problem that the number of individuals detected manually must be corrected.

The present invention has been made in view of such problems, and an object thereof is to easily correct the number of detected detection targets.

Accordingly, the present invention provides an image processing apparatus, and the image obtained by superimposing an acquisition unit configured to acquire an image captured, the first mark corresponding to the detected object is detected from the previous SL image by the imaging means, the and a display control means for displaying on the display unit and a number indicating the number of the first mark that is superimposed on the image, wherein the display control unit, and receives a user operation of adding a mark to be superposed on the image In this case, the image on which the first mark corresponding to the detection target and the second mark added according to the user operation are superimposed, and the number of the first mark and the second mark superimposed on the image The number corresponding to the total is displayed on the display means .

According to the present invention, the number of detected detection targets can be easily corrected.

It is a figure which shows an example of the hardware constitutions of an image processing apparatus. It is a figure which shows the software structure of an image processing apparatus. It is a flowchart which shows a detection process. It is a figure which shows an example of a target image. It is a figure which shows a screen transition. It is explanatory drawing of a user operation. It is explanatory drawing of the example of a change. It is explanatory drawing of the example of a change.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of a hardware configuration of an image processing apparatus 100 according to this embodiment. The image processing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, a hard disk drive (HDD) 204, a disk drive 105, a communication unit 106, an input unit 107, and a display unit 108.

The CPU 101 is a control device that integrally controls the image processing device 100. The ROM 102 stores a program for the CPU 101 to control the image processing apparatus 100. A secondary storage device may be provided instead of the ROM 102. The RAM 103 is a memory for the CPU 101 to expand the program read from the ROM 102 and execute the processing. The RAM 103 is also used as a temporary storage memory as a storage area for temporarily storing data to be processed. The HDD 104 stores various kinds of information such as an image file that is an image processing target of the image processing apparatus 100. An image file is input to the CPU 101 from the HDD 104. The image file may be input to the CPU 101 from the disk drive 105 or the communication unit 106 instead of the HDD 104. In this case, the CPU 101 does not have to have the HDD 104. The functions and processing of the image processing apparatus 100, which will be described later, are realized by the CPU 101 reading a program stored in the ROM 102 or the HDD 104 and executing the program.

The disk drive 105 reads an image file in a CD, a DVD, a Blu-ray disc, or a flexible disk, and the CPU 101 inputs the image file via the disk drive 105. When the CPU 101 inputs the image file from the HDD 104 or the communication unit 106, the disk drive 105 is not always necessary.

The communication unit 106 is a circuit that performs communication via the network 120. The CPU 101 inputs an image file via the network 120. When the CPU 101 inputs the image file from the HDD 104 or the disk drive 105, the communication unit 106 is not always necessary. The display unit 108 is a device such as a display that displays images and area frames. The input unit 107 includes a keyboard for inputting numerical values, a pointing device for instructing the display position of the display area of the display unit 108, a mouse, a touch panel, and the like.

FIG. 2 is a diagram showing a software configuration of the image processing apparatus 100. The image processing apparatus 100 includes a reception unit 201, a selection unit 202, a detection unit 203, a measurement unit 204, a display processing unit 205, and a file processing unit 206. The reception unit 201 receives various instructions input by the user via the input unit 107. The selection unit 202 selects an image to be processed from a plurality of images stored in a storage unit such as the HDD 104 according to a selection instruction from the user received by the reception unit 201. Hereinafter, the image to be processed will be referred to as a target image. The target image may be a still image or a moving image.

The detection unit 203 performs image analysis on the target image and detects a predetermined image. In the present embodiment, the case of detecting a person will be described as an example, but the detection target is not limited to a person, and may be a predetermined object such as a moving body such as a car. Further, the detection target is not limited to the object, and may be characters or symbols drawn on the object such as a car number. As described above, the detection target may be anything that can be detected from the image, and the type thereof is not limited to the embodiment. The detection unit 203 further records the detected position and size of the person in the storage unit such as the RAM 103. The method for detecting a person from an image is not particularly limited. For example, a method of recognizing a person by recognizing the face of the person, a method of recognizing the person from the size of the head of the person, a method of recognizing the person from the inclination of the shoulder, and the like can be given.

The measurement unit 204 measures the number of people (the number of people) detected from the target image by the detection unit 203. The display processing unit 205 controls to display various information on the display unit 108. The display processing unit 205 controls the display of various information on the display unit 108. The display processing unit 205 controls to display the target image on the display unit 108, for example. The display processing unit 205 further controls the display unit 108 to display the detection result of the detection unit 203. The file processing unit 206 creates and saves a target image and a result file including information such as the number of persons as the detection result obtained from the target image. The file processing unit 206 also appropriately outputs the result file to an external device or the like.

From the detection result displayed on the display unit 108 by the display processing unit 205, the user can confirm an area where a person other than a person is detected as a person, an area where a person is present but is not detected, and the like. .. Then, the user can appropriately input an instruction for correcting the detection result by a simple user operation. Upon receiving an instruction for correction from this user, the measurement unit 204 changes (corrects) the number of people as the measurement result according to this instruction. It should be noted that this processing will be described later in detail with reference to FIGS.

FIG. 3 is a flowchart showing the detection processing by the image processing apparatus 100. The detection process will be described below with reference to the target image 400 shown in FIG. 4 and the screen transition shown in FIG. The target image 400 shown in FIG. 4 shows nine persons 401. Here, for convenience of description, the persons from the lower left to the lower right will be referred to as a person A, a person B, a person C, and a person D in order. Furthermore, the persons from the upper left to the upper right are referred to as a person E, a person F, a person G, a person H, and a person I in order. In this embodiment, the image processing apparatus 100 is assumed to process an image captured by a surveillance camera installed on a wall or a pillar, for example. For this reason, the image to be processed is an image obtained by photographing a person in an oblique direction from above, and the person on the lower side of the image is larger than the person on the upper side of the image.

Returning to FIG. 3, in S300, the selection unit 202 selects a target image to be processed according to the image selection instruction received by the reception unit 201. For example, when the target image is a still image, the selection unit 202 specifies a still image file. Further, when the target image is a moving image, the selection unit 202 selects a target scene (frame) in the moving image file. For example, the display unit 108 reproduces and displays a live moving image or a recorded moving image, and when a desired scene is displayed, the user performs a user operation of designating this scene. On the other hand, the reception unit 201 receives the selection instruction in which the frame is designated.

Next, in S301, the detection unit 203 performs person detection. The detection unit 203 performs person detection, for example, by using a predetermined flat space such as the entire captured image or a predetermined region based on the center of the captured image as a detection target region. Then, the measurement unit 204 measures the number of people (the number of people) detected by the detection unit 203. Note that the measuring unit 204 may start the number of people when the receiving unit 201 receives a measurement instruction in response to a user operation.

Next, in S302, the display processing unit 205 controls to display the correction window. This process is an example of the display control process. FIG. 5A is a diagram showing an example of the correction window 500. The target image is displayed in the correction window 500. Further, on the target image, detection frames 501 to 509 are displayed in a superimposed manner on each area where a person is detected. Here, the detection frames 501 to 509 are examples of marks displayed corresponding to the detection target. For example, the detection frame 501 is displayed so as to be superimposed on the person A.

In addition, in the correction window 500 of FIG. 5A, the detection result when the person B is not detected as a person in the target image and a person is erroneously detected between the person C and the person D is displayed. It is shown. That is, the detection frame is not displayed for the person B, and the detection frame 503 is displayed between the person C and the person D.

Further, the number of people label 520 is displayed at the top of the correction window 500. The number of people measured by the measuring unit 204 is displayed on the number of people label 520. A save button 523 is displayed at the lower right of the correction window 500. The save button 523 is a user interface for receiving a save instruction to save the number of people as the measurement result together with the target image.

An addition mode button 521 and a deletion mode button 522 are displayed at the upper left of the correction window 500. Each of the buttons 521 and 522 is a user interface for receiving a start instruction of a corresponding mode. The addition mode and the deletion mode are operation modes for correcting the number of persons detected by the detection unit 203. In the addition mode, a person who is not detected can be added to the detection result according to the user operation, and in the deletion mode, a person (other than the person) involved in the false detection is detected from the detection result according to the user operation. Object) can be deleted.

In addition, in the present embodiment, as the mark indicating the detected person, a surrounding line surrounding the detected person is displayed, but the shape and size of the mark are not limited to the embodiment. .. Further, in the present embodiment, the mark is displayed on the area where the person is detected on the target image. However, if the mark is displayed so that the user can visually recognize the correspondence with the detected area. Well, it does not necessarily have to be displayed on the detected area. As another example, an arrow pointing to a person as the target image may be displayed. Further, as another example, the coordinates of the detected area may be displayed in an area different from the target image.

Returning to FIG. 3, after the processing of S302, the CPU 101 advances the processing to S303. In S303, when the accepting unit 201 accepts an instruction in response to a user input, the CPU 101 confirms the instruction content. When the instruction content is the additional mode selection instruction (additional mode selection instruction in S303), the CPU 101 advances the process to S304. When the instruction content is the deletion mode selection instruction (in S303, the deletion mode selection instruction), the CPU 101 advances the process to S307. When the instruction content is the save instruction (save instruction in S303), the CPU 101 advances the process to S310.

In S304, the reception unit 201 receives an instruction to add a new detection frame in response to a user operation. The addition instruction includes information that specifies the position and size of the detection frame to be added. Hereinafter, the detection frame newly added by the user operation will be referred to as an additional detection frame. FIG. 6 is an explanatory diagram of a user operation for designating the position and size of the additional detection frame. The user can specify the position and size of the additional detection frame by dragging the area to which the detection frame is to be added in the correction window 500. It is assumed that the user moves the mouse down at the point 601 and moves the mouse up at the point 602. In this case, the reception unit 201 specifies a rectangle having the points 601 and 602 as diagonals, and receives an additional instruction including the position and size of the ellipse 603 inscribed in the specified rectangle.

Next, in step S305, the display processing unit 205 displays the additional detection frame related to the addition instruction in the correction window 500. Here, the additional detection frame is an example of an additional mark. Next, in S306, the measuring unit 204 increases the number of persons by the number of additional detection frames added to the correction window 500 with respect to the measurement result. Then, the display processing unit 205 displays the number of detection frames after addition, that is, the number of corrected persons on the number-of-persons label 520. The CPU 101 then advances the process to S303.

For example, in the correction window 500 of FIG. 5A, it is assumed that the user presses the addition mode button 521 and designates a rectangle surrounding the person B. In this case, as shown in FIG. 5B, the additional detection frame 530 is added around the person B. Further, as shown in FIG. 5B, the number of people in the number-of-people label 520 increases from “9” in FIG. 5A to “10”.

Returning to FIG. 3, in S307, the reception unit 201 receives an instruction to delete the detection frame displayed in the correction window 500 according to a user operation. The deletion instruction includes information designating the detection frame to be deleted among the detection frames displayed in the correction window 500. For example, the user moves the cursor into the detection frame 503 displayed in the correction window 500 of FIG. 5B and clicks it. On the other hand, the reception unit 201 specifies the detection frame 503 including the clicked point as a deletion target and receives a deletion instruction of the specified detection frame 503. When the clicked point falls within the plurality of detection frames, all of the plurality of detection frames are designated as deletion targets.

Next, in S308, the display processing unit 205 deletes the detection frame related to the deletion instruction from the correction window 500. Next, in S309, the measurement unit 204 reduces the number of persons in the measurement result by the number of detection frames deleted from the correction window 500. Then, the display processing unit 205 displays the reduced number of detection frames, that is, the corrected number of persons on the number-of-persons label 520. The CPU 101 then advances the process to S303. For example, when the detection frame 503 is designated in the correction window 500 of FIG. 5B, the detection frame 503 is deleted as shown in FIG. 5C. Further, the number of people in the number of people label 520 is reduced from “10” in FIG. 5B to “9”.

Further, in S310, the file processing unit 206 causes the storage image such as the HDD 104 to store, as a result file, the superimposed image on which the detection frame is displayed, which is displayed in the correction window 500, and the number of people displayed on the number of people label 520. save. This is the end of the detection process. The file processing unit 206 can also output the result file to the outside. It should be noted that the file processing unit 206 may store the result file including at least the detected number of people (when corrected, the number of people after correction). The file processing unit 206 may store the target image, information indicating the position of the detection frame, and the detected number of people as a result file, for example.

As described above, the image processing apparatus 100 according to the present embodiment performs correction processing such as addition and deletion of detection frames according to a user operation on the display screen of the detection result of a person, and reflects the result of the correction processing. The number of detected detection targets can be saved and output. That is, the image processing apparatus 100 can easily correct the number of detected detection targets without requiring a complicated operation by the user.

A modification of the image processing apparatus 100 according to the embodiment will be described. The processing for determining the position and size of the additional detection frame in the additional mode is not limited to the embodiment. For example, the user may input an additional instruction that specifies only the position, instead of the position and the size. A case in which a detection frame is desired to be added to the person B will be specifically described with reference to FIG. 7. The user clicks the center point 701 of the person B. The center point 701 may be any point near the center. On the other hand, the display processing unit 205 receives an additional instruction specifying the center point 701 as the position. Further, in this case, the display processing unit 205 automatically determines the size of the additional detection frame. The processing for determining the size includes the following. Note that this process is an example of the size determination process.

The first is a process of setting a predetermined size as the size of the additional detection frame. That is, the display processing unit 205 displays a detection frame having a predetermined size as an additional detection frame at the position related to the addition instruction.

The second is a process of setting the size of the additional detection frame to the same size as the detection frame existing at the position closest to the specified position. The display processing unit 205 identifies the distance between the position related to the addition instruction and the center point of each of the surrounding additional detection frames. Then, the display processing unit 205 displays the additional detection frame having the same size as the size of the detection frame existing at the closest position.

The third is a process of setting the average value of the sizes of the detection frames having the center point within a predetermined distance range from the designated position as the size of the additional detection frame. The display processing unit 205 identifies the detection frame in which the center point exists within a predetermined distance range from the position related to the addition instruction. Then, the display processing unit 205 obtains an average value of the sizes of the specified detection frames, and displays the additional detection frame having the size of this average value. For example, in the example of FIG. 7, it is in the detection frames 501 and 502 of the person A and the person C that the center point exists in the reference range 710 that is the predetermined distance range. Therefore, the display processing unit 205 displays the additional detection frame 700 having the average size of the detection frames 501 and 502.

The fourth is a process of determining the size of the additional detection frame according to the tendency of the sizes of the plurality of detection frames in the target image. For example, when the center point 801 of the person B is designated as in the example of FIG. 8, the display processing unit 205 first specifies the sizes of all the detection frames displayed in the correction window 500, and according to the size. Group detection windows. For example, the display processing unit 205 obtains the number of groups from the distribution of the sizes of the plurality of detection frames, and specifies the detection frames that belong to each group. Then, a line connecting the center points of the detection frames belonging to each group is specified. This processing is an example of line identification processing. Then, the display processing unit 205 selects, from the identified lines, a line whose distance from the specified position is equal to or less than the distance threshold, and determines the average value of the sizes of the detection frames connected by the selected line as the additional detection frame. To determine the size of. In this way, the display processing unit 205 selects one line based on the positional relationship between the specified line and the specified position, and based on the size of the detection frame connected by the selected line, the additional detection frame Determine the size.

In the example of FIG. 8, the detection frames 501, 502, 503, 504 are grouped, and a straight line 811 passing through the center points of the detection frames 501, 502, 503, 504 is specified. Further, the detection frames 505, 506, 507, 508, 509 are grouped, and a straight line 812 passing through the center points of the respective detection frames 505, 506, 507, 508, 509 is specified. It is assumed that the designated position (center point 801) is a predetermined distance or less from the straight line 811. Further, it is assumed that the average size of the detection frames 505, 506, 507, 508, 509 is “5”, whereas the average size of the detection frames 501, 502, 503, 504 is “10”. .. In this case, the display processing unit 205 determines the size of the additional detection frame to be “10”.

Further, in this case, the display processing unit 205 may determine the position moved from the position designated by the user to the position on the straight line 811 (the position on the line) as the center point of the additional detection frame. In this case, the display processing unit 205 displays the additional detection frame 800 of size “10” with the determined position on the straight line 811 as the center point. In addition, as another example, it may be known that the center point is not located on the straight line 811 in consideration of the positional relationship with other persons. In such a case, the display processing unit 205 may determine a position where the distance between the straight line 811 and the center point is as short as possible as the center point of the additional detection frame. Also in this case, the display processing unit 205 displays the additional detection frame 800 with the determined position as the center point.

Further, the display processing unit 205 may move the display of the additional detection frame 800 by animation or the like according to the straight line 811. For example, the display processing unit 205 first displays the additional detection frame 800 of size “10” at the position designated by the user. Then, the display processing unit 205 moves the additional detection frame 800 from the position designated by the user to a position on the straight line 811 or a position where the distance to the straight line 811 becomes short.

As described above, according to the above-described embodiments, the number of detected detection targets can be easily corrected.

Although the preferred embodiments of the present invention have been described above in detail, the present invention is not limited to the specific embodiments, and various modifications are possible within the scope of the gist of the present invention described in the claims. -Can be changed.

(Other embodiments)
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. It can also be realized by the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 image processing apparatus 101 CPU
107 input unit 108 display unit 203 detection unit 205 display processing unit

Claims (19)

Acquisition means for acquiring an image captured by the imaging means,
Yes and the image obtained by superimposing the first mark corresponding to the detected object is detected from the previous SL image, and a display control means for displaying on the display unit and a number indicating the number of the first mark superimposed on said image Then
When the display control means receives a user operation of adding a mark to be superimposed on the image, the first mark corresponding to the detection target and the second mark added according to the user operation are superimposed. An image processing apparatus for displaying an image and a number corresponding to the total number of the first mark and the second mark superimposed on the image on the display means . When the display control unit receives a user operation that designates at least one of the first mark and the second mark superimposed on the image as a target to be deleted, the display control unit specifies the first target to be deleted. The number of the first mark and the second mark superimposed on the image other than the first mark or the second mark designated according to the user operation, and the image in which the mark or the second mark is deleted. The image processing apparatus according to claim 1, wherein a number corresponding to the total is displayed on the display unit. Wherein the display control unit, an image processing apparatus according to claim 1 or 2, characterized in that for displaying the image of the first mark is superimposed at a position corresponding to the detection target on the image on the display means .. The size of the second mark to be added in accordance with user operation, the image processing apparatus according to any one of claims 1 to 3, characterized in that it is the size specified by the user. The added according to a user operation size of the second mark, the image processing apparatus according to any one of claims 1 to 3, characterized in that it is a predetermined size. When the position on the front Symbol image accepts a user operation of adding the specified second mark, the size of the first mark which exists at a position closest to the position on the image of the second mark Further having a size determining means for determining the size,
Wherein the display control unit, an image processing apparatus according to any one of claims 1 to 3, characterized in that superimposes the second mark of the determined size by said size determining unit on the image. When receiving a user operation of adding the second mark position on the front Symbol image is specified, based on the size of the first mark present in the reference range with respect to the position on the image, Further comprising size determining means for determining the size of the second mark,
Wherein the display control unit, an image processing apparatus according to any one of claims 1 to 3, characterized in that superimposes the second mark of the determined size by said size determining unit on the image. The image processing apparatus according to claim 7 , wherein the size determination unit determines an average value of the sizes of the first marks existing in the reference range as the size of the second marks. The first mark in the image, grouped according to the size of the first mark, the line identifying means for identifying a line connecting the first mark of each group,
When the position on the front Symbol image accepts a user operation of adding a second mark that is specified by selecting one of the lines on the basis of said position on said image, the positional relationship between the line, A size determining unit that determines the size of the first mark connected by the selected line as the size of the second mark,
Wherein the display control unit, an image processing apparatus according to any one of claims 1 to 3, wherein the displaying the second mark of size determined by the size determination means. Wherein the display control unit, said determining the position of the selected lines by size determination means as the position of the center point of the second mark based on the position of the center point, characterized in that for displaying the second mark The image processing apparatus according to claim 9 . The display control unit determines, as the position of the center point of the second mark, a position such that the distance to the line selected by the size determination unit becomes shorter than a position determined according to a user operation, The image processing apparatus according to claim 9 , wherein the second mark is displayed based on the position of the center point. Wherein the display control unit, characterized in that the position on the front Symbol image when receiving a user operation of adding a second mark that is designated to display the second mark in correspondence to said designated location The image processing apparatus according to any one of claims 1 to 11 . Acquisition means for acquiring an image captured by the imaging means,
A display control unit that causes the display unit to display the image on which the first mark corresponding to the detection target detected from the image is superimposed and the number corresponding to the number of the first marks superimposed on the image. Then
When the display control unit receives a user operation that specifies at least one of the first marks superimposed on the image as a target to be deleted, the user operation of the first marks superimposed on the image is performed. The image in which the first mark designated in accordance with the above is deleted, and a number corresponding to the number of the first marks superimposed on the image other than the first mark designated in accordance with the user operation. An image processing device characterized by being displayed on. Further comprising detection means for detecting the detection target from the image,
Wherein the display control unit, an image according to any one of claims 1 to 13, characterized in that superimposed the first mark in correspondence with the detection target detected by the detecting means on the image Processing equipment. The detection target is an image processing apparatus according to any one of claims 1 to 14, characterized in that it is a object. The image processing apparatus according to claim 15 , wherein the detection target is a person. An acquisition step of acquiring an image captured by the imaging means,
And the image obtained by superimposing the first mark corresponding to the detected object is detected from the previous SL image, a first display control step of displaying on the display unit and a number indicating the number of the first mark superimposed on said image ,
When a user operation of adding a mark to be superimposed on the image is accepted, the image on which the first mark corresponding to the detection target and the second mark added according to the user operation are superimposed, and the image. A second display control step of causing the display means to display a number corresponding to the total number of the first marks and the second marks that have been superimposed.
Image processing method, which comprises a. An acquisition step of acquiring an image captured by the imaging means,
A first display control step of displaying on the display means the image on which the first mark corresponding to the detection target detected from the image is superimposed, and the number corresponding to the number of the first marks superimposed on the image. ,
When a user operation of designating at least one of the first marks superimposed on the image as a target to be deleted is received, the first mark of the first marks superimposed on the image designated according to the user operation is received. A second display that causes the display means to display the image in which one mark is deleted and a number corresponding to the number of the first marks superimposed on the image other than the first mark designated according to the user operation. Control process
An image processing method comprising: A program for causing a computer to function as each unit of the image processing apparatus according to claim 1.

Images