JP2019204374A - Object detecting apparatus - Google Patents

Abstract

To provide an object detecting apparatus which can increase visibility of a detection target in captured continuous images.SOLUTION: As to a detection target in captured continuous images, a frame with a size determined based upon a position of an image region is drawn in an image of each frame by setting, as a start point, a reference position for use in determination of a position of a detected object. Thus, fluctuations in position and size of a frame drawn in the captured continuous images are inhibited so as to allow the detection object as an image with a frame to have high visibility.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an object detection apparatus that detects an object to be detected based on a captured image.

  For example, a face detection frame or face detection frame data is output for a detected face (face image) on the assumption that an image of a human body or a part of the face is detected from the captured image and image processing is performed (Patent Document) 1 and 2) are known.

  For example, in Patent Document 1, the face detection frame data is associated between video frames, and in Patent Document 2, the size of the cut face image is changed and combined with the content image. ing.

  Regarding the face detection and person detection in Patent Documents 1 and 2, for example, it is considered that the screen is scanned and the person is identified based on the score of the human detector. When the method is adopted, for example, depending on various imaging conditions such as the sunshine conditions and the distance from the imaging position to the detection target, the position and size of the frame output according to the detection target in the captured continuous image varies. May occur and visibility may deteriorate.

JP 2012-94103 A JP 2014-67117 A

  The present invention has been made in view of the above points, and an object of the present invention is to provide an object detection device that can increase the visibility of a detection target in a captured continuous image.

  An object detection apparatus for achieving the above object includes a reference position determination unit that determines a reference position for a partial image to be detected in a continuous image, and a partial image to be detected by scale conversion determined for each position in the image. The frame conversion frame setting unit that determines the size of the frame that surrounds the frame, and the frame determined by the setting in the scale conversion frame setting unit with the reference position determined by the reference position determination unit as the base image And a frame drawing unit for drawing.

  In the above object detection apparatus, a frame whose size is determined for each position of the image range is drawn on the image for each frame from the reference position of the partial image to be detected in the continuous image as a base point. The variation of the position and size of the frame drawn in FIG. 3 can be suppressed, and the detection target as an image with a frame can be made highly visible.

  In a specific aspect of the present invention, the scale conversion frame setting unit sets the frame size at each position in the image to the distance from the corresponding actual position to the imaging position in the image captured from the fixed position. Determine based on. In this case, a frame of an appropriate size according to the distance can be displayed in the image.

  In another aspect of the present invention, a human detection unit that performs human detection using a human image as a detection target is provided, and the reference position determination unit determines the position of the foot of the human image specified by the human detection unit. The reference position to be determined. In this case, a person can be set as an object to be detected. For example, the visibility of the movement of a person on the station premises can be improved.

  In still another aspect of the present invention, the frame set in the scale conversion frame setting unit has a rectangular shape corresponding to the ratio of people as detection targets. In this case, the size and shape as a frame for capturing the person who is the detection target can be made appropriate.

  In still another aspect of the present invention, the image processing apparatus includes a tracking processing unit that associates images between frames in continuous image data and tracks the same object, and the frame drawing unit responds to the association of the same object in the tracking processing unit. Then, it is determined whether or not to draw a frame. In this case, the object captured by the tracking processing unit can be reliably identified, and an appropriate frame can be drawn accordingly.

  In still another aspect of the present invention, the frame drawing unit does not draw a frame for a new detection target. In this case, even if the object other than the target object is caught due to erroneous detection, it is possible to prevent the frame from being drawn on the erroneously detected object.

  In yet another aspect of the present invention, the tracking processing unit re-submits a partial image corresponding to a predetermined number of frames even if the partial image of the detection target is temporarily undetected between frames in the continuous image. If detected, they are associated and handled as the same object. In this case, it is possible to continue drawing the frame on the detected object even if it is temporarily undetected.

  In still another aspect of the present invention, the frame drawing unit is based on at least one of the continuous detection number, the cumulative detection number, and the continuous non-detection number between detection target frames associated as the same object in the tracking processing unit. Decide whether to draw a frame. It is possible to maintain an appropriate state for drawing the frame on the object.

1 is a block diagram conceptually showing an object detection apparatus according to an embodiment. It is a conceptual diagram which shows the mode of the station which deployed the object detection apparatus. It is a conceptual top view which shows the mode of the station which provided the object detection apparatus. (A)-(C) are conceptual image figures which show an example about the mode of the person tracked. It is a conceptual image figure for demonstrating regarding the detection in a person detection part. (A) And (B) is a conceptual image figure for demonstrating an example about the setting of a frame, (C) is a conceptual diagram which shows the relationship between the size of a frame, and the position on an image, (D) is a graph for conceptually explaining a method for setting the frame size and the position on the image. It is a flowchart for demonstrating an example of the whole operation | movement of an object detection apparatus. (A)-(C) is a conceptual image figure for demonstrating each aspect of the correlation between frames. It is a conceptual image figure for demonstrating an example of a tracking process. 10 is a time chart showing another example of the result of association between frames in the tracking process shown in FIG. 9. It is a flowchart for demonstrating an example about the process of data management.

  Hereinafter, an example of the object detection apparatus according to the present embodiment will be described with reference to FIG. FIG. 1 is a block diagram conceptually showing an object detection device 100 according to the present embodiment, and FIGS. 2 and 3 are conceptual diagrams showing a state of a station ST in which the object detection device 100 is provided. FIG. 2 shows a state where the train TR has stopped on the track TK, and FIG. 3 omits drawing of the train on the track TK. Here, as an example, the platform (platform) PF of the station ST where the train TR can arrive and depart is set as the monitoring target area by the object detection device 100. That is, the user HU in the station PF's home PF is set as an object detection target, and a partial image obtained by capturing the user HU as the detection target is extracted so that the position of the user HU in the home PF can be understood. A partial image, that is, a frame FR surrounding a person in the image is displayed (see FIG. 4). Thus, for example, when a passenger HU gets on and off the train TR, etc., the crew member or station staff can more easily recognize the presence position of the person, that is, the user HU based on the frame when performing safety confirmation via the display image. I have to. 2 and 3, the traveling direction of the train TR is the + Z direction, and in the plane perpendicular to the Z direction, the vertical direction, that is, the vertical direction is the Y direction, and the direction is perpendicular to both the Z direction and the Y direction. Let a certain horizontal direction be the X direction.

  In order to achieve the above object, the object detection apparatus 100 performs detection of a person who is a detection target in the station premises of the user HU (particularly on the home PF) based on the captured image, and accurately detects the position of the detected person. In addition, various processes such as setting and display of a frame FR (see FIG. 4) according to the size and position of the detected person on the image and processing of the image are performed. Here, as a representative example, as shown in FIG. 2 and FIG. 3, a state that can be monitored by the station clerk SA by confirming the presence of the user HU on the home PF and displaying various processed images. I have to. In an example here, as shown in FIG. 2, a display unit DA such as a monitor is provided in the vicinity of the stop position of the train TR so that it can be confirmed by a crew member (driver or conductor) of the train TR. Yes.

  Hereinafter, with reference to FIG. 1 etc., the structure of the object detection apparatus 100 in this embodiment, etc. are demonstrated concretely. The object detection apparatus 100 according to the present embodiment includes an imaging unit (imaging camera) 10 that images a monitoring target area in order to detect a user HU, that is, a person moving on a home PF, and image data acquired by the imaging unit 10. The control device 50 that performs various processes and controls to detect a visually impaired person based on the above, and the display display unit DA that performs image display based on the processing results in the control device 50 are provided.

  The imaging unit 10 is an imaging camera composed of an individual imaging element such as a CCD or a CMOS, for example, and is fixedly installed on the upper side (+ Y side) of the home PF as illustrated in FIGS. 2 and 3. The lower side (the direction having a component on the −Y side) is imaged. That is, the imaging unit 10 performs imaging for fixed point observation of the monitoring target area from the upper side toward the floor surface FL of the home PF. Here, in order to enable tracking processing for associating images between frames in continuous image data and tracking the same person, two-dimensional image data is acquired as continuous image data, for example, at a frame rate of 30 fps. . Note that one or more imaging units 10 are provided depending on the installation position, the shape of the home PF, and the like so as to enable imaging of the entire home PF.

  The control device 50 is configured by, for example, a CPU, a storage device, and the like, performs various processes on the continuous image data acquired by the imaging unit 10, and displays an image as a processing result on the display display unit DA. In order to enable the above processing, as illustrated in FIG. 1, the control device 50 includes a main control unit 60, a storage unit 70, and an image processing unit 80.

  The main control unit 60 includes, for example, a CPU and the like, and includes a human detection unit 61 that detects the presence of a person, a tracking processing unit 62 that tracks the same person, or a reference that determines a reference position for a detected partial image of the person. The position determination unit 63, and further, the scale conversion frame setting unit 64 that determines the size of a frame that surrounds the detected partial image of the person, and the image processing unit 80 draw a frame determined by the setting in the scale conversion frame setting unit 64. It functions as a frame drawing unit 65.

  The human detection unit 61 performs various types of information processing to extract a human image (partial image in the image) from the continuous image data that is a two-dimensional image acquired by the imaging unit 10 by shape extraction by machine learning.

  The tracking processing unit 62 associates images between frames in the continuous image data acquired from the imaging unit 10, thereby enabling tracking of the same person with respect to the person image extracted by the person detection unit 61. For this reason, the tracking processing unit 62 reads out a program from the storage unit 70 as appropriate, and performs processing for associating images between frames in the continuous image data for each human image detected or extracted by the human detection unit 61.

  The reference position determination unit 63 determines a reference position for a partial image of a person who is a detection target in the continuous images detected by the person detection unit 61. Here, as an example of the reference position, the position of the foot of the person image specified by the person detection unit 61 is employed. That is, in the present embodiment, the position of the person is obtained by setting the coordinates of one point (one pixel) indicating the position of the foot in the two-dimensional plane image as a reference position indicating the position where the detected person exists. Is identified.

  The scale conversion frame setting unit 64 determines the shape and size of the frame FR that surrounds the partial human image as illustrated in FIG. 4 and the like for the partial human image detected by the human detection unit 61. In this example, an image of a rectangular frame FR corresponding to the ratio of people can be displayed with the position of the foot of the person image determined by the reference position determination unit 63 as a base point. That is, in this embodiment, the size of the rectangular frame is determined based on the position of the foot.

  Here, as a premise for determining the shape and size of the frame FR in the scale conversion frame setting unit 64, in the present embodiment, a predetermined scale conversion is performed for each position in the image captured from the fixed position by the imaging unit 10. Has been made. For example, in the home PF of the station ST, when performing analysis related to human detection for an image in a certain range from a fixed position captured by the imaging unit 10, the floor surface of the home PF captured from the imaging position, that is, the installation position of the imaging unit 10 The distance to the FL is determined for each position in the image. In other words, in this case, the imaging distance to the floor surface FL can be determined in advance in units of pixels constituting each captured image (for each plane coordinate in the image). By using such information related to the distance, the scale conversion frame setting unit 64 sets the size of the frame FR to be displayed at each position in the image in the image captured from the fixed position by the imaging unit 10. Can be determined in advance according to the imaging distance. As a result, a frame FR of an appropriate size can be displayed in the image according to the actual distance. An example of a specific method for scale conversion as described above will be described later with reference to FIG.

  The frame drawing unit 65 uses the reference position determined by the reference position determination unit 63, that is, the position of a person's foot as a base point, and the frame determined by the setting in the scale conversion frame setting unit 64 is an image for each frame constituting the continuous image. To draw. That is, various processes for drawing the frame are performed by the image processing unit 80 on the human image on each image. Note that details regarding frame drawing, such as determination of whether or not to perform frame drawing performed in the frame drawing unit 65, will be described later with an example.

  Next, in the control device 50, the storage unit 70 is configured by a storage device or the like, stores various data and programs, and the main control unit 60 reads out these data and the like as appropriate. For this reason, the storage unit 70 includes a human feature amount information storage unit 72, a scale conversion information storage unit 73, and the like in addition to a tracking data storage unit 71 that stores data related to the same person in the tracking processing unit 62. Configured.

  Of the storage unit 70, the tracking data storage unit 71 includes a person storage unit 71a that allocates an ID to each detected person to identify a plurality of detected persons and manages image data and the like for each ID. Further, the person storage section 71a is provided with a reference position data storage section 71b for storing data related to the reference position for each partial image data of the person. That is, in the reference position data storage unit 71b, the coordinates of the foot of the partial image of the person are stored as the reference position of the person's presence position. Thus, the control device 50 can display a rectangular frame having a size and size corresponding to the reference position in the image.

  Of the storage unit 70, the human feature quantity information storage unit 72 is necessary for extracting a human image by shape extraction by machine learning using a known technique such as an HOG feature quantity for human detection by the human detection unit 61. Stores various information. Furthermore, various programs for extracting person images may be stored.

  In addition, the scale conversion information storage unit 73 in the storage unit 70 stores information about scale conversion for frame setting. A specific example of stored information will be described later.

  Next, in the control device 50, the image processing unit 80 is configured by, for example, a GPU or the like, and is acquired by the imaging unit 10 in accordance with an instruction from the control device 50 to enable the various image processing as described above. Various image processing is performed on each two-dimensional image data. In particular, in the present embodiment, the image processing unit 80 performs various image processing operations to display the image captured by the imaging unit 10 on the display display unit DA, and as described above, the frame drawing unit. In accordance with an instruction from 65, the frame determined by the scale conversion frame setting unit 64 is drawn on the image for each frame constituting the continuous image using the reference position determined by the reference position determination unit 63 as a base point.

  Finally, in the control device 50, the display unit DA is a monitor that displays an image picked up by the image pickup unit 10. As a typical example, as described above, a crew member of a train TR (a conductor or a driver). As shown in FIG. 4, it is provided in the vicinity of the rear end side or the front side of the train TR when the train TR stops in the home PF. In other words, after the train TR arrives at the station ST, the passenger (user) can monitor the passenger (user) until the passenger (user) gets on and off and departs. In addition to the case of illustration, it is good also as an aspect installed in a station service room etc. and alerting | reporting with respect to the station staff in a station service room etc., for example.

  Hereinafter, an example of the processing operation as the human detection unit 61 of the main control unit 60 among the above-described units constituting the control device 50 will be described in more detail.

  As the human detection unit 61, the main control unit 60 can confirm whether or not there is an image having the characteristics of a person (person) in each two-dimensional image data acquired by the imaging unit 10. Yes. For example, by using a known HOG feature quantity or support vector machine (SVM), it is possible to extract a person image from a two-dimensional image data by shape extraction by machine learning. That is, the human detection unit 61 extracts a straight line portion or the like from each image data acquired by the imaging unit 10, and has a characteristic unique to the person in the image data based on the extracted feature amount or the like of the straight line portion. It is judged whether there is what has been done. In other words, the person detection unit 61 classifies persons and non-persons in the image data. Thereby, the person detection part 61 extracts a person image based on the determination result. For this reason, the person detection unit 61 reads out a necessary program from the storage unit 70 as appropriate, and detects the presence of a person using the feature quantity data stored in the person feature quantity information storage unit 72. Yes. By using the HOG feature amount or the like, human detection can be performed even from a two-dimensional image.

  Here, when human detection is performed by the existing method as in the above example, the human detection unit 61 that is a human detector scans the screen according to a predetermined method based on, for example, the HOG feature amount. By assigning a score based on the index indicating humanity at each point or region and searching for a region with a high score, an image region to be detected as a person is determined and extracted as a partial image of the person image. However, when such a method is employed, for example, depending on various imaging conditions such as the sunshine conditions and the distance from the imaging position to the detection target, the position of the area having the best score (hereinafter referred to as the best score area). The size, range, and range do not necessarily correspond exactly to the size and position of a person on the image to be detected. That is, depending on the image data, for example, the best score area is larger than the image portion of the person to be captured, or on the contrary, becomes a small area, or the position is shifted. Therefore, if the best score area captured by the method as described above is used as a frame display showing a person image as it is, the position and size of the frame output depending on the detection target in the captured continuous image will fluctuate. May occur and visibility may deteriorate. On the other hand, in the present embodiment, in the case where the person detection unit 61 specifies the person detection unit 61, a reference position that defines the position of the detected person is further provided, and the reference position is used as a base point for the image range. By drawing a frame with a size defined for each position in an image for each frame, the variation in the position and size of the frame drawn in the captured continuous image is suppressed, and the detection target is an image with a frame. The human image is made highly visible.

  4 (A) to 4 (C) are conceptual diagrams illustrating an example of the state of a person to be tracked, and a frame FRx that displays the best score region by human detection as it is, and the present embodiment. Both the frame FR as a result of the processing are shown, and these can be compared. For example, FIG. 4A and FIG. 4B are obtained by imaging the same place with a difference of several frames (that is, a difference of several hundred milliseconds), but the person PE1 on the image of the same person. When the frame FRx that surrounds the frame FRx is compared with the frame FRx, the size of the frame FRx that displays the best score region in human detection as it is varies greatly, but the result of performing the processing in this embodiment It can be seen that the frame FR as an object accurately surrounds the target person PE1 with a small variation in size. Similarly, in FIG. 4C showing the imaging results at different times, the frame FRx is very large with respect to the person PE2 on the image to be detected, whereas the frame FR It can be seen that the size is suitable for the size of the person PE2 on the image.

  Hereinafter, the reason why the variation in the size of the rectangular frame or region in the conventional human detector will occur will be briefly described with reference to FIG. In general, in the conventional human detector as described above, when calculating a score indicating an index of whether or not the image is a human image, a rectangular image size serving as a reference for calculation is determined in advance as one, As described above, the score is calculated after the partial image PI to be calculated cut into a rectangular shape is enlarged or reduced to match the reference image size. For this reason, the area where the score calculation result is the best due to the size and enlargement / reduction ratio of the clipped image, or the brightness of the image of the part other than the person included when cutting into a rectangle, It is not necessarily an area that is appropriately extracted so as to include only people, for example, it includes people but also includes many areas other than people. It may be missing. Depending on the case, there is a possibility that an object other than a person is captured as a human image. In contrast, in the present embodiment, various methods are provided to deal with these problems.

  Hereinafter, a specific example of setting the frame FR in the present embodiment will be described with reference to FIG. Here, as an image of the home PF (see FIGS. 2 and 3) imaged from the fixed position in the imaging unit 10, the range of the image FLi for the floor surface FL of the image GI shown in FIG. 6 (A). The distance to the imaging unit 10 at each point (pixel) is set. Based on this, it is possible to set the size of the frame FR as the reference position at each point (pixel) of the image FLi, that is, the position of the foot. Here, it is assumed that the floor surface FL of the home PF has one end as a boundary with the line TK and the other end as a boundary with a wall opposite to the line TK. That is, in the case of illustrating the image GI of FIG. 6A and the image GIa of 6B, the setting is made for the range from the boundary BD1 with the image TKi of the line TK to the boundary BD2 with the wall image WLi. Will do.

  First, as shown in FIG. 6A, the shapes and sizes of the rectangular frames FR at the known reference positions SP1 and SP2 that are different from each other in the distance to the imaging unit 10 that has captured the image GI and are at known positions. Set. The shape and size of the frame FR at the known reference positions SP1 and SP2 and the known reference positions SP1 and SP2 may be determined in advance, but may be actually measured from the imaging unit 10, for example. As a specific example, real dolls DL1 and DL2 having a standard human size (for example, 170 cm) are placed at a predetermined distance, and these are picked up by the image pickup unit 10 while directly viewing the picked-up images. The positions of the feet of the dolls DL1 and DL2 can be determined as known reference positions SP1 and SP2 on the image, and further, the position, shape and size of the rectangular frame FR with reference to the known reference positions SP1 and SP2 can be determined. . In this case, for example, the known reference positions SP1 and SP2 that are the positions of the feet are set to a position of 7: 1 in the vertical direction and a position of 1: 1 in the horizontal direction in the rectangular shape of the frame FR on the image to be determined. The length including the foot to the top of the projected dolls DL1 and DL2 is defined as the length of one side in the vertical direction, and the length including the horizontal width of the dolls DL1 and DL2 is defined as the length of one side in the horizontal direction. A rectangle can be set as the shape of the frame to be drawn. Note that the aspect ratio of the rectangle may be determined in advance such as, for example, vertical: horizontal = 2: 1, based on the vertical / horizontal ratio in a normal human normal upright state.

  Next, as shown in FIG. 6B, the image is analyzed based on the two known reference positions SP1 and SP2 that are different from each other as determined above, and an image GIa as an analysis result is analyzed. Depth direction (y direction) is defined. Here, in order to simplify the description, a straight line connecting the known reference positions SP1 and SP2 is defined as a y-axis, and a direction in which the y-axis extends is defined as a depth direction. Further, for each point on the y-axis, the x-axis is determined along a set of points that pass through the point and have a same distance from the point to the position of the imaging unit 10.

6C and 6D are conceptual diagrams and graphs showing the frame size according to the xy coordinates determined as described above. For example, the horizontal axis of FIG. That is, it corresponds to the distance from the imaging position, and the vertical axis determines the height of the frame FR to be displayed, that is, the size of the frame FR. That is, in the above case, when the y coordinate is determined, the height H (y) of the frame FR, that is, the size or shape of the frame FR is determined accordingly. In the example of FIG. 6D, the y-coordinate values y ₁ and y ₂ (corresponding to the distance from the imaging position) of the known reference positions SP1 and SP2 and the corresponding heights H (y ₁ ) and H (y ₂ ) The size of the frame FR at each position is determined from the straight line L1 determined on the basis of ₂ ). In FIG. 6D, for the sake of simplicity, the frame FR is set from the y-coordinate with a straight line L1 connecting two points relating to the values y ₁ and y ₂ . If the y coordinate is the same, a frame having the same size and shape is selected even if the value of the x coordinate is different. By defining the position of each pixel constituting the image GIa (particularly the range of the image FLi with respect to the floor surface FL) by the xy coordinates determined as described above, the frame FR having a shape corresponding to the y coordinate of each coordinate. Is displayed. In other words, in the above case, the scale conversion for the frame setting is performed by setting the y coordinate. The above is the premise for enabling frame drawing in the present embodiment. By adopting the above-described aspect, compared with frame drawing using the result in the human detection unit 61 as it is, the variation of the position and size of the frame FR is suppressed, and the person who is the detection target as a framed image Can be made highly visible. In particular, in this case, the visibility with respect to the spatial change is improved.

  Note that information on scale conversion for the frame setting exemplified with reference to FIG. 6, specifically, information on the known reference positions SP1 and SP2 and information such as the straight line L1 determined based on the information are scale conversion information. It is stored in the storage unit 73. The scale conversion frame setting unit 64 determines the shape and size of the frame FR by appropriately reading information stored in the scale conversion information storage unit 73 as necessary.

  Hereinafter, an example of the outline of the overall operation of the object detection apparatus 100 will be described with reference to the flowchart of FIG. Here, with respect to the person image extracted by the human detection unit 61, first, a series of processes related to tracking processing including firstly assigning the extracted person by ID and further including association between frames for each ID is performed. Do. Then, secondly, it is determined whether or not to display a frame (frame drawing) for each person image.

  In FIG. 7, first, the control device 50 of the object detection device 100 analyzes the presence / absence of a human image in the continuous image data acquired by the imaging unit 10 by the human detection unit 61 as the start of the object detection operation. After identifying the person image that is the tracking target, the tracking processing unit 62 performs tracking processing (step S101). Furthermore, the tracking processing unit 62 performs event classification on each person's image extracted from the tracking processing in step S101 (step S102). Specifically, as illustrated in FIGS. 8A and 8B, each image is new or updated in relation to the previous frame image and the subsequent frame image. It is determined whether it is. Further, as illustrated in FIG. 8C, in addition to the above, when there is no longer what existed in the previous frame image, it is determined whether or not to interpolate.

  As illustrated in FIG. 8A, when the current frame image (frame image at time t) is compared with the past frame image (frame image at time t−1), that is, the previous frame image, It is determined that the person image PG1 in the current frame image does not exist in the previous frame image even if the association by ID is considered based on the data stored in the tracking data storage unit 71. The tracking processing unit 62 registers as a new person image.

  On the other hand, as illustrated in FIG. 8B, when the current frame image (the frame image at time t) is compared with the past frame image (the frame image at time t−1), the result of association by ID When there is a person image PG2 that is determined to be the same before and after the frame image, a linking process is performed to indicate that the person image PG2 is image data for the ID. For example, when it is determined that the images are for the same person based on whether the positional relationship between the two frame images is within the integrated range, these links are performed.

  Further, as illustrated in FIG. 8C, when the current frame image (the frame image at time t) and the past frame image (the frame image at time t−1) are compared, Even if the person image PG3 corresponding to the ID registered is not found in the current frame image, the person image PG3 is not immediately absent, and if the predetermined condition is satisfied, the ID information in the past frame image is continued. I have to. That is, the information is stored without being deleted. In other words, regarding the partial image of the person who is the detection target, the tracking processing unit 62 detects the corresponding partial image again within the predetermined number of frames even if it is temporarily undetected between frames in the continuous image. Are associated with each other and handled as the same object.

  Returning to FIG. 7, according to the event classification result in step S102, the control device 50 performs the necessary ID management processing (step S103). That is, for the ID management data to be stored in the tracking data storage unit 71, the information to be stored is changed such as new, updated, or interpolated. Based on the processing result in step S103, the control device 50 performs a drawing operation including frame drawing in the image processing unit 80 (step S104).

  Hereinafter, a series of processes in steps S101 to S104 in FIG. 7 described with reference to specific examples in FIG. 8 and the like will be briefly summarized with appropriate supplements, and general processing contents will be described. This will be described as an example.

  First, in step S101, preparations are made for comparing the distance between the current foot coordinates of the detection target person and the previous foot coordinates of the detection target. In this comparison, the shortest detection target is updated as the same for those within the predetermined integration range, while those outside the integration range are determined as new targets. Such processing is performed for the number of person images extracted as detection targets.

  Next, in step S102, the human image to be detected is classified into three patterns of new, update, and interpolation. That is, based on the extraction result in step S101, the detection target is classified into new and updated. If there is a past person image that has not been used in the extraction result in step S101, it is determined whether or not this should be classified as interpolation. For example, in the interpolation, if a certain number is not updated, it is excluded from the detection target.

  Next, in step S103, ID management is performed. In other words, the ID is added / deleted / updated according to the state (new, updated, interpolated) in step S102 regarding the person to be detected. Each ID holds, for example, information on set coordinate information and detection information (continuous detection number, cumulative detection number, continuous non-detection number). A specific example of these will be described later.

  Lastly, in step S104, based on the ID management detection information in step S103, the detection of false detection and the reliability of detection are improved when drawing a frame, and non-detection is interpolated. In drawing, a rectangle corresponding to the person image to be detected can be drawn from the scale conversion described above.

  Here, for example, in the event classification in step S102 among the above, even if there is no person image corresponding to the ID registered in the past frame image in the current frame image, it is not immediately absent, If the predetermined condition is satisfied, the ID information in the past frame image is continued. With regard to this processing, for example, in the frame drawing processing in step S104, it is conceivable to perform processing such as drawing a frame drawn in a past frame image as it is on the next frame image as it is. Furthermore, in the present embodiment, in the case of a new case, even if a partial image as a person is detected, the corresponding frame drawing is not performed. That is, even if a person is detected, the frame drawing is not performed for the first sheet. By setting it as the above aspects, even if there is no detection or a false detection in human detection, it is suppressed that improper frame drawing is made in connection with this.

  As described above, when extracting a human image, depending on the situation of the image data and the human detection method applied (type of engine that determines the score), non-human objects may be detected as detection targets ( Erroneous detection), or a case where a person to be captured cannot be detected and is absent (not detected). As a typical example of erroneous detection, for example, an image of a train window part or an image of a utility pole is erroneously detected as a person, such as a frame FRx in which the best score area by human detection illustrated in FIG. 4C is displayed as it is. May occur. Such erroneous detection is instantaneous such that it occurs only in one frame of consecutive images (frame images), and it is difficult for a situation in which the erroneous detection state continues. In addition, as a typical example of non-detection, for example, a case where only one frame of continuous images (frame images) is not detected may occur instantaneously. However, even in this case, it is unlikely that an undetected state continues. The above situation occurs because, for example, the instantaneous brightness has changed or the temporary shape has happened to have a high human detection score. Conceivable.

  Based on the above facts, in the present embodiment, in response to a case of erroneous detection, the frame drawing unit 65 prevents the frame from being drawn for a new detection target when drawing the frame, or if it is not detected. Corresponding to the above, the tracking processing unit 62 maintains an appropriate state for frame drawing by performing interpolation between frames.

  Here, the tracking processing unit 62 counts the number of continuous detections, the cumulative number of detections, and the number of continuous non-detections between frames for the detection target associated with the same object, that is, the tracking target, and draws a frame based on these values. In part 65, it is determined whether or not to draw a frame. In this example, the drawing condition is that the number of consecutive detections between frames is 2 or more, and the drawing condition is that the number of consecutive undetections between frames is 3 or more. Note that setting the number of continuous detections to 2 or more corresponds to setting the rising delay for delaying the start of drawing to 1. Further, setting the number of consecutive undetected values to 3 or more corresponds to setting the falling delay for delaying the stop of drawing to 2. Note that the cumulative number of detections between frames is also determined as appropriate, but in one example here, the cumulative number of detections is 2 or more as a drawing condition. Each numerical value described above is an example, and can be appropriately changed according to various factors such as an imaging state.

  Hereinafter, an example of the above operation will be described with reference to FIG. FIG. 9 is a conceptual image diagram for explaining an example of tracking processing in the tracking processing unit 62. FIG. 10 is a time chart showing another example of the determination result related to drawing in association between frames in the tracking process as exemplified in FIG.

  First, in FIG. 9, the upper stage α shows an example in which frame display is performed by reflecting the detection result as it is without assuming the correspondence to the above-described erroneous detection or non-detection. In this case, in the seven consecutive frame images illustrated in the figure, the third, fourth, and sixth frames are not detected in the frame image, and no frames are displayed in these frames. In such a display mode, a frame is displayed or disappears as a series of moving images. On the other hand, the lower stage β shows a case where an example of handling the above-described erroneous detection and non-detection as described above is performed under the situation of the upper stage α. Here, as an example, frame drawing is not performed on the first first sheet (rising delay is set to 1), thereby suppressing frame drawing on an object that is not subject to erroneous detection. In addition, even if it is not detected, the frame drawing is continued until the second non-detection (falling delay is set to 2), thereby suppressing the default of frame drawing for the target object due to the non-detection. Yes. In other words, in the case of the lower stage β, the first frame is detected as a new frame, but the frame drawing is intentionally not performed, while the third, fourth, and sixth frames are not detected. Even if it is, the frame is drawn. As a result, in the lower stage β, the frame drawing is continued in all of the second to seventh frames except the first frame. In other words, the frame is not displayed and disappears, and the display of the frame continues. As described above, when the rising delay is set to 1 and the falling delay is set to 2, the relationship between the detection / non-detection of the human image and whether or not to draw the frame is, for example, other than the same case. An example is a time chart shown in FIG. In the time chart of FIG. 10, the horizontal axis, that is, the time axis direction is indicated by the number of frames of the continuous image, and the detection result / non-detection result of the person image in each frame image is shown. Note that the criteria for determining whether or not to draw a frame in each frame is the same as in the case illustrated in FIG. That is, FIG. 10 shows another example of the result of association between frames in the tracking process shown in FIG. In the drawing, each numerical value in the first, third and twelfth frames is described as an example. In the case of FIG. 10 as well, as in the case of the lower stage β in FIG. 9, the drawing of the frame is continued except for the first frame.

  Here, in the case shown in the lower part β of FIG. 9 and further in the case of FIG. 10, the frame is not displayed for the first first frame, and the display of the frame for one frame is started as compared with the upper part α of FIG. It will be late. However, for example, when the imaging unit 10 acquires two-dimensional image data at a frame rate of 30 fps, the delay as described above is about 1/30 seconds. Such a delay is unlikely to interfere with the confirmation of people getting on and off the station. As described above, visibility can be further improved by performing frame display corresponding to erroneous detection or non-detection of a human image. In particular, in this case, the visibility with respect to temporal change (time-series change) is improved.

  Hereinafter, an example of data management processing (corresponding to part of the processing in step S103) in the tracking processing unit 62 will be described with reference to the flowchart of FIG.

  First, the tracking processing unit 62 determines whether or not the number of consecutive undetected images has reached a threshold (in the above example, 2 which is the number of falling delays) for the detected image of one person whose ID is managed. Is confirmed (step S201). In step S201, when it is determined that the number of consecutive undetected values has not reached the threshold (step S201: No), the tracking processing unit 62 updates the ID related to the one person's image as continuing (step S202). ). On the other hand, when it is determined in step S201 that the consecutive undetected number has reached the threshold (step S201: Yes), the tracking processing unit 62 deletes the ID related to the image of the one person (step S203). That is, for the first time at this point, it is determined that there is no longer a person corresponding to the ID.

  As described above, in consideration of erroneous detection or non-detection of a human image, the frame drawing unit 65 determines whether to draw a frame in accordance with the association of the same object in the tracking processing unit 62. By adopting the mode of determination, visibility can be improved by making the frame display in the temporal change appropriate.

[Others]
The present invention is not limited to the above embodiment, and can be implemented in various modes without departing from the scope of the invention.

  First, in the above embodiment, in order to simplify the description of the frame setting with reference to FIG. 6, for example, the frame setting is performed with the straight line connecting two known reference positions SP1 and SP2 as the y axis. However, the present invention is not limited to this, and various modes can be considered for frame setting. For example, two or more reference position known values are extracted and their root mean squares are used to set the frame at each reference position. The shape and size may be set.

  In the above description, as an example, the monitoring target area is the home PF of the station ST. However, the present invention is not limited to this, and various places where accurate human detection is desired can be monitored. For example, it can be applied to user monitoring in escalators and elevators.

  Further, the arrangement of the imaging unit 10 is not limited to the case where the imaging unit 10 is installed on the upper side of the home PF and images the lower side. For example, it is conceivable to install an imaging unit in a home located on the opposite side of the home to be imaged.

  In the above description, train crew members and station staff members are listed as examples of notification destinations regarding the detection results of the control device 50. However, the present invention is not limited to this, and notifications may be made to various places and people. Is possible.

  Further, when a plurality of imaging units 10 are set, for example, information between imaging units with adjacent imaging ranges may be linked in the data. As a result, it is possible to continuously track a person over a wider range.

  DESCRIPTION OF SYMBOLS 10 ... Imaging part, 50 ... Control apparatus, 60 ... Main control part, 61 ... Human detection part, 62 ... Tracking processing part, 63 ... Reference position determination part, 64 ... Scale conversion frame setting part, 65 ... Frame drawing part, 70 ... storage unit 71 ... tracking data storage unit 71a ... human storage unit 71b ... reference position data storage unit 72 ... human feature quantity information storage unit 73 ... scale conversion information storage unit 80 ... image processing unit 100 ... Object detection device, BD1, BD2 ... boundary, DA ... display display unit, DL1, DL2 ... doll, FL ... floor surface, FLi ... image, FR ... frame, FRx ... frame, GI, GIa ... image, HU ... user, L1 ... straight line, PE1, PE2 ... person (image), PF ... home, PG1-PG3 ... person image, PI ... partial image, SA ... station employee, SP1, SP2 ... known reference position, ST ... station, TK ... track, TKi ... Picture , TR ... train, WLi ... image, y1, y2 ... value, α ... upper, β ... lower

Claims (8)

A reference position determination unit that determines a reference position for a partial image to be detected in a continuous image;
A scale conversion frame setting unit that determines the size of a frame that surrounds the partial image to be detected by scale conversion determined for each position in the image;
An object detection apparatus comprising: a frame drawing unit that draws a frame determined by the setting in the scale conversion frame setting unit using the reference position determined by the reference position determination unit as a base point on an image of each frame constituting a continuous image .   The scale conversion frame setting unit determines the size of the frame at each position in the image based on the distance from the corresponding actual position to the imaging position in an image captured from a fixed position. The object detection apparatus described. A human detection unit that performs human detection by extracting a human image,
3. The object detection device according to claim 1, wherein the reference position determination unit sets a position of a foot of the person image specified by the person detection unit as a reference position to be determined. 4.   The object detection apparatus according to claim 3, wherein the frame set in the scale conversion frame setting unit has a rectangular shape corresponding to a ratio of persons as detection targets. A tracking processing unit that tracks the same object by associating images between frames in continuous image data,
5. The object detection device according to claim 1, wherein the frame drawing unit determines whether or not to draw a frame in accordance with association of the same object in the tracking processing unit.   The object detection apparatus according to claim 5, wherein the frame drawing unit does not draw a frame for a new detection target.   The tracking processing unit relates to a partial image as a detection target, even if it is temporarily undetected between frames in a continuous image, if a corresponding partial image within a predetermined number of frames is detected again, The object detection apparatus according to claim 5, wherein the object detection apparatus handles the same object.   Whether the frame drawing unit draws a frame based on at least one of the continuous detection number, the cumulative detection number, and the continuous non-detection number between detection target frames associated as the same object in the tracking processing unit. The object detection device according to claim 7, wherein the determination is made.

Images