JP2019128799A - Target person detection device - Google Patents

Abstract

To provide a target person detection device that, during monitoring, enables proper grasping of a target person with an attachment, such as a visually handicapped person having a cane, even if the attachment accompanying the target person to be monitored, such as a cane, may not be detected.SOLUTION: When a main control unit 60 as a cane determination unit 63 in a control device 50 determines that there is a cane image, a person who is regarded as the same person for a corresponding person image can be stored as a visually handicapped person, so that even if a cane is not detected during monitoring, the person who has been determined as having a cane is still treated as a visually handicapped person. That is, a visually handicapped person having a cane can be properly grasped.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a target person detection device that detects a target person such as a visually handicapped person, for example, when monitoring at a station platform or the like.

  For example, a visually handicapped person (a visually handicapped person) who uses a walking stick (typically a white cane) as a target to be monitored can walk alone, for example, according to a guidance block installed on the ground. Do. However, there are cases where the surrounding situation cannot be judged correctly only with the guidance block, and for example, a fall at a station platform is a problem. Therefore, early detection of the user of the cane at the station platform is important for those who perform station duties.

  On the other hand, for example, a pedestrian is identified based on three-dimensional distance information obtained from a stereo camera, a portion where a white cane may exist is image-processed, and a white cane is used by detecting the white cane That detect a person (see Patent Document 1) is known.

  However, in Patent Document 1, for example, the white cane is hidden behind the white cane user in the imaging range of the camera, or the white cane user is far away from the camera because of the resolution of the camera. No measures have been taken for the case where the white cane becomes undetected because the white cane can not be recognized on the image.

JP 2007-274234 A

  The present invention has been made in view of the above-described points, and appropriately has an appendage even if there is a possibility that an appendage accompanying a subject to be monitored such as a cane becomes undetected during monitoring. An object of the present invention is to provide a subject detection apparatus which enables capturing of a subject (typically, a visually impaired person having a walking stick).

  In order to achieve the above object, the subject detection device according to the present invention relates to an incident determination unit that determines the presence or absence of an accompanying image in a person image, and a person image that is determined to have an accompanying image by the incident determination unit. A tracking processing unit for tracking the same person by associating images between frames in continuous image data, and a target person storage unit for storing the same person as a target person.

  In the above-described subject detection device, if it is determined by the accessory determination unit that there is an accessory image, it is possible to maintain a state in which a person who is the same person as the target person image is stored as a target person. Even if the cane that is an object may be undetected, the person who is determined to have the cane is still treated as the visually impaired person who is the subject. That is, a visually handicapped person having a walking stick can be properly captured.

  In a specific aspect of the present invention, an image capturing unit that captures an image of a monitoring target area and acquires two-dimensional image data as continuous image data, and a human image by shape extraction by machine learning from the two-dimensional image data acquired by the image capturing unit. And a person detection unit that detects the presence of a person by extracting. In this case, for example, even from the two-dimensional image data that can be acquired by a camera or the like that is already provided for monitoring or the like, the human image can be accurately extracted by the human detection unit.

  In another aspect of the present invention, the tracking processing unit determines whether or not the images are the same person based on the information on the position change of the person image between frames of the continuous image data. In this case, it is possible to reliably identify the imaged person.

  In still another aspect of the present invention, the tracking processing unit is configured such that when a person image corresponding to a person detected in one image data does not exist in subsequent image data for a predetermined number of frames, It is determined that the frame is out. In this case, it is possible to appropriately cope with a case where image data to be detected cannot be obtained due to an error in image recognition.

  In still another aspect of the present invention, the incidental body determination unit performs mask processing for cutting out a portion corresponding to the lower body of a person in a person image. In this case, a quick and reliable cane determination can be performed.

  In yet another aspect of the present invention, the incidental determination unit determines the presence or absence of an incidental image based on color information obtained by HSV conversion processing. In this case, the cane can be quickly and reliably determined based on the color features.

  In yet another aspect of the present invention, the incidental determination unit determines the presence or absence of an incidental image based on straight line component information obtained by the Hough transform process. In this case, the cane can be quickly and reliably determined based on the shape feature.

It is a block diagram which shows notionally the subject person detection device concerning an embodiment. It is a conceptual diagram which shows the mode of the station which deployed the subject person detection apparatus. (A)-(C) are conceptual image figures which show an example about the mode of the person tracked. (A)-(C) are the data tables which show an example about the data regarding the person tracked. (A)-(C) is a conceptual image figure for demonstrating the matching between the frames of a person. (A)-(D) are conceptual image figures which show an example about the method of extracting the wand image as an incidental thing image. It is a flowchart for demonstrating an example of a tracking process. It is a flowchart for demonstrating an example of the extraction process of the cane image as an accompanying image in one image.

  Hereinafter, with reference to FIG. 1 etc., an example is demonstrated about the person with a blind person detection apparatus which is one aspect | mode of the object person detection apparatus which concerns on this embodiment. FIG. 1 is a block diagram conceptually showing a visually impaired person detection device 100 according to the present embodiment, and FIG. 2 is a conceptual diagram showing a state of a station ST in which the visually impaired person detection device 100 is provided. Here, as an example, the home (platform) PF of the station ST where the train TR can arrive and exit is the area to be monitored by the visually impaired person detection device 100. That is, the user HU in the station PF's home PF is a detection target, and whether or not the user HU is a visually impaired person, that is, whether or not it is a target person to be monitored is detected. ing.

  In order to achieve the above object, the visually-impaired person detection device 100 is a visually-impaired person by detecting the position of the user HU in the station premises (especially on the home PF) and capturing the characteristics of the user HU. It is determined whether or not. Here, as a representative example, as shown in FIG. 2, the presence of a user HU having a cane CA is confirmed, and the confirmation result is notified to the station staff SA in the station service room SO. There is. Here, as a representative example, detection of a person with a visual impairment will be described by detecting whether or not a white stick is included as an accessory.

  Hereinafter, the configuration and the like of the visually impaired person detection device 100 according to the present embodiment will be specifically described with reference to FIGS. The visually impaired person detection device 100 according to the present embodiment is acquired by the imaging unit (imaging camera) 10 that images the monitoring target area and the imaging unit 10 in order to detect the user HU, that is, the person who moves on the home PF. And a control device 50 that performs various processes and controls to detect a visually impaired person based on the image data.

  The imaging unit 10 is an imaging camera configured by an individual imaging element such as a CCD or a CMOS, for example, and is installed on the upper side of the home PF as illustrated in FIG. 2 and images the lower side. That is, the imaging unit 10 captures an image of the monitoring target area from the upper side toward the floor surface FL of the home PF. Here, in particular, two-dimensional image data is acquired as continuous image data at a frame rate of, for example, 30 fps in order to enable tracking processing for associating images between frames in continuous image data and tracking the same person. (See, for example, FIGS. 3A to 3C). Note that one or more imaging units 10 are provided according to the installation position, the shape of the home PF, and the like in order to enable imaging of the entire home PF.

  The control device 50 is configured by, for example, a CPU, a storage device, and the like, and tracks the same person based on the association of the continuous image data acquired by the imaging unit 10, and a white cane as an example of an incidental person with respect to the imaged person. It is determined whether or not it has. That is, the presence / absence of a cane image as an example of an accompanying image in a person image is determined for the acquired continuous image data. In order to enable the above processing, as illustrated in FIG. 1, the control device 50 includes a main control unit 60, a storage unit 70, and an image processing unit 80.

  The main control unit 60 includes, for example, a CPU and performs various information processing, thereby extracting a person image by shape extraction by machine learning from continuous image data that is a two-dimensional image acquired by the imaging unit 10. The person detection unit 61 that detects the presence of a person, the tracking processing unit 62 that tracks the same person by associating images between frames in the continuous image data acquired from the imaging unit 10, or the presence or absence of a cane image in the person image It functions as a cane determining unit 63 for determining. That is, the cane determination unit 63 performs a determination on a cane image that is an accompanying image as an example of the accompanying object determination unit.

  The storage unit 70 is configured by a storage device or the like and stores various data and programs. For example, a person who satisfies a certain requirement among persons who are identified as the same person by the tracking processing unit 62 is a visually impaired person. In addition to the tracking data storage unit 71 including the visually handicapped person storage unit 71a to be stored, it includes a human feature amount information storage unit 72, a color information storage unit 73, a linear component information storage unit 74, and the like. In this case, the visually impaired person storage unit 71a functions as a subject person storage unit that stores a person who satisfies certain requirements as a visually impaired person, that is, a subject person.

  Hereinafter, the above-mentioned each part which constitutes control device 50 is explained in more detail.

  First, the main control unit 60 can confirm whether or not an image having a human (person) characteristic exists in each two-dimensional image data acquired by the imaging unit 10 as the human detection unit 61. It has become. For example, by using a known HOG feature quantity or support vector machine (SVM), it is possible to extract a person image from a two-dimensional image data by shape extraction by machine learning. That is, the human detection unit 61 extracts a straight line portion or the like from each image data acquired by the imaging unit 10, and has a characteristic unique to the person in the image data based on the extracted feature amount or the like of the straight line portion. It is determined whether there is a problem. In other words, the person detection unit 61 classifies persons and non-persons in the image data. Thereby, the person detection part 61 extracts a person image based on the determination result. For this reason, the person detection unit 61 reads out a necessary program from the storage unit 70 as appropriate, and detects the presence of a person using the feature quantity data stored in the person feature quantity information storage unit 72. There is. By using the HOG feature amount or the like, human detection can be accurately performed even from a two-dimensional image. In this case, for the imaging unit 10, data acquisition can be performed without requiring a special configuration such as taking out three-dimensional information such as a stereo camera. That is, for example, it is possible to apply an existing camera or the like for monitoring at the time of arrival and departure, which is installed at the station ST.

  In addition, the main control unit 60 enables tracking of the same person as the tracking processing unit 62. For this reason, the tracking processing unit 62 reads out a program from the storage unit 70 as appropriate, and performs processing for associating images between frames in the continuous image data for each human image detected or extracted by the human detection unit 61. In particular, in the present embodiment, for a person image that has been determined to have a cane image even once by the cane determination unit 63, a state in which a person who is the same person for the corresponding person image is stored as a visually impaired person is maintained. As a result, for example, even if a cane may not be detected during monitoring, a person who has been determined to have a cane will continue to be treated as a visually impaired person. Therefore, it is a mode in which a visually handicapped person having a cane can be captured appropriately and more reliably.

  Further, the main control unit 60, as the cane determination unit 63, determines the presence or absence of the cane image in each person image detected or extracted by the person detection unit 61. Here, as an example, the wand determination unit 63 uses the data stored in the color information storage unit 73 and the linear component information storage unit 74 in addition to the mask processing for masking a part of the person image, and uses the data stored in the color and shape. Judgment processing is performed. Details of each process will be described later.

  Next, in the control device 50, the storage unit 70 stores various data as a storage device. Here, for example, in the tracking process related to the human image detected as described above, the data related to each human image is stored in the tracking data storage unit 71 as needed (for example, the data shown in FIGS. 4A to 4C). See the table). In particular, at the time of such data storage, information on the visually impaired person, which is one of the data on each person image, is stored in the visually impaired person storage unit 71a.

  In addition, as described above, the human feature information storage unit 72 of the storage unit 70 relates to human detection by the human detection unit 61, and extracts a human image by shape extraction by machine learning using a known method such as an HOG feature. Various information necessary for extraction is stored. Furthermore, various programs for extracting person images may be stored.

  Further, in the storage unit 70, the color information storage unit 73 stores various pieces of information for color determination regarding the wand determination by the wand determination unit 63. In particular, in this embodiment, in order to extract the color of a white cane, that is, to extract white, an HSV conversion process, that is, in a color space composed of three components of hue, saturation, and value. It stores information for enabling the original processing. Further, the linear component information storage unit 74 stores information for measuring the degree of parallelism of the rod-like linear shape, which is a characteristic matter of the shape of the white stick. In the case of a cane, the two-dimensional image data is extracted as a thin stick, and whether or not the parallelism, length, etc. of the edge portions at both ends (the end on the image of the side of the stick) is a cane This is considered to be an important geometric feature in judgment.

  Finally, in the control device 50, the image processing unit 80 is configured by, for example, a GPU or the like, and is acquired by the imaging unit 10 in accordance with an instruction from the control device 50 to enable the various image processing as described above. Various image processing is performed on each two-dimensional image data.

  In addition to the above, the visually-impaired person detection device 100, as shown in FIG. 2, serves as a station employee notification unit for notifying a station employee SA existing in a station premises such as a station service room or a platform. , A station staff speaker PA, and a display display unit DA. These are installed in the station service room SO, for example. The station staff speaker PA and the display display unit DA are connected to the control device 50, and in accordance with an instruction from the control device 50, notify the station staff SA present in the station office SO or the like. In addition, by providing various communication means, notification may be given to a crew member (driver or conductor) in the train TR that is arriving and departing.

  Hereinafter, with reference to FIG. 3, the tracking process in the visually-impaired person detection apparatus 100 about the person who becomes a monitoring object is demonstrated. FIGS. 3A to 3C are conceptual image diagrams illustrating an example of a state in which the same person is tracked by association between frames in continuous image data. In the illustrated example, the person tracking extracted image HGa including the person image HUi and the cane image CAi, which are persons on the image, over time are shown in FIG. 3 (A), FIG. 3 (B), FIG. 3 (C). In this order, the entire image GI that is the entire range imaged by one imaging unit 10 moves in the direction indicated by the arrow AR1. In the illustrated case, the extracted image A1 in FIG. 3A, the extracted image A2 in FIG. 3B, and the extracted image A3 in FIG. 3C are associated with each other as the same person image and are captured as the extracted image HGa. By that, it is tracked. At this time, extraction of the cane image CAi existing as a partial image in the extracted image HGa is important. For this reason, each of the extracted images A1 to A3 or the extracted image HGa obtained by collecting them is extracted so as to be in a necessary and sufficient range so that the cane image CAi can be captured together with the person image HUi. Here, as illustrated in the figure, an image is extracted in a rectangular area. Specifically, if the person image HUi is detected by the person detection by the person detection unit 61, and if the person image HUi is included and the person image HUi is accompanied by the cane image CAi, A range in which it is estimated that the wand image CAi is also included is taken as an extracted image HGa of a predetermined rectangular range.

  Based on the above, particularly in the present embodiment, once the wand image CAi is detected by the wand detection for the extracted image A1 in FIG. 3A, the extracted image A2 as shown in FIG. In the case where the cane image CAi is hidden by the person image HUi and is difficult to see, and even if it is not detected even if the cane is detected, it is associated with the extracted image HGa by tracking processing. In the tracking target image, the wand image CAi is continuously handled as an accompanying image. Thereby, the person can be regarded as a person having a cane, that is, a visually handicapped person.

  FIGS. 4A to 4C are data tables showing an example of data stored in the storage unit 70 regarding a person who is tracked by the above-described tracking processing. Here, as shown in the figure, an ID number is assigned to each person to be tracked, and information on the position (center coordinates) on the two-dimensional image as position information for each ID number, that is, for each person, Information and information about whether or not the person is a white cane, that is, whether or not the person is visually impaired, is recorded in the tracking data storage unit 71.

  The above information will be described more specifically. First, for the position information of each person, the center position of the extracted image is indicated by xy coordinates on the horizontal axis and the vertical axis of the two-dimensional screen. That is, the center of each extracted image in the whole image GI (see FIG. 3 etc.), that is, the center of the rectangular area is indicated by the values of the x coordinate and the y coordinate for the whole image GI. As for the information on the traveling direction, for example, the reference direction on the two-dimensional screen can be determined as a direction having an angle of zero, and can be determined by an angle relative to this direction. Furthermore, for information on whether or not a person is a white cane, that is, whether or not he / she is visually impaired, “True” and “False” indicate whether or not a cane image is detected from an extracted image of a person image. ing. Here, as shown in the drawing, for example, for the person assigned ID number 1, as a result of tracking, it was determined that “False”, that is, no wand image at the time shown in FIG. If it is determined that “True”, that is, there is a wand image at the time point shown in FIG. 4B, then, as shown in FIG. 4C, for example, “True” is displayed regardless of the detection result of the presence or absence of the wand image. That is, it is treated as being a white cane person, ie, a visually impaired person. In other words, once “True” is reached, “True” is subsequently maintained. On the other hand, it may occur that what was initially “False” is changed to “True” in the middle. By handling as described above, when a visually impaired person is detected, it is possible to monitor the person more surely by grasping the person thereafter.

  Hereinafter, as a more specific example regarding the tracking process, the correspondence between the frames of the person according to the present embodiment will be described with reference to FIG. FIGS. 5A to 5C are image diagrams conceptually showing a specific example of a method of association between person frames.

  First, as shown in FIG. 5A, for example, as shown in FIG. 5A, a person tracking extraction image A1 including a person image HUi is detected by detecting a human image in the human detection unit 61, and further, In the frame image shown in FIG. 5B, which is a frame image in a frame later than that in FIG. 5A, when a plurality of other extracted images A2 and B1 are detected, the extracted image A1 and An association is made between As an example of the association method, it is conceivable that the tracking processing unit 62 determines whether to associate or associate person images based on a difference in the amount of distance moved in the xy plane of the two-dimensional image. In the case shown in the figure, the extracted image A2 that is closer to the extracted image A1 than the extracted image A1 among the extracted images A2 and B1 by comparison of the center positions in the entire image GI is an image of the same person as the extracted image A1. And these are associated as being extracted images HGa indicating the same person. On the other hand, the extracted image B1 is treated as one in which a new person is detected, that is, one in which another person frames in the entire image GI. That is, the tracking processing unit 62 determines whether or not the images are the same person based on the information on the position change of the person image between frames of the continuous image data from the imaging unit 10.

  Furthermore, as another aspect, as shown in FIG. 5C, for example, an extracted image Ax that has been previously associated as an extracted image HGa in a frame image in a frame later than FIG. 5A. Is no longer detected from the image, depending on the conditions, it may be treated as if it could not be detected simply because of an error during image acquisition, etc. It is divided into the case where it goes out of range, that is, the case where it is handled as being out of frame. Here, when an image corresponding to the extracted image HGa is not detected continuously for a predetermined number of frames (for example, about 5 or 6 frames), it is assumed that the person is out of the entire image GI. . In other words, the tracking processing unit 62 continues the image data after the human image corresponding to the person detected in the one image data acquired from the imaging unit 10 for a predetermined number of frames (for example, about 5 or 6 frames). If it does not exist, it is determined that the corresponding person has framed out.

  As described above, the tracking process of the person image is performed from the time when each person enters the imaging range until the person leaves the frame.

  Hereinafter, with reference to FIG. 6, a specific example of a method for extracting a wand image regarding wand determination by the wand determination unit 63 will be described. As shown in FIG. 6A, when the extracted image A1 including the person image HUi that should be determined whether or not there is a cane image is extracted, the extracted image A1 is normalized as necessary. For example, normalization conditions are appropriately determined according to the arrangement of the imaging unit 10 (see FIG. 2 and the like), the imaging range, the angle of view, and the like. By normalization, for example, in the case illustrated in FIG. 6B, the extracted image A1 is a rectangular image having a length of 2: 1 in the human height direction. That is, in the drawing, the ratio of the vertical width LA to the horizontal width LB is 2: 1. In this state, as shown in FIG. 6C, the upper half, that is, the upper half of the body is masked, and only the lower half, that is, the lower half of the body is extracted to extract the partial image A1a. That is, a portion below the broken line PX is taken out as a partial image A1 a with reference to the central broken line PX in the vertical direction of the extracted image A1 shown in FIG. 6B. Thereby, it becomes easier to detect the cane image CAi located on the lower body side of the body.

  Here, with respect to the partial image A1a shown in FIG. 6C, white extraction which is one of feature extraction based on the color feature is performed. For white extraction, the presence or absence of a cane image is determined based on color information obtained by HSV conversion processing. Accordingly, it is possible to reliably detect a candidate portion for the white cane image CAi.

  Further, the presence / absence of a cane image in the candidate image portion of the cane image is determined based on straight line component information by a Hough transform process (Hough transform), which is one of known methods for feature extraction based on a shape feature. Specifically, as shown in FIG. 6D, the edge portions EG1 and EG2 for the white portion are extracted, and the length L1 of the edge portions EG1 and EG2 or between the edge portion EG1 and the edge portion EG2 is extracted. By analyzing the width L2 and the parallelism between the two based on the Hough transform process, it is determined whether or not the image extracted by the white extraction is a cane image. As described above, the cane image CAi can be detected quickly and reliably.

  Hereinafter, an example of the tracking process will be described with reference to the flowchart of FIG. First, when the control device 50 starts an operation for person monitoring, the main control unit 60 confirms whether the entire image data of the monitoring target area is acquired from the imaging unit 10 as the tracking processing unit 62 (step S101). ). In step S101, it is sequentially detected whether or not the next image data has been input for continuous image data, including the case of the first image data input.

  In step S101, when the first image data after the start of operation or the second and subsequent image data is acquired (step S101: Yes), the main control unit 60 acts as a human detection unit 61 in the acquired image. Detection is performed, and it is determined whether or not non-detection has occurred for a human image that has existed up to the detection of a person or an image of the previous frame (in the second and subsequent cases) (step S102). In step S102, when no person is detected and no undetected person is generated in the second and subsequent times, no special processing is performed (step S102: No), and the processing operation from step S101 is performed. repeat.

  On the other hand, if a person is detected in step S102, or if a non-detection has occurred for a person image that has existed up to the image of the previous frame, the association processing before and after the frame is performed (step S103). ). Specifically, as in the case illustrated in FIGS. 5A and 5B, in the frame image in the previous frame, data was acquired for only one person image, whereas In an aspect in which two person images appear in the frame image in the frame of, one of the two person images is associated with the same person as the person of the frame image in the previous frame, and the other is A process of registering new data as a newly detected person is performed. Further, as shown in FIGS. 5 (A) and 5 (C), when a non-detection occurs in a human image that has existed up to the frame image of the previous frame, further association according to the case is performed. The process for performing (step S104 and subsequent steps) is performed.

  When it is confirmed that a person is detected or a non-detected person is generated in step S102 and the process proceeds to step S103, a staff image extraction process from step S201 described later with reference to FIG. 8 is started.

  Hereinafter, referring back to FIG. 7, the processing operation after step S104 will be described. In the association in step S103, when no human image has been detected (step S104), that is, as illustrated in FIGS. 5A and 5C, the frame images between the previous and subsequent frames. If it is determined that the image corresponding to the person image that existed in step S104 does not exist in the frame image in the next frame (step S104: Yes), the non-detection is determined a predetermined number of times (for example, It is confirmed whether or not it continues for 5 times or 6 times (step S105).

  If it is determined in step S105 that the predetermined number of times has not been reached (step S105: No), a process of adding +1 to the non-detection count is performed (step S106a), and the corresponding human image that has not been detected yet is detected. While the data is present, processing for handling the hold is performed (step S107a), and data update with the above contents is performed (step S108). That is, it is regarded that there is an error in human detection etc., and the number of occurrences of non-detection is increased once while maintaining the information of the previous one.

  On the other hand, if it is determined in step S105 that the predetermined number of times (for example, 5 times or 6 times) has been reached (step S105: Yes), a process of clearing the non-detection count is performed (step S106b), A process corresponding to the detected person image data being deleted out of the detection range is performed (step S107b), and the data is updated with the above contents (step S108). That is, the corresponding person is regarded as being out of frame, the corresponding data is deleted, and the count of the number of occurrences of the non-detection up to that point is cleared.

  If no non-detection is confirmed in step S104 (step S104: No), a process of clearing the non-detection count is performed (step S107c), and the data is updated with the content (step S108).

  When the data update in step S108 is completed, the process returns to step S101 again. That is, in the series of tracking processing, the presence or absence of the next data acquisition is confirmed. As described above, a series of processes relating to tracking of a person image is performed.

  Hereinafter, an example of the wand image extraction process in one acquired image will be described with reference to the flowchart of FIG. 8.

  As described above, when the occurrence of the person detection or the non-detection person in step S102 in FIG. 7 is confirmed and the process proceeds to step S103, the main control unit 60 as the cane determination unit 63 relates to the cane image extraction process. First, it is determined whether or not a wand image extraction process is necessary, that is, whether or not it is necessary to determine the visually impaired person (step S201). Specifically, as a result of the human detection in step S102 or the association of the human images in step S103, if all the detected human images have already been determined to be “True”, that is, there is a cane image, There is no need to detect the presence or absence of a cane image again.

  If it is determined in step S201 that the visually impaired person determination is unnecessary (step S201: No), the process ends without performing any processing and waits until a new person is detected.

  On the other hand, when it is determined that the visually impaired person determination is necessary (step S201: Yes), that is, as a result of the association in step S103, the person image determined as “False”, that is, no wand image is 1 If at least one image exists, image processing for determining the presence or absence of a wand image is sequentially performed on the person image (steps S202 to S205).

  Hereinafter, a series of image processing for determining the presence or absence of a cane image will be specifically described. First, the main control unit 60 as the wand determination unit 63 performs mask processing to cut out a portion corresponding to the lower body of a person in a person image (step S202). That is, as described with reference to FIGS. 6 (B) and 6 (C), the aspect ratio of the target image is normalized to 2: 1 as necessary, and the lower half of the partial image corresponding to the lower body (For example, the partial image A1a in FIG. 6C) is taken out. Next, the wand determination unit 63 performs a white extraction process on the lower half of the partial image extracted in step S202 (step S203). That is, as described with reference to FIG. 6C, the white portion extraction based on the color information by the HSV conversion process is performed as the feature extraction based on the color feature. In this way, it is possible to extract a candidate image portion to be a wand image from the extracted image. Further, the wand determination unit 63 performs a line segment extraction, that is, a linear component extraction process on the image portion extracted in step S203 (step S204). Specifically, as described with reference to FIG. 6D, the edge portion of the white portion is extracted, and a Hough transform process or the like is performed to calculate the length, interval (width), parallelism, and the like. And based on the calculation result, it is possible to determine whether or not the extracted image is a wand image. Based on the analysis results of steps S202 to S204, the cane determination unit 63 determines whether or not there is a cane image, that is, whether or not the target person image is an image of a visually impaired person (white cane determination). This is performed (step S205).

  If it is determined in step S205 that the person is visually impaired (step S205: Yes), a visually impaired person (white cane person) is added (step S206). That is, for the person image of the target ID number in the data table shown in FIG. 4, the “False” in the white cane column in the data table is changed to “True”.

  On the other hand, if it is determined in step S205 that the person is not visually impaired (step S205: No), the process ends without performing the visually impaired person (white cane person) addition process. That is, the state of “False” in the column is maintained for the person image having the target ID number. As described above, a series of processes related to extraction of a cane image from one extracted image is performed.

  As described above, in the visually impaired person detection device 100 according to the present embodiment, if the main control unit 60 as the cane determination unit 63 in the control device 50 determines that there is a cane image, the corresponding person image is obtained. Since the person who is identified as the same person by the tracking processing in the tracking processing unit 62 can be maintained as a visually impaired person, even if the cane may be undetected during monitoring, Those who have been determined to continue to be treated as visually impaired. That is, the visually impaired person having a cane can be caught appropriately.

[Others]
The present invention is not limited to the above embodiment, and can be implemented in various modes without departing from the scope of the invention.

  First, in the above embodiment, as an example, after detecting a person, the necessity of the determination is confirmed (step S201 in FIG. 8), and the cane image is determined. The determination of the wand image may be performed on all the detected images.

  Further, in the above, for example, when detecting a wand image, first, white extraction, that is, color extraction is performed, and then linear component extraction, that is, extraction by shape is performed. In this case, it is considered that, for example, canes or umbrellas that are not white or similar in color are excluded, and those similar in shape can be excluded in advance. However, the present invention is not limited to the above case, and for example, a case where the order is changed and extraction based on the shape feature is performed first is also conceivable.

  In the above, the cane possessed by the visually impaired is a white cane, that is, a white cane, but other colors and combinations of colors are also conceivable, and the shape is considered to be characterized in another place. In such a case, it is conceivable to select an appropriate color extraction or shape extraction method in accordance with these.

  In the above description, as an example, the monitoring target area is the home PF of the station ST. However, the present invention is not limited to this, and various places where it is desired to appropriately capture visually impaired persons can be monitored.

  Further, the arrangement of the imaging unit 10 is not limited to the case where the imaging unit 10 is installed on the upper side of the home PF and images the lower side. For example, it is conceivable to install an imaging unit in a home located on the opposite side of the home to be imaged.

  Moreover, although the station service room SO is mentioned as an example of the notification destination about the detection result in the control apparatus 50 in the above, it is possible to notify not only this but to various places. Furthermore, the notification destination is not limited to a station employee or the like, and may be a control circuit that controls a platform door or the like installed at a station. In this case, for example, there is an increased possibility that the walking stick will come into contact with a home door where a visually handicapped person is nearby. In consideration of this, a criterion for determining whether to stop the opening / closing operation of the home door is set. can do.

  Further, in the case where a plurality of imaging units 10 are set, for example, information between imaging units having adjacent imaging ranges may be linked in data. As a result, it is possible to continuously track a person over a wider range.

  In the above, a white cane is taken up as a typical example of the accompaniment, and detection / judgment is made as to whether or not it has a white cane. It is also possible to use a wheelchair. Further, in the above description, the target person to be monitored is described as a visually impaired person as a typical example, and the target person detection apparatus is described as a visually impaired person detection apparatus. The target person detection device according to the embodiment can target various persons according to the monitoring mode. Moreover, according to the characteristic of a subject person, it can be made not only a white stick and a guide dog but also various things about incidentals to be subjected to detection / determination etc.

  DESCRIPTION OF SYMBOLS 10 ... Imaging part, 50 ... Control apparatus, 60 ... Main control part, 61 ... Human detection part, 62 ... Tracking processing part, 63 ... Walking stick determination part, 70 ... Storage part, 71 ... Tracking data storage part, 71a ... Visual disorder Person storage unit 72 ... Human feature amount information storage part 73 ... Color information storage part 74 74 Straight line component information storage part 80 ... Image processing part 100 ... Visually impaired person detection device A1, A2, A3, Ax, B1 ... extracted image, AR1 ... arrow, CA ... cane, CAi ... cane image, DA ... display display, EG1, EG2 ... edge portion, FL ... floor surface, GI ... overall image, HGa ... extracted image, HU ... user , HUi ... person image, L1 ... length, L2 ... width, LA ... length, LB ... width, PA ... station staff speaker, PF ... home, PX ... dashed line, SA ... station staff, SO ... station office, ST ... Station, TR ... train

Claims (7)

An incident determination unit that determines the presence or absence of an incident image in a person image;
A tracking processing unit for tracking the same person by associating an image between frames in continuous image data with respect to a person image determined to have the accompanying image by the accompanying object determination unit;
And a target person detection unit that stores the same person as a target person. An imaging unit for imaging a monitoring target area and acquiring two-dimensional image data as the continuous image data;
The subject detection apparatus according to claim 1, further comprising: a person detection unit that detects a person's presence by extracting a person image from the two-dimensional image data acquired by the imaging unit by shape extraction by machine learning.   3. The tracking processing unit according to claim 1, wherein the tracking processing unit determines whether or not the images are the same person based on information on a change in position of the person image between frames of the continuous image data. The subject detection device described.   The tracking processing unit determines that the corresponding person is out of frame when a person image corresponding to the person detected in one image data does not exist in subsequent image data for a predetermined number of frames. Item 4. The subject detection apparatus according to any one of Items 1 to 3.   The subject detection apparatus according to any one of claims 1 to 4, wherein the accompanying matter determination unit performs a mask process of cutting out a portion corresponding to a lower body of a person in a person image.   The subject person detection apparatus according to any one of claims 1 to 5, wherein the incidental object determination unit determines the presence or absence of an incidental image based on color information by HSV conversion processing.   The target person detection device according to any one of claims 1 to 6, wherein the incidental body determination unit determines the presence or absence of an incidental body image based on linear component information by Hough transform processing.

Images