WO2023281897A1 - Système de vidéosurveillance et procédé de vidéosurveillance - Google Patents

Système de vidéosurveillance et procédé de vidéosurveillance Download PDF

Info

Publication number
WO2023281897A1
WO2023281897A1 PCT/JP2022/018073 JP2022018073W WO2023281897A1 WO 2023281897 A1 WO2023281897 A1 WO 2023281897A1 JP 2022018073 W JP2022018073 W JP 2022018073W WO 2023281897 A1 WO2023281897 A1 WO 2023281897A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
specific event
video
feature
specific
Prior art date
Application number
PCT/JP2022/018073
Other languages
English (en)
Japanese (ja)
Inventor
健一 森田
敦 廣池
智明 吉永
良起 伊藤
Original Assignee
株式会社日立製作所
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 株式会社日立製作所 filed Critical 株式会社日立製作所
Publication of WO2023281897A1 publication Critical patent/WO2023281897A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B13/00—Burglar, theft or intruder alarms
    • G08B13/18—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength
    • G08B13/189—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems
    • G08B13/194—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems
    • G08B13/196—Actuation by interference with heat, light, or radiation of shorter wavelength; Actuation by intruding sources of heat, light, or radiation of shorter wavelength using passive radiation detection systems using image scanning and comparing systems using television cameras
    • G—PHYSICS
    • G08—SIGNALLING
    • G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
    • G08B25/00—Alarm systems in which the location of the alarm condition is signalled to a central station, e.g. fire or police telegraphic systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • the present disclosure relates to a video surveillance system and a video surveillance method.
  • video surveillance systems are used to display surveillance images transmitted from cameras installed at surveillance locations. It is important to perceive specific events that are events. Note that the specific event is, for example, an incident such as a crime or an accident, and an event that requires security, police, or emergency response.
  • Patent Document 1 discloses a video monitoring system in which a user registers, as attribute information, the level of possible occurrence of an incident in past monitoring video of an incident. This video surveillance system detects incidents from new surveillance videos based on registered attribute information.
  • Patent Document 1 The technology described in Patent Document 1 is based on the premise that the user who has checked the surveillance video can register appropriate attribute information about the incident. There is a problem that it cannot be detected accurately.
  • An object of the present disclosure is to provide a video monitoring system and a video monitoring method capable of detecting specific events without a user having specialized knowledge.
  • a video surveillance system includes a feature search for searching for desired first video data from a plurality of feature data indicating features of a subject captured in each of a plurality of first video data.
  • a storage unit for storing specific event data extracted in response to a query; an analysis unit for generating analysis result data indicating characteristics of a subject appearing in the second image data; and based on the analysis result data and the specific event data and a detection unit for detecting the presence or absence of a specific event, which is a specific monitoring event appearing in the second video data.
  • FIG. 1 is a diagram illustrating an example of a video monitoring system according to a first embodiment of the present disclosure
  • FIG. It is a figure which shows an example of a search query table.
  • FIG. 10 is a diagram showing an example of a frame-by-frame table; It is a figure which shows an example of a table classified by tracking ID.
  • FIG. 4 is a diagram illustrating another example of the video monitoring system according to the first embodiment of the present disclosure;
  • FIG. 9 is a flowchart for explaining an example of specific event detection processing; 6 is a flowchart for explaining an example of specific event determination processing; It is a figure for demonstrating an example of a specific event determination process.
  • FIG. 4 is a diagram illustrating another example of the video monitoring system according to the first embodiment of the present disclosure; FIG. It is a figure which shows an example of the video monitoring system of 2nd Embodiment of this indication.
  • FIG. 10 is a diagram illustrating another example of the video monitoring system according to the second embodiment of the present disclosure; FIG. 9 is a flowchart for explaining another example of specific event detection processing;
  • the video surveillance system of this embodiment described below extracts and accumulates specific event data, which is data relating to a specific event, which is a specific monitoring event, from feature data indicating the features of a predetermined subject appearing in video data, This is a system that detects a specific event appearing in video data to be monitored based on the accumulated specific event data.
  • the specific event is, for example, an incident such as a crime or an accident, an event that requires security, police or ambulance action, or an event that is a sign of the occurrence of such an incident or event.
  • FIG. 1 is a diagram showing a configuration for accumulating specific event data in the video surveillance system according to the first embodiment of the present disclosure.
  • the video monitoring system includes a video distribution device 1, a video processing device 2, an FDB (Feature Database) server 3, a user terminal 4, a video search device 5, and a specific event DB server 6.
  • the devices 1 to 6 are connected to communicate with each other via, for example, a communication network (not shown).
  • the video distribution device 1 is a device that distributes video data.
  • the video distribution device 1 is a camera that acquires and distributes video data of a predetermined monitoring location, but it may be a recorder or a VMS (Video Management System).
  • Video data is moving image data composed of a plurality of frames in this embodiment.
  • the video distribution apparatus 1 distributes only the video data, or distributes the video data with meta information on the video data.
  • the meta information includes, for example, a camera ID, which is identification information for identifying the video distribution apparatus 1, and an acquisition date and time, which is the date and time when the video data was acquired.
  • the video processing device 2 is a device that performs predetermined video analysis processing on video data distributed by the video distribution device 1 .
  • the video processing device 2 specifically has a video input unit 21 , a video analysis unit 22 , and an FDB registration unit 23 .
  • the video input unit 21 receives video data distributed by the video distribution device 1 .
  • the video data is the first video data for generating the specific event data in the example of FIG.
  • the image analysis unit 22 performs predetermined image analysis processing on the image data received by the image input unit 21 .
  • the video analysis processing includes feature estimation processing for estimating (obtaining) features of a predetermined subject appearing in video data.
  • the predetermined subject is a person in this embodiment, it may be an animal or an object other than a person.
  • the feature estimation processing includes person detection processing for detecting a person who is a predetermined subject, attribute estimation processing for estimating attributes of the detected person detected by the person detection processing, and action estimation processing for estimating the behavior of the detected person.
  • the attributes and behavior of the detected person are features. Attributes are appearance attributes (attributes determined from appearance), such as age (age division), gender, and color of clothes. Actions include, for example, running and looking around (looking around).
  • the video analysis processing includes processing for performing feature estimation processing, such as background recognition processing for recognizing the background of video data and skeleton estimation processing for estimating the skeleton of a detected person.
  • the feature estimation processing is performed for each frame of the video data, and includes tracking processing for identifying the common person when a common person appears in a plurality of frames.
  • the feature estimation process may also include a process of calculating an image feature amount of an area in which a person is shown in a frame of video data as a person image feature amount.
  • the FDB registration unit 23 registers the video data received by the video input unit 21 and feature data indicating the result of video analysis processing on the video data by the video analysis unit 22 in the FDB server 3 .
  • the feature data is, for example, data in which a camera ID, an acquisition date and time, and feature information indicating the features of a person captured in the frame are associated with each frame of video data. If meta information is not attached to the video data, the FDB registration unit 23 may generate the camera ID and the date and time of acquisition.
  • the feature information has a plurality of estimation items corresponding to each feature type.
  • the estimation item includes an action item indicating a feature related to a person's behavior, an attribute item indicating a feature related to a person's attribute, and an image feature item indicating a person image feature amount. There may be multiple action items and attribute items.
  • the item value of the action item indicates whether or not there is an action corresponding to the action item.
  • the item value of the attribute item indicates the attribute corresponding to the attribute item. If the attribute is an age category, the item value may be, for example, "0" if under 15 years old, “1” if between 15 and 45 years old, “2” if between 45 and 60 years old, and “2" if over 60 years old. If there is, it indicates "3".
  • the FDB server 3 is a server having an FDB 31, which is a storage unit for storing video data and feature data.
  • the user terminal 4 is a terminal device operated by a user who uses the video surveillance system.
  • the user terminal 4 has an input section 41 and a display section 42 .
  • the input unit 41 receives various information such as a video search query, which is a search query (search condition) for searching for desired video data from the user, and transmits the information to the video search device 5 .
  • the display unit 42 receives and displays various information such as search results that are responses to video search queries.
  • the video search query includes, for example, camera ID, acquisition date and time, and search conditions related to feature information.
  • the video search device 5 is a device that searches video data from the FDB 31 based on video search queries from the user terminal 4 .
  • the video search device 5 has a video search unit 51 and a specific event DB registration unit 52 .
  • the video search unit 51 searches the FDB 31 for video data that matches the video search query from the user terminal 4 and transmits the search results to the user terminal 4 . Specifically, based on the feature data stored in the FDB 31, the video search unit 51 searches for video data showing a person to be tracked, who has features matching the search conditions of the video search query.
  • a specific event DB registration unit 52 extracts feature data corresponding to a video search query from the feature data stored in the FDB 31 of the FDB server 3 as specific event data relating to the specific event and registers it in the specific event DB server 6. Department. Specifically, the specific event DB registration unit 52 registers the feature data corresponding to the searched video data, which is the video data searched by the video search unit 51 in response to the video search query, as specific event data to the specific event DB server. Register for 6.
  • the specific event data includes a frame-by-frame table in which feature data is aggregated for each frame, and a tracking ID-by-tracking ID table in which feature data is aggregated for each tracked person. The table by tracking ID is generated from the table by frame.
  • the specific event DB registration unit 52 may register the video search query in the specific event DB server 6 .
  • the specific event DB registration unit 52 may register, for example, in the form of a search query table in which video search queries are collected, or may be registered in the form of addition to a frame-by-frame table, or both. You can register in the form
  • the specific event DB server 6 is a server having a specific event DB 61, which is a storage unit for storing video search queries and specific event data.
  • FIG. 2 is a diagram showing an example of a search query table.
  • a search query table 200 shown in FIG. 2 is table information having each video search query as a record, and includes fields 201-204.
  • a field 201 stores the camera ID of the video distribution device 1 that acquired the video data.
  • a field 202 stores the time period when the video data was acquired.
  • a field 203 is provided for each action item of feature information, and stores action feature designation information indicating whether or not the action item is designated as a search target. In the example of FIG. 2, the action feature specifying information indicates "1" when the action item is specified, and indicates "0" when the action item is not specified.
  • a field 204 is provided for each attribute item of feature information, and stores attribute feature specifying information indicating a specified value specifying one of the possible item values of the attribute item. Note that a video search query need not have values for all of fields 201-204.
  • FIG. 3 is a diagram showing an example of a frame-by-frame table included in feature event data.
  • a frame-by-frame table 300 shown in FIG. 3 is table information having feature data as a record, and includes fields 301 to 308 .
  • a field 301 stores the camera ID of the video distribution device 1 that acquired the retrieved video data.
  • Field 302 stores a frame ID that identifies a frame of retrieved video data.
  • Field 303 stores the time when the frame was acquired.
  • a field 304 stores a person ID that identifies a person to be tracked among person IDs that identify each person in the frame.
  • a field 305 stores a tracking ID that identifies a tracked person in the frame.
  • the person ID is an ID that identifies a person within a frame, and the tracking ID is an ID that is assigned to the same person between frames.
  • a field 306 is provided for each action item, and stores the item value and certainty of the action item for the person to be tracked in the frame.
  • a field 307 is provided for each attribute item, and stores the item value and certainty of the attribute item for the person to be tracked in the frame.
  • fields 306 and 307 store the item value and confidence in the form of a numeric string represented by "item value, confidence" such as "1, 0.9".
  • a field 308 stores the person image feature amount of the frame.
  • a field for storing the video search query is added to the frame-by-frame table 300, for example.
  • the field stores, for example, items specified in the video search query (eg, exercise items A and B, age category, etc.).
  • FIG. 4 is a diagram showing an example of a tracking ID-by-tracking ID table included in feature event data.
  • the tracking ID-by-tracking ID table 400 shown in FIG. 4 includes fields 401-406.
  • a field 401 stores the camera ID of the video distribution device 1 that acquired the video data having the frame showing the person to be tracked.
  • Field 402 stores the acquisition time of each frame depicting the tracked person.
  • Field 403 stores a tracking ID that identifies the tracked person.
  • a field 404 is provided for each action item and stores an aggregated value obtained by aggregating the item values of the action item for the tracked person.
  • a field 405 is provided for each attribute item and stores an aggregated value obtained by aggregating the item values of the attribute item for the person to be tracked.
  • a field 406 stores an aggregated value obtained by aggregating the person image feature amount of each frame showing the person to be tracked.
  • FIG. 5 is a flowchart for explaining an example of video search processing of the video surveillance system.
  • the input unit 41 of the user terminal 4 receives a video search query from the user and transmits the video search query to the video search device 5 .
  • the video search unit 51 of the video search device 5 searches the FDB server 3 for video data showing a person to be tracked, who has characteristics matching the video search query.
  • the specific event DB registration unit 52 registers the video search query as a record of the search query table in the specific event DB 61 of the specific event DB server 6 (step S102).
  • the video search unit 51 transmits a list of searched video data, which is the searched video data, to the user terminal 4 as a search result.
  • the display unit 42 of the user terminal 4 receives the search result and displays the search result (step S103).
  • the search result may include a thumbnail image of the searched video data.
  • the input unit 41 receives a selection request from the user to select one of the search video data shown in the search results, and transmits the selection request to the video search device 5 .
  • the video search device 5 transmits to the user terminal 4 selected video data, which is search video data selected in response to a selection request from the user terminal 4 .
  • the display unit 42 of the user terminal 4 receives the selected video data and displays (reproduces) the selected video data (step S104).
  • the specific event DB registration unit 52 of the video search device 5 identifies a person who most accurately matches the video search query from the selected video data as a person to be tracked (step S105).
  • the specific event DB registration unit 52 extracts the tracking ID for identifying the target person from the feature data corresponding to the selected video data stored in the FDB 31 (step S106).
  • the specific event DB registration unit 52 extracts the feature data including the extracted tracking ID (feature data corresponding to each of all frames showing the person to be tracked) from the FDB 31, and uses the feature data as a frame-by-frame table for the specific event. It is registered in the specific event DB 61 of the DB server 6 (step S107).
  • the specific event DB registration unit 52 creates a table by tracking ID based on the table by frame, registers it in the specific event DB 61 of the specific event DB server 6 (step S108), and ends the search process.
  • the item value of each estimated item in the table by tracking ID is determined, for example, by one of the following three first to third methods.
  • the specific event DB registration unit 52 adds, for each estimated item, the item value with the highest appearance frequency among the item values whose certainty is equal to or higher than the judgment threshold in the frame-by-frame table. value.
  • the specific event DB registration unit 52 specifies, for each estimated item, the item value with the highest appearance frequency in the frame-by-frame table as the mode estimate value, and for the action item, the mode value is used as the mode estimate value.
  • a value obtained by multiplying the frequency of appearance of the estimated most frequent value is used as the item value of the table by tracking ID, and for attribute items, the estimated most frequent value is used as the item value of the table by tracking ID.
  • the appearance frequency is the ratio of the number of frames corresponding to the estimated value to the total number of frames, which is the total number of frames in which the target person of the tracking ID is captured.
  • the specific event DB registration unit 52 calculates, for each estimated item, a vector value obtained by connecting the item values corresponding to each frame in the frame-by-frame table as the item value of the tracking ID-by-tracking table. At this time, the specific event DB registration unit 52 may normalize the vector value to a predetermined number of dimensions by, for example, thinning out elements of the vector value.
  • the specific event DB registration unit 52 registers the feature data corresponding to the video search query as the specific event data. 4 directly from the user.
  • the specific event DB registration unit 52 may provide the user terminal 4 with an interface for specifying feature data.
  • a search threshold which is a threshold for the degree of certainty of the item values, may be included.
  • the fields 203 and 204 of the search query table shown in FIG. 2 store numeric strings represented by "specified value, search threshold" such as "1, 0.8".
  • search threshold when the action item corresponding to the action A (running) is "1, 0.8", the video search unit 51 searches for video data with a certainty factor of 0.8 or more and judged to be "1".
  • the search threshold may be adjusted by the specific event DB registration unit 52 .
  • FIG. 6 is a flowchart for explaining an example of adjustment processing for adjusting the search threshold.
  • the adjustment process is executed when the video search unit 51 of the video search device 5 receives a video search query from the user terminal 4 .
  • the adjustment process may be executed when the video search unit 51 of the video search device 5 receives an adjustment instruction from the user terminal 4 based on the user's operation.
  • the specific event DB registration unit 52 refers to the frame-by-frame table registered in the specific event DB 61 of the specific event DB server 6 (step S201). Then, the specific event DB registration unit 52 acquires the certainty factor of each item value of each estimated item included in the video search query from the frame-by-frame table (step S202).
  • the specific event DB registration unit 52 calculates the minimum value as the statistical value of the certainty of each item value for each estimated item (step S203).
  • the statistical value is not limited to the minimum value, and may be, for example, an average value or a value obtained by subtracting the standard deviation from the average value.
  • the specific event DB registration unit 52 transmits to the user terminal 4 a request screen requesting selection of whether to adjust the search threshold.
  • the display unit 42 of the user terminal 4 receives the request screen and displays the request screen (step S204).
  • the input unit 41 transmits the adjustment request to the video search device 5 .
  • the specific event DB registration unit 52 of the video search device 5 sets the search threshold value of each item value to the minimum value of the certainty factor for each estimated item (step S205), and ends the adjustment process. do.
  • the specific event DB registration unit 52 may automatically set the search threshold for the item value to the minimum value of the certainty factor without displaying the request screen.
  • FIG. 7 shows a configuration for detecting specific events in a video surveillance system.
  • the video monitoring system shown in FIG. 7 differs from the video monitoring system shown in FIG. 1 in that it has a specific event search device 7 instead of the video search device 5 .
  • the video surveillance system may include both the video search device 5 and the specific event search device 7, or the video search device 5 and the specific event search device 7 may be physically the same device. good.
  • the video data received by the video input unit 21 of the video processing device 2 from the video distribution device 1 is the second video data to be detected for detecting whether or not the specific event is captured.
  • the video analysis unit 22 performs video analysis processing on the second video data, and transmits the processing result to the specific event search device 7 as analysis result data.
  • the video analysis processing is the same as the processing described with reference to FIG. 1 and the like, and the analysis result data may be data similar to the feature data.
  • the specific event search device 7 is a detection device that detects whether or not a specific event appears in video data.
  • the specific event search device 7 has a specific event search section 71 and an integrated display section 72 .
  • the specific event search unit 71 retrieves analysis result data, which is the result of video analysis processing on video data by the video analysis unit 22 of the video processing device 2, and specific event data stored in the specific event DB 61 of the specific event DB server 6. Based on the above, it is detected whether or not the specific event appears in the video data.
  • the integrated display unit 72 integrates the alert information indicating the specific event and video data, and transmits the alert information and the video data to the user terminal 4. It is displayed on the display unit 42 of the user terminal 4 .
  • Alert information is integrated with video data as pop-up information, for example.
  • the specific event data stored in the specific event DB 61 in addition to the data accumulated in the video monitoring system shown in FIG. It may be added based on feature data, specific event data, and the like. Also, the video surveillance system may upload feature data and specific event data to the cloud.
  • FIG. 8 is a flowchart for explaining an example of specific event detection processing for detecting a specific event by the video surveillance system.
  • the video analysis unit 22 performs video analysis processing on the video data, and receives analysis result data, which is the result of the video analysis processing. and video data to the specific event search device 7 (step S302).
  • the specific event search unit 71 of the specific event search device 7 generates an event search query, which is a search query for searching for specific event data, based on the analysis result data from the video processing device 2 .
  • the specific event search unit 71 searches for specific event data stored in the specific event DB 61 based on the event search query (step S303).
  • the specific event search unit 71 analyzes the search results and determines whether or not the video data contains a specific event (step S304).
  • the integrated display unit 72 transmits video data to the user terminal 4.
  • the display unit 42 of the user terminal 4 receives the video data, displays the video data (step S305), and ends the process.
  • the integrated display unit 72 generates alert information and transmits the video data and the alert information to the user terminal 4.
  • the display unit 42 of the user terminal 4 receives the video data and the alert information, displays the video data and the alert information (step S306), and ends the process.
  • FIG. 9 is a flowchart for explaining in more detail the specific event determination process, which is the process of steps S303 to S306 in FIG.
  • FIG. 10 is a diagram for explaining an example of the specific event determination process.
  • the specific event search unit 71 acquires analysis result data from the video processing device 2 (step S401).
  • the analysis result data includes not only the processing result corresponding to the latest frame in the video data, but also the processing result associated with the tracking ID of each person appearing in the latest frame.
  • the specific event search unit 71 converts the analysis result data into data by tracking ID that summarizes the feature data related to the person to be tracked for each tracking ID (step S402).
  • the conversion method is the same as the method for converting the table by tracking ID from the table by frame.
  • FIG. 10 shows analysis result data 500 as an example of analysis result data by tracking ID.
  • the specific event search unit 71 generates an event search query using, as a search condition, a vector composed of item values of the action item (item 501 in FIG. 10) that is the first item in the analysis result data by tracking ID. Then, a table by tracking ID stored in the specific event DB 61 is searched (step S403).
  • the specific event search unit 71 calculates the degree of similarity between the event search query and each record in the table by tracking (table 550 in FIG. 10) in the action item, and searches each record in descending order from the one with the highest degree of similarity. Sort.
  • the degree of similarity for example, the reciprocal of the distance between features such as the Euclidean distance can be used.
  • the specific event search unit 71 calculates a statistical value such as the sum or average value of the similarity of each action item as the similarity between the event search query and each record in the tracking table. good too.
  • the specific event search unit 71 removes records with a degree of similarity less than a predetermined degree of similarity from the table by tracking, and extracts data with a degree of similarity equal to or greater than the predetermined degree of similarity (step S404). If the number of extracted data is equal to or less than the set value, the specific event search unit 71 may extract data with the highest degree of similarity from the tracking-specific table.
  • the set value is 0, for example.
  • the specific event search unit 71 analyzes the distribution of item values for each of the second items (attribute item, camera ID, acquisition time, etc.) other than the action item, and determines the bias (locality) of the item value distribution. is calculated (step S405).
  • the evaluation value is, for example, standard deviation.
  • the specific event search unit 71 extracts items with an evaluation value equal to or greater than a predetermined evaluation value as biased items (step S406).
  • the specific event search unit 71 compares the item value of the event search query with the mode value, which is the item value with the highest frequency in the tracking table, for the biased item (step S407).
  • the specific event search unit 71 determines that the video data contains a specific event, and generates alert information (step S408). In this embodiment, when the item values of all the biased items match, the specific event search unit 71 generates a warning alert indicating that the specific event is photographed with high probability as alert information, and If the item values partially match, a caution alert is generated as alert information indicating that a specific event may be captured.
  • the specific event search unit 71 may associate the alert information with a specific event ID that identifies video data determined to include a specific event and store them in the specific event DB 61 . Further, if the item values do not match for all biased items, the specific event search unit 71 associates low alert information indicating that the possibility of the specific event being captured is low with the specific event ID, and You may store in DB61. Also, caution alerts may be used instead of low alert information.
  • FIG. 11 shows a configuration for displaying a list of specific events in the video surveillance system.
  • the video monitoring system shown in FIG. 11 is different from the video monitoring system shown in FIG. 7 in that the specific event search device 7 includes a content generation unit 73 instead of the specific event search unit 71 and the integrated display unit 72. different in that respect.
  • the specific event search device 7 may include a specific event search section 71 , an integrated display section 72 and a content generation section 73 .
  • the content generation unit 73 generates a list of specific event IDs stored in the specific event DB 61 as a specific event list, and displays on the display unit 42 an event selection screen for selecting one of the specific event IDs. After that, when the input unit 41 selects one of the specific event IDs, the content generation unit 73 acquires video data corresponding to the selected specific event ID from the FDB 31 or the like, and transmits the video data to the user terminal 4. , is displayed on the display unit 42 . Also, the content generation unit 73 may acquire related video data related to the video data from, for example, the FDB 31, the specific event DB 61, or the video distribution device 1 (in the case of a recorder) and display it on the display unit 42.
  • Related video data is, for example, video data showing the same person as the person shown in the video data.
  • the specific event DB 61 may hold content (video data and related video data) generated by the content generation unit 73 . Also, this content may be uploaded to a cloud or the like.
  • the specific event DB 61 searches for desired first video data from a plurality of feature data indicating features of subjects captured in each of a plurality of first video data.
  • the video analysis unit 22 generates analysis result data indicating features of a subject appearing in the second video data.
  • the specific event search unit 71 detects whether or not a specific event, which is a specific monitoring event, appears in the second video data. Therefore, whether or not the specific event is captured is detected based on the past search results of searches performed by the user thinking that there is a possibility that the specific event is captured. can also detect specific events.
  • the specific event search unit 71 compares the features indicated by the analysis result data with the features indicated by the subject-specific data (each record of the tracking ID-specific table), and based on the comparison result, to detect whether or not a specific event is captured. Therefore, since the specific event is detected based on the features of images that have been searched in the past, it is possible to more accurately detect whether or not the specific event appears.
  • the specific event search unit 71 extracts subject-specific data indicating a feature whose similarity to the feature indicated by the analysis result data is equal to or higher than a predetermined similarity in the first feature item, The feature indicated by the analysis result data is compared with the feature indicated by each of the extracted object-specific data.
  • the first item is an action item. In this case, it is possible to more accurately detect whether or not the specific event is captured.
  • the specific event search unit 71 determines the value of the item. Among them, the value with the highest frequency is compared with the value of the item in the analysis result data. Then, when the values of each item match, the specific event search unit 71 detects that the specific event is captured.
  • the first item preferably includes an attribute item. In this case, it is possible to more accurately detect whether or not the specific event is captured.
  • the specific event DB registration unit 52 extracts specific event data from the feature data according to the video search query and registers it in the specific event DB 61 . Therefore, since it is possible to update the specific event data, it is possible to more accurately detect whether or not the specific event is captured.
  • the specific event DB registration unit 52 extracts feature data selected by the user from feature data matching the video search query as specific event data. Therefore, whether or not the specific event is captured is detected based on the past search results of searches performed by the user thinking that there is a high possibility that the specific event is captured. It is possible to detect with higher accuracy.
  • the integrated display unit 72 displays the alert information indicating the specific event and the second video data when the specific event is captured. Therefore, it is possible to appropriately notify the user that the specific event is captured.
  • FIG. 12 is a diagram showing the configuration for performing machine learning in the video monitoring system of this embodiment.
  • the video monitoring system shown in FIG. 12 is distinguished from the video monitoring system shown in FIG. 1 in that it further includes a machine learning device 8 .
  • the machine learning device 8 may be configured by a device that is physically the same as the video search device 5 .
  • the machine learning device 8 builds a specific event model, which is a machine learning model for detecting specific events, based on the specific event data stored in the specific event DB 61 of the specific event DB server 6.
  • the machine learning device 8 has a specific event learning section 81 and a specific event model section 82 .
  • the specific event learning unit 81 uses the specific event data stored in the specific event DB 61 of the specific event DB server 6 as learning data to classify the analysis result data of the video analysis unit 22 into classes based on the presence or absence of the specific event. build a model; For example, the specific event learning unit 81 performs multi-label machine learning using the camera ID and acquisition time of the specific event data as data, and the item values (estimated classes) and certainty of the action items and attribute items as teacher labels. By doing so, a specific event model is constructed.
  • the machine learning technique is not particularly limited, but may be, for example, multiple regression analysis.
  • the specific event model unit 82 holds the specific event model constructed by the specific event learning unit 81.
  • FIG. 13 is a diagram showing a configuration for detecting a specific event using a machine learning model in the video surveillance system of this embodiment.
  • the video monitoring system shown in FIG. 13 differs from the video monitoring system shown in FIG. 12 in that it has a specific event search device 7 and a machine learning device 8 instead of the video search device 5 .
  • the specific event search device 7 differs from the specific event search device 7 shown in FIG. 7 in that it includes a specific event estimation unit 74 instead of the specific event search unit 71 .
  • the machine learning device 8 shows only the specific event model section 82 in FIG. 13 .
  • the specific event estimation unit 74 uses the specific event model set in the specific event model unit 82 to detect whether or not the specific event appears in the video data based on the analysis result data and the specific event data. Department.
  • FIG. 14 is a flowchart for explaining an example of specific event detection processing for detecting a specific event by the video monitoring system according to this embodiment.
  • the video analysis unit 22 performs video analysis processing on the video data, and receives analysis result data, which is the result of the video analysis processing. and video data to the specific event search device 7 (step S502).
  • the specific event estimation unit 74 of the specific event search device 7 inputs the analysis result data from the video processing device 2 to the specific event model held in the specific event model unit 82 of the machine learning device 8 (step S503).
  • the specific event estimator 74 checks the output of the specific event model and determines whether or not the specific event appears in the video data (step S504).
  • the integrated display unit 72 transmits video data to the user terminal 4.
  • the display unit 42 of the user terminal 4 receives the video data, displays the video data (step S505), and ends the process.
  • the integrated display unit 72 generates alert information and transmits the video data and the alert information to the user terminal 4.
  • the display unit 42 of the user terminal 4 receives the video data and the alert information, displays the video data and the alert information (step S506), and ends the process.
  • the user who has confirmed the alert information may determine whether or not the specific event is captured in the video data, and if the specific event is not captured, input to the input unit 41 to that effect.
  • the specific event estimator 74 may update the specific event data stored in the specific event DB 61 accordingly. In this case, since the learning data used for machine learning by the specific event learning unit 81 is updated, it is possible to update the specific event model.
  • Each of the devices 1 to 8 described above may be configured by a computer system including a processor (computer) and memory (both not shown), for example.
  • the components and functions of each of the devices 1 to 8 are implemented by, for example, reading a computer program by a processor and executing the read computer program.
  • the computer program can be recorded in a computer-readable recording medium such as memory.
  • each of the devices 1 to 8 may have an input/output device for inputting/outputting information, an auxiliary storage device for storing information, and the like, as required.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Emergency Management (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Alarm Systems (AREA)
  • Image Analysis (AREA)

Abstract

L'objectif de la présente invention est de détecter un événement spécifique même sans la présence d'un utilisateur ayant des connaissance spécialisées. Selon la présente invention, une base de données d'événements spécifiques (61) stocke des données d'événements spécifiques extraites d'une pluralité de données de caractéristiques indiquant les caractéristiques d'un sujet apparaissant dans chaque données de la pluralité de premières données vidéo, les données d'événements spécifiques étant extraites conformément à une demande de recherche relative à des caractéristiques permettant de rechercher les premières données vidéo souhaitées. Une unité d'analyse vidéo (22) génère des données de résultat d'analyse indiquant les caractéristiques d'un sujet apparaissant dans des secondes données vidéo. Une unité de recherche d'événements spécifiques (71) détecte, d'après les données de résultat d'analyse et les données d'événements spécifiques, si un événement spécifique qui est un événement de surveillance spécifique apparaît dans les secondes données vidéo.
PCT/JP2022/018073 2021-07-05 2022-04-18 Système de vidéosurveillance et procédé de vidéosurveillance WO2023281897A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2021-111778 2021-07-05
JP2021111778A JP2023008315A (ja) 2021-07-05 2021-07-05 映像監視システム及び映像監視方法

Publications (1)

Publication Number Publication Date
WO2023281897A1 true WO2023281897A1 (fr) 2023-01-12

Family

ID=84801475

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/018073 WO2023281897A1 (fr) 2021-07-05 2022-04-18 Système de vidéosurveillance et procédé de vidéosurveillance

Country Status (2)

Country Link
JP (1) JP2023008315A (fr)
WO (1) WO2023281897A1 (fr)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012078950A (ja) * 2010-09-30 2012-04-19 Sogo Keibi Hosho Co Ltd 自律移動体を用いた監視システム、監視装置、自律移動体、監視方法、及び監視プログラム
WO2014050518A1 (fr) * 2012-09-28 2014-04-03 日本電気株式会社 Dispositif, procédé et programme de traitement d'informations
JP2016119627A (ja) * 2014-12-22 2016-06-30 セコム株式会社 追跡処理装置

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012078950A (ja) * 2010-09-30 2012-04-19 Sogo Keibi Hosho Co Ltd 自律移動体を用いた監視システム、監視装置、自律移動体、監視方法、及び監視プログラム
WO2014050518A1 (fr) * 2012-09-28 2014-04-03 日本電気株式会社 Dispositif, procédé et programme de traitement d'informations
JP2016119627A (ja) * 2014-12-22 2016-06-30 セコム株式会社 追跡処理装置

Also Published As

Publication number Publication date
JP2023008315A (ja) 2023-01-19

Similar Documents

Publication Publication Date Title
CN109858365B (zh) 一种特殊人群聚集行为分析方法、装置及电子设备
US9141184B2 (en) Person detection system
CN106709424B (zh) 一种优化的监控视频存储系统
CN108229674B (zh) 聚类用神经网络的训练方法和装置、聚类方法和装置
CN111222373B (zh) 一种人员行为分析方法、装置和电子设备
CN101300588A (zh) 确定收集中的特定人的方法
US10586115B2 (en) Information processing device, information processing method, and computer program product
CN102831615A (zh) 对象监控方法和装置,以及监控系统操作方法
JP2008165701A (ja) 画像処理装置、電子機器、画像処理方法、及び、プログラム
JP2015072578A (ja) 人物特定装置、人物特定方法及びプログラム
US11023713B2 (en) Suspiciousness degree estimation model generation device
JP2011018238A (ja) 画像検索システム及び画像検索方法
CN113269127A (zh) 实时自动建库的人脸识别和行人重识别监控方法及系统
US11841902B2 (en) Information processing apparatus, information processing method, and storage medium
JP5202419B2 (ja) 警備システムおよび警備方法
JP2022003526A (ja) 情報処理装置、検出システム、情報処理方法、及びプログラム
US10902249B2 (en) Video monitoring
WO2023281897A1 (fr) Système de vidéosurveillance et procédé de vidéosurveillance
Zhang et al. A Multiple Instance Learning and Relevance Feedback Framework for Retrieving Abnormal Incidents in Surveillance Videos.
JP6604413B1 (ja) 画像検索装置、画像検索システム、画像検索方法及びプログラム
KR20150029324A (ko) 감시카메라 영상을 이용한 실시간 결제 이벤트 요약 시스템 및 그 방법
WO2022262013A1 (fr) Procédé et système de collecte et de surveillance d'informations d'état de véhicule
US11893084B2 (en) Object detection systems and methods including an object detection model using a tailored training dataset
CN112069357B (zh) 视频资源处理方法、装置、电子设备及存储介质
JP2023039656A (ja) 事例検索装置、方法及びプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22837318

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE