WO2022030549A1 - Information retrieval device, information retrieval method, and program - Google Patents

Information retrieval device, information retrieval method, and program Download PDF

Info

Publication number
WO2022030549A1
WO2022030549A1 PCT/JP2021/028964 JP2021028964W WO2022030549A1 WO 2022030549 A1 WO2022030549 A1 WO 2022030549A1 JP 2021028964 W JP2021028964 W JP 2021028964W WO 2022030549 A1 WO2022030549 A1 WO 2022030549A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
images
output
target person
unit
Prior art date
Application number
PCT/JP2021/028964
Other languages
French (fr)
Japanese (ja)
Inventor
鮎美 松本
哲希 柴田
育弘 宇田
真一 根本
篤 佐藤
知也 児玉
貴司 塩崎
Original Assignee
エヌ・ティ・ティ・コミュニケーションズ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by エヌ・ティ・ティ・コミュニケーションズ株式会社 filed Critical エヌ・ティ・ティ・コミュニケーションズ株式会社
Publication of WO2022030549A1 publication Critical patent/WO2022030549A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/53—Querying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73—Querying
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Definitions

  • An embodiment of the present invention relates to, for example, an information retrieval device, an information retrieval method, and a program for analyzing video data from a surveillance camera and searching for an image of a target person.
  • An information processing device such as a general-purpose personal computer receives video data from a camera, stores the received video data in a storage unit, analyzes the video data, and detects an image of a target person. In addition, the information processing device displays the detected image of the target person on a monitor or the like.
  • the video data contains a large number of frames, and the burden of processing related to feature extraction for the faces contained in these frames is heavy, and a technology for realizing high-speed processing with a low-cost device has been proposed (). See, for example, Patent Document 1).
  • the information processing device Since the information processing device detects the target person from a plurality of frames included in the stored video data, many similar images of the target person may be output as the detection result.
  • the present invention has been made by paying attention to the above circumstances, and is intended to provide a technique for improving the visibility of a detected image of the same person.
  • the information retrieval device is based on a search condition from a database in which detected images detected from a plurality of frames included in video data from one or more cameras are registered.
  • a search unit that searches for target person images and all target person images searched by the search unit based on the first output specification are selected, and all targets searched by the search unit based on the second output specification. It includes a selection unit that selects a first predetermined number of target person images that satisfy the output conditions and are regarded as the same person among the person images, and an output unit that outputs the target person images selected by the selection unit.
  • FIG. 1 is a diagram showing an example of a configuration of a monitoring system including a monitoring information processing apparatus according to an embodiment of the present invention.
  • FIG. 2 is a block diagram showing an example of a hardware configuration of a Web server device used as a monitoring information processing device according to an embodiment of the present invention.
  • FIG. 3 is a block diagram showing an example of a software configuration of a Web server device used as a monitoring information processing device according to an embodiment of the present invention.
  • FIG. 4 is a flowchart showing an example of a search process by the system according to the embodiment of the present invention.
  • FIG. 5 is a flowchart showing an example of a target image selection process in the search process by the system according to the embodiment of the present invention.
  • FIG. 6 is a flowchart showing an example of a target image selection process for each tracking ID within the angle of view in the search process by the system according to the embodiment of the present invention.
  • FIG. 1 is a diagram showing an overall configuration of a system including a monitoring information processing apparatus according to an embodiment of the present invention.
  • a plurality of surveillance cameras C1 to Cn are distributed and arranged in the aisles and sales floors of large-scale stores such as shopping malls and department stores.
  • surveillance cameras C1 to Cn are attached to, for example, a ceiling or a wall surface, capture images of each surveillance area, and output video data thereof.
  • the surveillance cameras C1 to Cn are equipped with video analysis engines VE1 to VEn, respectively.
  • the video analysis engines VE1 to VEn correspond to the video analysis unit, and the video analysis unit analyzes each video data from the surveillance cameras C1 to Cn.
  • the video analysis engines VE1 to VEn each perform intra-angle tracking on a plurality of image frames included in the video data output from the corresponding surveillance cameras C1 to Cn, and the images are imaged from the plurality of image frames. The same person image is determined based on the position information in the frame.
  • the video analysis engines VE1 to VEn are not arranged one-to-one with respect to the surveillance cameras C1 to Cn, but a smaller number of video analysis engines are arranged for a plurality of cameras, and a plurality of video analysis engines are used.
  • the video data of the surveillance camera may be collectively processed.
  • the system of one embodiment includes a Web server device SV used as a monitoring information processing device.
  • the video analysis engines VE1 to VEn are capable of data communication with the Web server device SV via the network NW, and transmit the generated video analysis result to the Web server device SV via the network NW.
  • NW for example, a wired LAN (Local Area Network) or a wireless LAN is used, but any other network may be used.
  • the web server device SV includes video analysis engines VE1 to VEN or one video analysis engine, and the video analysis engines VE1 to VEN of the web server device SV or one video analysis engine is a surveillance camera via a network NW.
  • Each video data from C1 to Cn may be received and the received video data may be analyzed.
  • Web server device SV 2 and 3 are block diagrams showing an example of a hardware configuration and a software configuration of the Web server device SV, respectively.
  • the Web server device SV includes a control unit 1 having a hardware processor such as a central processing unit (CPU), and the program storage unit 2 and a data storage unit are provided to the control unit 1 via a bus 6.
  • the storage unit having 3 is connected to the input / output interface (input / output I / F) 4 and the communication interface (communication I / F) 5.
  • a monitor device MT and an administrator terminal OT are connected to the input / output I / F4.
  • the monitoring device MT is used for the observer to visually monitor the monitoring area, and displays images of the surveillance cameras C1 to Cn, information indicating the detection result or the tracking result of the query to be monitored, and the like.
  • the administrator terminal OT is used by the system administrator for system management and maintenance, and displays various setting screens and information indicating the operating status in the system, and the system administrator manages and operates the system. When inputting various necessary data, it has a function of accepting the data and setting it in the Web server device SV.
  • the communication I / F5 transmits data between the video analysis engines VE1 to VEn using a communication protocol defined by the network NW under the control of the control unit 1, for example, a wired LAN or a wireless LAN. It is composed of the corresponding interfaces.
  • the program storage unit 2 includes, for example, a non-volatile memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that can be written and read at any time as a storage medium, and a non-volatile memory such as a ROM (Read Only Memory).
  • a non-volatile memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that can be written and read at any time as a storage medium
  • a non-volatile memory such as a ROM (Read Only Memory).
  • middleware such as an OS (Operating System)
  • OS Operating System
  • the data storage unit 3 is, for example, a combination of a non-volatile memory such as an HDD or SSD capable of writing and reading at any time and a volatile memory such as a RAM (RandomAccessMemory) as a storage medium, and is one of the present inventions.
  • a camera information table 31 and a setting information table 32 are provided as main storage units necessary for implementing the embodiment.
  • the camera information table 31 stores, for example, information representing the name, performance, and installation position of the surveillance camera in association with the identification information (hereinafter referred to as the camera ID) for each of the surveillance cameras C1 to Cn.
  • Information representing performance includes, for example, resolution and aspect ratio.
  • the information indicating the installation position includes, for example, latitude / longitude, imaging direction, and imaging angle.
  • the camera information table 31 stores a database, and the database registers the tracking target person image (detection image) detected from the video data from the surveillance cameras C1 to Cn.
  • the setting information table 32 stores the image feature amount of the query.
  • the setting information table 32 stores the image feature amount of the query input from the administrator terminal OT via the input / output I / F4.
  • the setting information table 32 stores the image feature amount of the query detected from the video data transmitted from the surveillance cameras C1 to Cn via the communication I / F5.
  • the setting information table 32 stores the alert determination condition input via the administrator terminal OT or the like.
  • the setting information table 32 stores the first or second alert determination condition input via the administrator terminal OT or the like.
  • the administrator presses the tracking button on the administrator terminal OT for the person (image) to be tracked.
  • the control unit 1 automatically registers the latest set of detected images (face image and whole body image) as a query image (query image feature amount) in response to pressing the tracking button, and starts tracking.
  • the administrator presses the history button on the administrator terminal OT for the person (image) that the administrator wants to track.
  • the control unit 1 selects an arbitrary image from the history list, registers it as a query image, and starts tracking.
  • control unit 1 performs a person search from the image of the surveillance camera according to the history search from the administrator, registers the image selected from the person search results by the administrator as a query image, and starts tracking. .. Further, the administrator selects a person (image) included in the surveillance image data obtained in real time, and the control unit 1 registers the selected person as a query image and starts tracking. In addition, the administrator takes in the image provided by the requester from the administrator terminal OT, registers it as a query image, and starts tracking.
  • the control unit 1 has an information acquisition unit 11, an image detection unit 12, a search condition setting unit 13, an image search unit 14, and an image selection unit (filter unit) 15 as processing functions according to an embodiment of the present invention. And an output unit 16. Each unit is realized by causing the hardware processor of the control unit 1 to execute the program stored in the program storage unit 2.
  • the information acquisition unit 11 acquires video data and video analysis results from the video analysis engines VE1 to VEn connected to the surveillance cameras C1 to Cn or the video analysis engines VE1 to VEn provided in the Web server device SV.
  • the video analysis engines VE1 to VEn each determine the same person from a plurality of image frames included in the video data output from the corresponding surveillance cameras C1 to Cn based on the position information in the image frame, and the determination result is obtained. Output the video analysis result including the above.
  • the information acquisition unit 11 can acquire not only the video data from the surveillance cameras C1 to Cn but also the moving image file input via the input / output I / F4 or the communication I / F5.
  • the information acquisition unit 11 transfers a moving image file having a file name in the specified format to the specified folder.
  • the information acquisition unit 11 registers the camera name and the shooting time included in the file name of the moving image file. This makes it possible to search for moving image files by specifying the camera name or shooting time. Searching based on video files allows image search without the need for real-time tracking. Since it does not require real-time tracking processing, it is possible to import video files at high speed.
  • the image search process based on the moving image file is substantially the same as the image search process based on the video data from the surveillance cameras C1 to Cn, except that the real-time tracking process is not required.
  • the image detection unit 12 comprehensively analyzes the video analysis result and the video data from the surveillance cameras C1 to Cn to detect the tracked person image.
  • the video analysis engines VE1 to VEn are, for example, from a plurality of image frames included in the video data from the surveillance cameras C1 to Cn based on the image feature amount (feature amount of the tracked person image) of the query given in advance. , A person image (tracked person image) having an image feature amount similar to the image feature amount of the query is extracted. For example, a plurality of queries are given in advance, and a plurality of person images having an image feature amount similar to the image feature amounts of the plurality of queries are extracted.
  • the video analysis engines VE1 to VEn include information indicating the degree of similarity between the extracted person image and the query image, the camera IDs of the surveillance cameras C1 to Cn, the tracking ID within the angle of view, and the shooting time (date, time, minute, and second). ) And the video analysis result including.
  • the portrait image includes a face image and a whole body image
  • the similarity information includes the similarity corresponding to each of the face image and the whole body image.
  • the camera ID is identification information unique to the surveillance camera.
  • the in-angle tracking ID is an ID for tracking images regarded as the same person in the same surveillance camera.
  • the image detection unit 12 registers the detected image information including the detected tracked person image (detected image) in the database of the camera information table 31.
  • the image detection unit 12 detects a huge number of tracked person images from a plurality of frames included in each video data from the surveillance cameras C1 to Cn, and stores a huge number of detected image information in the camera information table.
  • the detected image information includes the detected image and the attribute information.
  • the attribute information includes a camera ID, a tracking ID within the angle of view, a score of personality, an image type (face image or whole body image), a shooting date and time, and the like.
  • the database of the camera information table 31 not only the tracking target person image is registered as a search target, but also various person images are registered.
  • the search condition setting unit 13 sets the search conditions to be read from the setting information table 32.
  • the search condition includes a period (start date and time and end date and time). Further, the search condition may include a camera ID, or may include a search target image (face image, whole body image, or face and whole body image).
  • the image search unit 14 searches for the target person image according to the search condition from the database in which the detected image information including the detected images detected from the plurality of frames included in the video data from the surveillance cameras C1 to Cn is registered. ..
  • the image search unit 14 searches for the detected image detected in the designated period as a target person image (image of a plurality of people).
  • the search result includes a plurality of images of each person.
  • the image search unit 14 searches for an image similar to the image of a certain person among the detected images detected in the designated period as a target person image (image of a certain person).
  • the search results include multiple images of a person.
  • the image selection unit 15 selects all the target person images searched by the image search unit 14 based on the first output designation (for example, the filter function OFF) of the setting information table 32. Further, the image selection unit 15 satisfies the output conditions among all the target person images searched by the image search unit 14 based on the second output designation (for example, the filter function is ON) of the setting information table 32, and is regarded as the same person.
  • a first predetermined number of subject images are selected. The first predetermined number is one or two or more arbitrarily set.
  • the output unit 15 outputs the target person image selected by the image selection unit 15.
  • the output unit 16 outputs a target person image to be displayed on the monitor device MT.
  • the first output designation is set in the setting information table 32
  • all the target person images searched by the image search unit 14 are displayed on the monitor device MT
  • the second output designation is made in the setting information table 32.
  • a first predetermined number of target person images that satisfy the output conditions and are regarded as the same person among all the target person images searched by the image search unit 14 are displayed on the monitor device MT. To.
  • the case where the tables 31 and 32 provided in the data storage unit 3 are provided in the Web server device SV is taken as an example.
  • the present invention is not limited to this, and it may be provided in a database server or a file server arranged outside the Web server device SV.
  • the Web server device SV accesses the tables 31 and 32 in the database server or the file server, and performs each process by acquiring necessary information.
  • FIG. 4 is a flowchart showing an example of a search process by the system according to the embodiment of the present invention.
  • Surveillance cameras C1 to Cn start shooting and output video data (ST1).
  • the video analysis engines VE1 to VEN analyze video data from the corresponding surveillance cameras C1 to Cn, respectively (ST2).
  • the video analysis engines VE1 to VEn each perform intra-angle tracking on a plurality of image frames included in the video data output from the corresponding surveillance cameras C1 to Cn, and the images are imaged from the plurality of image frames. The same person is determined based on the position information in the frame.
  • the video analysis engines VE1 to VEn output video data and the same person determination.
  • the communication I / F5 of the Web server device SV receives the video data and the same person determination from the video analysis engines VE1 to VEn.
  • the information acquisition unit 11 acquires video data from the video analysis engines VE1 to VEn and determination of the same person (ST3).
  • the image detection unit 12 comprehensively analyzes the video data from the video analysis engines VE1 to VEn and the same person determination, and detects the person image from a plurality of frames included in the video data from the video analysis engines VE1 to VEn. (ST4). For example, the number of frames is enormous, and the number of detected human images is enormous.
  • the image detection unit 12 registers the detected image information including the detected person image in the database of the camera information table 31 (ST5).
  • the search condition setting unit 13 reads the search condition from the setting information table 32 according to the instruction from the administrator terminal OT, and sets the search condition in the image search unit 14.
  • the search condition includes a start date and time D1 and an end date and time D2.
  • the search condition setting unit 13 reads the first or second output designation from the setting information table 32 according to the instruction from the administrator terminal OT, and sets the first or second output designation to the image selection unit 15. do.
  • the image search unit 14 searches the database registered in the camera information table 31 for the target person image according to the set search condition (ST6).
  • the image search unit 14 refers to the attribute information included in the detected image information registered in the database, and searches for the target person image included in the period of the start date / time D1 and the end date / time D2.
  • the image selection unit 15 selects all or part of the target person images searched by the image search unit 14 based on the first or second output designation (ST7).
  • the output unit 16 outputs the target person image selected by the image selection unit 15 to the monitoring device MT via the input / output I / F4 or the like (ST8).
  • FIG. 5 is a flowchart showing an example of a target image selection process in the search process by the system according to the embodiment of the present invention.
  • FIG. 5 is a flowchart illustrating the details of ST7 shown in FIG.
  • the image selection unit 15 is the all target image searched by the image search unit 14 based on the first output designation. Is selected (ST702).
  • the image selection unit 15 is searched by the image search unit 14 based on the second output designation.
  • a first predetermined number of subject images that satisfy the output conditions and are considered to be the same person are selected from all the subject images (ST704).
  • the image selection unit 15 determines the second output designation. Based on this, one target person image that satisfies the output conditions and is regarded as the same person is selected, and the output unit 16 outputs one selected target person image. In this case, the subject image having the highest score of civilization satisfies the output condition.
  • the image selection unit 15 is the second. Based on the output designation, two target person images that satisfy the output conditions and are regarded as the same person are selected, and the output unit 16 outputs the two selected target person images. In this case, among the facial images, the top two subject images having a high score of civilization satisfy the output condition.
  • the output unit 16 When all the target person images searched by the image search unit 14 are selected based on the first output designation, the output unit 16 outputs all the selected target person images to the monitoring device MT. Further, the output unit 16 selects a first predetermined number of target person images that satisfy the output conditions and are regarded as the same person among all the target person images searched by the image search unit 14 based on the second output designation. If so, the selected first predetermined number of target images are output to the monitoring device MT.
  • the image selection unit 15 sets the target person image up to a first predetermined number or a first predetermined number considered to be the same person for the video data from one camera based on the second output designation. You may choose.
  • the output unit 16 outputs the target person image up to the first predetermined number or the first predetermined number regarded as the same person for each camera, and the monitoring device MT outputs the target person image. Is displayed.
  • the image selection unit 15 has a first predetermined number that is regarded as the same person for a plurality of frames for a continuous predetermined time included in the video data from one camera.
  • a target person image whose upper limit is the first predetermined number may be selected.
  • the output unit 16 outputs the target person image up to the first predetermined number or the first predetermined number regarded as the same person, and the monitoring device MT displays the output target person image. It is possible to display a first predetermined number of the same person at predetermined time intervals, which is excellent in visibility and enables detailed confirmation.
  • FIG. 6 is a flowchart showing an example of a target image selection process for each tracking ID within the angle of view in the search process by the system according to the embodiment of the present invention.
  • FIG. 6 is a flowchart illustrating the details of ST7 shown in FIG.
  • the image selection unit 15 receives the search result from the image search unit 14 (ST711) and reads the search result file (ST712).
  • the search result file includes the target person image information
  • the target person image information includes the target person image and the attribute information.
  • the attribute information includes a camera ID, a tracking ID within the angle of view, a score of personality, an image type (face image or whole body image), a shooting date and time, and the like.
  • the image selection unit 15 sorts the target person images according to the tracking ID within the angle of view (ST713). For example, the image selection unit 15 rearranges the target person images with the first priority as the date and time and the second priority as the score of person-likeness.
  • the image selection unit 15 sorts the images in descending order of similarity.
  • the image selection unit 15 sorts the images in descending order of the face image similarity for the images whose face image similarity exceeds the threshold, and the face image similarity is high. For images that do not exceed the threshold, the images are sorted in descending order of similarity to the whole body image.
  • the image selection unit 15 selects all the searched target image. If the tracking ID filter flag in the angle of view is True (filter function ON) (ST714, YES), the image selection unit 15 selects a part of the target person images among all the searched target person images (ST714, YES). ST715 to ST720).
  • the image selection unit 15 examines the tracking ID in each angle of view in order.
  • the target person image is registered in the display list for each tracking ID within the angle of view.
  • the output unit 16 outputs a display list, and the monitoring device MT displays a target person image based on the display list.
  • the present embodiment it is possible to provide a system, an apparatus, a method, and a program for improving the visibility of a detected image of the same person.
  • the images of the same person are included in a plurality of frames of the video data from this camera.
  • the images of the same person are included in the plurality of frames of the video data from the plurality of cameras.
  • the Web server device SV displays a large number of images of the same person by the setting of the first output designation, and displays a small number of narrowed-down images of the same person by the setting of the second output designation.
  • the image displayed by the second output designation setting is an image that does not match the purpose, it may be changed to the first output designation setting. For example, if the first output designation is used, a person can be confirmed in detail from many images, and if the second output designation is used, a person can be confirmed from a small number of images in a short time. Further, if the second output designation is used, the time required for the display process can be shortened.
  • the program according to the present embodiment may be transferred in a state of being stored in an electronic device, may be transferred in a state of being stored in a storage medium, or may be transferred by downloading via a network or the like.
  • the recording medium is a non-temporary computer-readable storage medium such as a magnetic disk, an optical disk, or a flash memory.
  • the present invention is not limited to the above embodiment, and can be variously modified at the implementation stage without departing from the gist thereof.
  • each embodiment may be carried out in combination as appropriate, in which case the combined effect can be obtained.
  • the above-described embodiment includes various inventions, and various inventions can be extracted by a combination selected from a plurality of disclosed constituent requirements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, if the problem can be solved and the effect is obtained, the configuration in which the constituent elements are deleted can be extracted as an invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

Provided is a technique for efficiently outputting images of the same person detected.　An information retrieval device according to one aspect of the present invention is provided with: a retrieval unit that retrieves, from a database in which detected images detected from a plurality of frames included in video data from at least one camera are registered, a subject image corresponding to a retrieval condition; a selecting unit that selects, on the basis of a first output designation, all subject images retrieved by the retrieval unit, and selects, on the basis of a second output designation, a first predetermined number of subject images among all subject images retrieved by the retrieval unit that satisfy an output condition and are considered to be of the same person; and an output unit that outputs the subject images selected by the selecting unit.

Description

情報検索装置、情報検索方法、及びプログラムInformation retrieval device, information retrieval method, and program
　この発明の実施形態は、例えば監視カメラからの映像データを解析して対象者の画像を検索する情報検索装置、情報検索方法、及びプログラムに関する。 An embodiment of the present invention relates to, for example, an information retrieval device, an information retrieval method, and a program for analyzing video data from a surveillance camera and searching for an image of a target person.
　近年、防犯対策の一環として、様々な場所にカメラが設置されている。カメラは、監視対象エリアを撮影し、映像データを出力する。汎用のパーソナルコンピュータ等の情報処理装置は、カメラからの映像データを受信し、受信した映像データを記憶部に記憶し映像データを解析して対象者の画像を検出する。また、情報処理装置は、検出した対象者の画像をモニタ等に表示する。 In recent years, cameras have been installed in various places as part of crime prevention measures. The camera captures the monitored area and outputs video data. An information processing device such as a general-purpose personal computer receives video data from a camera, stores the received video data in a storage unit, analyzes the video data, and detects an image of a target person. In addition, the information processing device displays the detected image of the target person on a monitor or the like.
　映像データには多数のフレームが含まれ、これらフレームに含まれる顔を対象とした特徴量抽出に関する処理の負担が重く、低コストな装置で高速処理を実現するための技術が提案されている（例えば特許文献１を参照）。 The video data contains a large number of frames, and the burden of processing related to feature extraction for the faces contained in these frames is heavy, and a technology for realizing high-speed processing with a low-cost device has been proposed (). See, for example, Patent Document 1).
日本国特許第６５６８４７６号公報Japanese Patent No. 6568476
　情報処理装置は、記憶された映像データに含まれる複数のフレームから、対象者を検出するため、検出結果として、似たような対象者の画像が多く出力されてしまうことがある。 Since the information processing device detects the target person from a plurality of frames included in the stored video data, many similar images of the target person may be output as the detection result.
　対象者が、同じ場に長時間滞在していた場合、この傾向は強まる。また、多くの画像を出力するためには、例えば、出力のための表示処理の負荷が大きくなり、表示処理に時間がかかることもある。 If the target person stayed in the same place for a long time, this tendency becomes stronger. Further, in order to output a large number of images, for example, the load of the display processing for output becomes large, and the display processing may take time.
　このように、同一人物の画像が多く出力されてしまうと、他の対象者の画像が埋もれてしまうこともある。 In this way, if many images of the same person are output, the images of other subjects may be buried.
　この発明は上記事情に着目してなされたもので、検出された同一人物の画像の視認性の向上を図る技術を提供しようとするものである。 The present invention has been made by paying attention to the above circumstances, and is intended to provide a technique for improving the visibility of a detected image of the same person.
　上記課題を解決するためにこの発明の一態様の情報検索装置は、１以上のカメラ からの映像データに含まれる複数のフレームから検出された検出画像が登録されたデータベースから、検索条件に応じた対象者画像を検索する検索部と、第１の出力指定に基づき前記検索部により検索された全ての対象者画像を選択し、第２の出力指定に基づき前記検索部により検索された全ての対象者画像のうち出力条件を満たし且つ同一人物と見なされる第１の所定数の対象者画像を選択する選択部と、前記選択部により選択される対象者画像を出力する出力部と、を備える。 In order to solve the above problems, the information retrieval device according to one aspect of the present invention is based on a search condition from a database in which detected images detected from a plurality of frames included in video data from one or more cameras are registered. A search unit that searches for target person images and all target person images searched by the search unit based on the first output specification are selected, and all targets searched by the search unit based on the second output specification. It includes a selection unit that selects a first predetermined number of target person images that satisfy the output conditions and are regarded as the same person among the person images, and an output unit that outputs the target person images selected by the selection unit.
　この発明の一態様によれば、検出された同一人物の画像の視認性の向上を図る技術を提供することができる。　 According to one aspect of the present invention, it is possible to provide a technique for improving the visibility of a detected image of the same person. It was
図１は、この発明の一実施形態に係る監視情報処理装置を含む監視システムの構成の一例を示す図である。FIG. 1 is a diagram showing an example of a configuration of a monitoring system including a monitoring information processing apparatus according to an embodiment of the present invention. 図２は、この発明の一実施形態に係る監視情報処理装置として用いられるＷｅｂサーバ装置のハードウェア構成の一例を示すブロック図である。FIG. 2 is a block diagram showing an example of a hardware configuration of a Web server device used as a monitoring information processing device according to an embodiment of the present invention. 図３は、この発明の一実施形態に係る監視情報処理装置として用いられるＷｅｂサーバ装置のソフトウェア構成の一例を示すブロック図である。FIG. 3 is a block diagram showing an example of a software configuration of a Web server device used as a monitoring information processing device according to an embodiment of the present invention. 図４は、この発明の一実施形態に係るシステムによる検索処理の一例を示すフローチャートである。FIG. 4 is a flowchart showing an example of a search process by the system according to the embodiment of the present invention. 図５は、この発明の一実施形態に係るシステムによる検索処理のうちの対象画像の選択処理の一例を示すフローチャートである。FIG. 5 is a flowchart showing an example of a target image selection process in the search process by the system according to the embodiment of the present invention. 図６は、この発明の一実施形態に係るシステムによる検索処理のうちの画角内追跡ＩＤ毎の対象画像の選択処理の一例を示すフローチャートである。FIG. 6 is a flowchart showing an example of a target image selection process for each tracking ID within the angle of view in the search process by the system according to the embodiment of the present invention.
　以下、図面を参照してこの発明に係る実施形態を説明する。 Hereinafter, embodiments according to the present invention will be described with reference to the drawings.
　［一実施形態］
　（構成例）
　（１）システム
　図１は、この発明の一実施形態に係る監視情報処理装置を含むシステムの全体構成を示す図である。　
　例えば、ショッピングモールや百貨店などの大規模店舗の通路や売り場には、複数台の監視カメラＣ１～Ｃｎが分散配置されている。監視カメラＣ１～Ｃｎは、例えば天井または壁面に取着され、それぞれの監視エリアを撮像してその映像データを出力する。 [One Embodiment]
(Configuration example)
(1) System FIG. 1 is a diagram showing an overall configuration of a system including a monitoring information processing apparatus according to an embodiment of the present invention.
For example, a plurality of surveillance cameras C1 to Cn are distributed and arranged in the aisles and sales floors of large-scale stores such as shopping malls and department stores. Surveillance cameras C1 to Cn are attached to, for example, a ceiling or a wall surface, capture images of each surveillance area, and output video data thereof.
　例えば、監視カメラＣ１～Ｃｎには、それぞれ映像解析エンジンＶＥ１～ＶＥｎが付設されている。映像解析エンジンＶＥ１～ＶＥｎは映像解析部に相当し、映像解析部は、監視カメラＣ１～Ｃｎからのそれぞれの映像データを解析する。例えば、映像解析エンジンＶＥ１～ＶＥｎはそれぞれ、対応する監視カメラＣ１～Ｃｎから出力される映像データに含まれる複数の画像フレームに対して、画角内追跡を実施し、複数の画像フレームから、画像フレーム内の位置情報等に基づき同一人物画像を判定する。 For example, the surveillance cameras C1 to Cn are equipped with video analysis engines VE1 to VEn, respectively. The video analysis engines VE1 to VEn correspond to the video analysis unit, and the video analysis unit analyzes each video data from the surveillance cameras C1 to Cn. For example, the video analysis engines VE1 to VEn each perform intra-angle tracking on a plurality of image frames included in the video data output from the corresponding surveillance cameras C1 to Cn, and the images are imaged from the plurality of image frames. The same person image is determined based on the position information in the frame.
　なお、映像解析エンジンＶＥ１～ＶＥｎは監視カメラＣ１～Ｃｎに対し一対一に配置せず、複数台のカメラに対しそれより少数の映像解析エンジンを配置して、少数の映像解析エンジンにより複数台の監視カメラの映像データを一括処理するようにしてもよい。 The video analysis engines VE1 to VEn are not arranged one-to-one with respect to the surveillance cameras C1 to Cn, but a smaller number of video analysis engines are arranged for a plurality of cameras, and a plurality of video analysis engines are used. The video data of the surveillance camera may be collectively processed.
　また、一実施形態のシステムは、監視情報処理装置として使用されるＷｅｂサーバ装置ＳＶを備える。映像解析エンジンＶＥ１～ＶＥｎは、ネットワークＮＷを介してＷｅｂサーバ装置ＳＶとの間でデータ通信が可能となっており、生成された映像解析結果をネットワークＮＷを介してＷｅｂサーバ装置ＳＶへ送信する。ネットワークＮＷには、例えば有線ＬＡＮ（Local Area Network）または無線ＬＡＮが用いられるが、他のどのようなネットワークが使用されてもよい。 Further, the system of one embodiment includes a Web server device SV used as a monitoring information processing device. The video analysis engines VE1 to VEn are capable of data communication with the Web server device SV via the network NW, and transmit the generated video analysis result to the Web server device SV via the network NW. For the network NW, for example, a wired LAN (Local Area Network) or a wireless LAN is used, but any other network may be used.
　なお、Ｗｅｂサーバ装置ＳＶが、映像解析エンジンＶＥ１～ＶＥｎ又は１つの映像解析エンジンを備え、Ｗｅｂサーバ装置ＳＶの映像解析エンジンＶＥ１～ＶＥｎ又は１つの映像解析エンジンが、ネットワークＮＷを介して、監視カメラＣ１～Ｃｎからのそれぞれの映像データを受信し、受信した映像データを解析してもよい。 The web server device SV includes video analysis engines VE1 to VEN or one video analysis engine, and the video analysis engines VE1 to VEN of the web server device SV or one video analysis engine is a surveillance camera via a network NW. Each video data from C1 to Cn may be received and the received video data may be analyzed.
　（２）Ｗｅｂサーバ装置ＳＶ
　図２および図３は、それぞれＷｅｂサーバ装置ＳＶのハードウェア構成およびソフトウェア構成の一例を示すブロック図である。　
　Ｗｅｂサーバ装置ＳＶは、中央処理ユニット（Central Processing Unit：ＣＰＵ）等のハードウェアプロセッサを有する制御部１を備え、この制御部１に対し、バス６を介して、プログラム記憶部２およびデータ記憶部３を有する記憶ユニットと、入出力インタフェース（入出力Ｉ／Ｆ）４と、通信インタフェース（通信Ｉ／Ｆ）５とを接続したものとなっている。 (2) Web server device SV
2 and 3 are block diagrams showing an example of a hardware configuration and a software configuration of the Web server device SV, respectively.
The Web server device SV includes a control unit 1 having a hardware processor such as a central processing unit (CPU), and the program storage unit 2 and a data storage unit are provided to the control unit 1 via a bus 6. The storage unit having 3 is connected to the input / output interface (input / output I / F) 4 and the communication interface (communication I / F) 5.
　入出力Ｉ／Ｆ４には、例えばモニタ装置ＭＴおよび管理者端末ＯＴが接続される。モニタ装置ＭＴは、監視者が監視エリアを目視監視するために使用されるもので、監視カメラＣ１～Ｃｎの映像や、監視対象となるクエリの検知結果または追跡結果を表す情報などを表示する。 For example, a monitor device MT and an administrator terminal OT are connected to the input / output I / F4. The monitoring device MT is used for the observer to visually monitor the monitoring area, and displays images of the surveillance cameras C1 to Cn, information indicating the detection result or the tracking result of the query to be monitored, and the like.
　管理者端末ＯＴは、システム管理者がシステム管理や保守等のために使用するもので、各種設定画面やシステム内の動作状態を表す情報を表示すると共に、システム管理者がシステムの管理・運用に必要な種々データを入力したときに当該データを受け付けてＷｅｂサーバ装置ＳＶに設定する機能を有する。 The administrator terminal OT is used by the system administrator for system management and maintenance, and displays various setting screens and information indicating the operating status in the system, and the system administrator manages and operates the system. When inputting various necessary data, it has a function of accepting the data and setting it in the Web server device SV.
　通信Ｉ／Ｆ５は、制御部１の制御の下、ネットワークＮＷにより定義される通信プロトコルを使用して、映像解析エンジンＶＥ１～ＶＥｎとの間でデータ伝送を行うもので、例えば有線ＬＡＮまたは無線ＬＡＮに対応するインタフェースにより構成される。 The communication I / F5 transmits data between the video analysis engines VE1 to VEn using a communication protocol defined by the network NW under the control of the control unit 1, for example, a wired LAN or a wireless LAN. It is composed of the corresponding interfaces.
　プログラム記憶部２は、例えば、記憶媒体としてＨＤＤ（Hard Disk Drive）またはＳＳＤ（Solid State Drive）等の随時書込みおよび読出しが可能な不揮発性メモリと、ＲＯＭ（Read Only Memory）等の不揮発性メモリとを組み合わせて構成したもので、ＯＳ（Operating System）等のミドルウェアに加えて、この発明の一実施形態に係る各種制御処理を実行するために必要なプログラムを格納する。 The program storage unit 2 includes, for example, a non-volatile memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that can be written and read at any time as a storage medium, and a non-volatile memory such as a ROM (Read Only Memory). In addition to middleware such as an OS (Operating System), a program necessary for executing various control processes according to an embodiment of the present invention is stored.
　データ記憶部３は、例えば、記憶媒体として、ＨＤＤまたはＳＳＤ等の随時書込みおよび読出しが可能な不揮発性メモリと、ＲＡＭ（Random Access Memory）等の揮発性メモリと組み合わせたもので、この発明の一実施形態を実施するために必要な主たる記憶部として、カメラ情報テーブル３１と、設定情報テーブル３２とを備えている。 The data storage unit 3 is, for example, a combination of a non-volatile memory such as an HDD or SSD capable of writing and reading at any time and a volatile memory such as a RAM (RandomAccessMemory) as a storage medium, and is one of the present inventions. A camera information table 31 and a setting information table 32 are provided as main storage units necessary for implementing the embodiment.
　カメラ情報テーブル３１は、監視カメラＣ１～Ｃｎ毎に、その識別情報（以後カメラＩＤと称する）に対応付けて、例えば監視カメラの名称、性能および設置位置を表す情報を記憶する。性能を表す情報には、例えば解像度やアスペクト比が含まれる。また設置位置を示す情報には、例えば緯度経度、撮像方向および撮像角度が含まれる。また、カメラ情報テーブル３１はデータベースを記憶し、データベースは、監視カメラＣ１～Ｃｎからの映像データから検出される追跡対象者画像（検出画像）を登録する。 The camera information table 31 stores, for example, information representing the name, performance, and installation position of the surveillance camera in association with the identification information (hereinafter referred to as the camera ID) for each of the surveillance cameras C1 to Cn. Information representing performance includes, for example, resolution and aspect ratio. The information indicating the installation position includes, for example, latitude / longitude, imaging direction, and imaging angle. Further, the camera information table 31 stores a database, and the database registers the tracking target person image (detection image) detected from the video data from the surveillance cameras C1 to Cn.
　設定情報テーブル３２は、クエリの画像特徴量を記憶する。例えば、設定情報テーブル３２は、入出力Ｉ／Ｆ４を介して管理者端末ＯＴから入力されるクエリの画像特徴量を記憶する。或いは、設定情報テーブル３２は、通信Ｉ／Ｆ５を介して監視カメラＣ１～Ｃｎから送信される映像データから検出されるクエリの画像特徴量を記憶する。また、設定情報テーブル３２は、管理者端末ＯＴ等を介して入力されるアラート判定条件を記憶する。例えば、設定情報テーブル３２は、管理者端末ＯＴ等を介して入力される第１又は第２のアラート判定条件を記憶する。 The setting information table 32 stores the image feature amount of the query. For example, the setting information table 32 stores the image feature amount of the query input from the administrator terminal OT via the input / output I / F4. Alternatively, the setting information table 32 stores the image feature amount of the query detected from the video data transmitted from the surveillance cameras C1 to Cn via the communication I / F5. Further, the setting information table 32 stores the alert determination condition input via the administrator terminal OT or the like. For example, the setting information table 32 stores the first or second alert determination condition input via the administrator terminal OT or the like.
　ここで、クエリ画像の登録例について補足する。例えば、リアルタイムで得られるアラートに基づき、管理者が追跡したい人（画像）について、管理者端末ＯＴ上で追跡ボタンを押下する。制御部１は、この追跡ボタンの押下に対応して、自動的に最新検知画像のセット（顔画像と全身画像）をクエリ画像（クエリの画像特徴量）として登録し、追跡を開始する。また、リアルタイムで得られるアラートに基づき、管理者が追跡したい人（画像）について、管理者端末ＯＴ上で履歴ボタンを押下する。制御部１は、この履歴ボタンの押下に対応して、履歴一覧から任意の画像を選択してクエリ画像として登録し、追跡を開始する。また、制御部１は、管理者からの履歴検索に従い、監視カメラの画像から人物検索を実施し、管理者により人物検索結果の中から選択される画像をクエリ画像として登録し、追跡を開始する。また、管理者は、リアルタイムで得られる監視画像データに含まれる人（画像）を選択し、制御部１は、この選択された人をクエリ画像として登録し、追跡を開始する。また、管理者は、依頼者から提供された画像を管理者端末ＯＴから取り込み、クエリ画像として登録し、追跡を開始する。 Here, supplement the registration example of the query image. For example, based on the alert obtained in real time, the administrator presses the tracking button on the administrator terminal OT for the person (image) to be tracked. The control unit 1 automatically registers the latest set of detected images (face image and whole body image) as a query image (query image feature amount) in response to pressing the tracking button, and starts tracking. Also, based on the alert obtained in real time, the administrator presses the history button on the administrator terminal OT for the person (image) that the administrator wants to track. In response to pressing the history button, the control unit 1 selects an arbitrary image from the history list, registers it as a query image, and starts tracking. Further, the control unit 1 performs a person search from the image of the surveillance camera according to the history search from the administrator, registers the image selected from the person search results by the administrator as a query image, and starts tracking. .. Further, the administrator selects a person (image) included in the surveillance image data obtained in real time, and the control unit 1 registers the selected person as a query image and starts tracking. In addition, the administrator takes in the image provided by the requester from the administrator terminal OT, registers it as a query image, and starts tracking.
　制御部１は、この発明の一実施形態に係る処理機能として、情報取得部１１と、画像検出部１２と、検索条件設定部１３と、画像検索部１４と、画像選択部（フィルタ部）１５と、出力部１６とを備えている。各部は、何れもプログラム記憶部２に格納されたプログラムを制御部１のハードウェアプロセッサに実行させることにより実現される。 The control unit 1 has an information acquisition unit 11, an image detection unit 12, a search condition setting unit 13, an image search unit 14, and an image selection unit (filter unit) 15 as processing functions according to an embodiment of the present invention. And an output unit 16. Each unit is realized by causing the hardware processor of the control unit 1 to execute the program stored in the program storage unit 2.
　情報取得部１１は、監視カメラＣ１～Ｃｎに接続された映像解析エンジンＶＥ１～ＶＥｎ、又はＷｅｂサーバ装置ＳＶに設けられた映像解析エンジンＶＥ１～ＶＥｎからの映像データ及び映像解析結果等を取得する。例えば、映像解析エンジンＶＥ１～ＶＥｎはそれぞれ、対応する監視カメラＣ１～Ｃｎから出力される映像データに含まれる複数の画像フレームから、画像フレーム内の位置情報等に基づき同一人物を判定し、判定結果等を含む映像解析結果を出力する。 The information acquisition unit 11 acquires video data and video analysis results from the video analysis engines VE1 to VEn connected to the surveillance cameras C1 to Cn or the video analysis engines VE1 to VEn provided in the Web server device SV. For example, the video analysis engines VE1 to VEn each determine the same person from a plurality of image frames included in the video data output from the corresponding surveillance cameras C1 to Cn based on the position information in the image frame, and the determination result is obtained. Output the video analysis result including the above.
　また、情報取得部１１は、監視カメラＣ１～Ｃｎからの映像データに限らず、入出力Ｉ／Ｆ４又は通信Ｉ／Ｆ５を介して入力される動画ファイルを取得することもできる。情報取得部１１は、指定フォーマットのファイル名の動画ファイルを指定フォルダに転送する。例えば、情報取得部１１は、動画ファイルのファイル名に含まれるカメラ名及び撮影時刻を登録する。これにより、カメラ名又は撮影時刻の指定による動画ファイルの検索が可能となる。動画ファイルに基づく検索では、リアルタイム追跡処理を必要とせずに、画像検索が可能である。リアルタイム追跡処理を必要としないため、高速に動画ファイルの取り込みが可能となる。動画ファイルに基づく画像検索処理は、リアルタイム追跡処理を必要としない点以外は、監視カメラＣ１～Ｃｎからの映像データに基づく画像検索処理と実質同一である。 Further, the information acquisition unit 11 can acquire not only the video data from the surveillance cameras C1 to Cn but also the moving image file input via the input / output I / F4 or the communication I / F5. The information acquisition unit 11 transfers a moving image file having a file name in the specified format to the specified folder. For example, the information acquisition unit 11 registers the camera name and the shooting time included in the file name of the moving image file. This makes it possible to search for moving image files by specifying the camera name or shooting time. Searching based on video files allows image search without the need for real-time tracking. Since it does not require real-time tracking processing, it is possible to import video files at high speed. The image search process based on the moving image file is substantially the same as the image search process based on the video data from the surveillance cameras C1 to Cn, except that the real-time tracking process is not required.
　画像検出部１２は、映像解析結果及び監視カメラＣ１～Ｃｎからの映像データを統合的に解析し追跡対象者画像を検出する。映像解析エンジンＶＥ１～ＶＥｎは、例えば事前に与えられたクエリの画像特徴量（追跡対象者画像の特徴量）をもとに、監視カメラＣ１～Ｃｎからの映像データに含まれる複数の画像フレームから、クエリの画像特徴量と類似する画像特徴量を有する人物画像（追跡対象者画像）を抽出する。例えば、事前に与えられるクエリは複数であり、複数のクエリの画像特徴量と類似する画像特徴量を有する人物画像が複数抽出される。 The image detection unit 12 comprehensively analyzes the video analysis result and the video data from the surveillance cameras C1 to Cn to detect the tracked person image. The video analysis engines VE1 to VEn are, for example, from a plurality of image frames included in the video data from the surveillance cameras C1 to Cn based on the image feature amount (feature amount of the tracked person image) of the query given in advance. , A person image (tracked person image) having an image feature amount similar to the image feature amount of the query is extracted. For example, a plurality of queries are given in advance, and a plurality of person images having an image feature amount similar to the image feature amounts of the plurality of queries are extracted.
　また、映像解析エンジンＶＥ１～ＶＥｎは、抽出された人物画像とクエリ画像との類似度を表す情報と、監視カメラＣ１～ＣｎのカメラＩＤと、画角内追跡ＩＤと、撮影時刻（日時分秒）とを含む映像解析結果を生成する。人物画像には、顔（face）の画像と、全身（body）の画像が含まれ、類似度情報には顔画像および全身画像それぞれに対応する類似度が含まれる。カメラＩＤは、監視カメラに固有の識別情報である。画角内追跡ＩＤは、同一の監視カメラ内で同一人物とみなす画像を追跡するためのＩＤである。 Further, the video analysis engines VE1 to VEn include information indicating the degree of similarity between the extracted person image and the query image, the camera IDs of the surveillance cameras C1 to Cn, the tracking ID within the angle of view, and the shooting time (date, time, minute, and second). ) And the video analysis result including. The portrait image includes a face image and a whole body image, and the similarity information includes the similarity corresponding to each of the face image and the whole body image. The camera ID is identification information unique to the surveillance camera. The in-angle tracking ID is an ID for tracking images regarded as the same person in the same surveillance camera.
　画像検出部１２は、検出された追跡対象者画像（検出画像）を含む検出画像情報をカメラ情報テーブル３１のデータベースへ登録する。例えば、画像検出部１２は、監視カメラＣ１～Ｃｎからのそれぞれの映像データに含まれる複数のフレームから、膨大な数の追跡対象者画像を検出し、膨大な数の検出画像情報をカメラ情報テーブル３１のデータベースへ登録する。検出画像情報は、検出画像及び属性情報を含む。属性情報は、カメラＩＤ、画角内追跡ＩＤ、人物らしさのスコア、画像種別（顔画像又は全身画像）、撮影日時等を含む。なお、カメラ情報テーブル３１のデータベースには、検索対象として追跡対象者画像が登録されるだけでなく、様々な人物画像が登録される。 The image detection unit 12 registers the detected image information including the detected tracked person image (detected image) in the database of the camera information table 31. For example, the image detection unit 12 detects a huge number of tracked person images from a plurality of frames included in each video data from the surveillance cameras C1 to Cn, and stores a huge number of detected image information in the camera information table. Register in 31 databases. The detected image information includes the detected image and the attribute information. The attribute information includes a camera ID, a tracking ID within the angle of view, a score of personality, an image type (face image or whole body image), a shooting date and time, and the like. In the database of the camera information table 31, not only the tracking target person image is registered as a search target, but also various person images are registered.
　検索条件設定部１３は、設定情報テーブル３２から読み出される検索条件を設定する。検索条件は、期間（開始日時及び終了日時）を含む。また、検索条件は、カメラＩＤを含んでもよいし、検索対象画像（顔画像、全身画像、又は顔及び全身画像）を含んでもよい。 The search condition setting unit 13 sets the search conditions to be read from the setting information table 32. The search condition includes a period (start date and time and end date and time). Further, the search condition may include a camera ID, or may include a search target image (face image, whole body image, or face and whole body image).
　画像検索部１４は、監視カメラＣ１～Ｃｎからの映像データに含まれる複数のフレームから検出された検出画像を含む検出画像情報が登録されたデータベースから、検索条件に応じた対象者画像を検索する。例えば、画像検索部１４は、指定された期間において検出された検出画像を対象者画像（複数の人物の画像）として検索する。検索結果には、各人物の画像が複数含まれる。或いは、画像検索部１４は、指定された期間において検出された検出画像のうち、ある人物の画像と類似するものを対象者画像（ある人物の画像）として検索する。検索結果には、ある人物の画像が複数含まれる。 The image search unit 14 searches for the target person image according to the search condition from the database in which the detected image information including the detected images detected from the plurality of frames included in the video data from the surveillance cameras C1 to Cn is registered. .. For example, the image search unit 14 searches for the detected image detected in the designated period as a target person image (image of a plurality of people). The search result includes a plurality of images of each person. Alternatively, the image search unit 14 searches for an image similar to the image of a certain person among the detected images detected in the designated period as a target person image (image of a certain person). The search results include multiple images of a person.
　画像選択部１５は、設定情報テーブル３２の第１の出力指定（例えばフィルタ機能ＯＦＦ）に基づき画像検索部１４により検索された全ての対象者画像を選択する。また、画像選択部１５は、設定情報テーブル３２の第２の出力指定（例えばフィルタ機能ＯＮ）に基づき画像検索部１４により検索された全ての対象者画像のうち出力条件を満たし且つ同一人物と見なされる第１の所定数の対象者画像を選択する。第１の所定数は、任意に設定される１又は２以上の数である。例えば、同一人物とみなされる対象者画像を絞り込んで表示したい場合は、第１の所定数に１又はそれに近い数を設定し、同一人物とみなされる対象者画像を比較的多く表示したい場合は、第１の所定数に３以上の数を設定する。 The image selection unit 15 selects all the target person images searched by the image search unit 14 based on the first output designation (for example, the filter function OFF) of the setting information table 32. Further, the image selection unit 15 satisfies the output conditions among all the target person images searched by the image search unit 14 based on the second output designation (for example, the filter function is ON) of the setting information table 32, and is regarded as the same person. A first predetermined number of subject images are selected. The first predetermined number is one or two or more arbitrarily set. For example, if you want to narrow down and display the target person images that are regarded as the same person, set 1 or a number close to the first predetermined number, and if you want to display a relatively large number of target person images that are regarded as the same person, A number of 3 or more is set as the first predetermined number.
　出力部１５は、画像選択部１５により選択される対象者画像を出力する。例えば、出力部１６は、モニタ装置ＭＴで表示するための対象者画像を出力する。設定情報テーブル３２で第１の出力指定が設定されている場合には、画像検索部１４により検索された全ての対象者画像がモニタ装置ＭＴで表示され、設定情報テーブル３２で第２の出力指定が設定されている場合には、画像検索部１４により検索された全ての対象者画像のうち出力条件を満たし且つ同一人物と見なされる第１の所定数の対象者画像がモニタ装置ＭＴで表示される。 The output unit 15 outputs the target person image selected by the image selection unit 15. For example, the output unit 16 outputs a target person image to be displayed on the monitor device MT. When the first output designation is set in the setting information table 32, all the target person images searched by the image search unit 14 are displayed on the monitor device MT, and the second output designation is made in the setting information table 32. When is set, a first predetermined number of target person images that satisfy the output conditions and are regarded as the same person among all the target person images searched by the image search unit 14 are displayed on the monitor device MT. To.
　なお、以上の説明では、データ記憶部３に設けられた各テーブル３１、３２をＷｅｂサーバ装置ＳＶ内に設けた場合を例にとった。しかし、それに限らず、Ｗｅｂサーバ装置ＳＶ外に配置されたデータベースサーバまたはファイルサーバ内に設けるようにしてもよい。この場合、Ｗｅｂサーバ装置ＳＶがデータベースサーバまたはファイルサーバ内の各テーブル３１、３２に対しアクセスし、必要な情報を取得することにより各処理を行う。 In the above description, the case where the tables 31 and 32 provided in the data storage unit 3 are provided in the Web server device SV is taken as an example. However, the present invention is not limited to this, and it may be provided in a database server or a file server arranged outside the Web server device SV. In this case, the Web server device SV accesses the tables 31 and 32 in the database server or the file server, and performs each process by acquiring necessary information.
　（動作例）
　次に、以上のように構成されたシステムの動作例を説明する。　
　図４は、この発明の一実施形態に係るシステムによる検索処理の一例を示すフローチャートである。　
　監視カメラＣ１～Ｃｎは、撮影を開始し、映像データを出力する（ＳＴ１）。映像解析エンジンＶＥ１～ＶＥｎはそれぞれ、対応する監視カメラＣ１～Ｃｎからの映像データを解析する（ＳＴ２）。例えば、映像解析エンジンＶＥ１～ＶＥｎはそれぞれ、対応する監視カメラＣ１～Ｃｎから出力される映像データに含まれる複数の画像フレームに対して、画角内追跡を実施し、複数の画像フレームから、画像フレーム内の位置情報等に基づき同一人物を判定する。映像解析エンジンＶＥ１～ＶＥｎは、映像データ及び同一人物判定を出力する。 (Operation example)
Next, an operation example of the system configured as described above will be described.
FIG. 4 is a flowchart showing an example of a search process by the system according to the embodiment of the present invention.
Surveillance cameras C1 to Cn start shooting and output video data (ST1). The video analysis engines VE1 to VEN analyze video data from the corresponding surveillance cameras C1 to Cn, respectively (ST2). For example, the video analysis engines VE1 to VEn each perform intra-angle tracking on a plurality of image frames included in the video data output from the corresponding surveillance cameras C1 to Cn, and the images are imaged from the plurality of image frames. The same person is determined based on the position information in the frame. The video analysis engines VE1 to VEn output video data and the same person determination.
　Ｗｅｂサーバ装置ＳＶの通信Ｉ／Ｆ５は、映像解析エンジンＶＥ１～ＶＥｎから映像データ及び同一人物判定を受信する。情報取得部１１は、映像解析エンジンＶＥ１～ＶＥｎからの映像データ及び同一人物判定を取得する（ＳＴ３）。画像検出部１２は、映像解析エンジンＶＥ１～ＶＥｎからの映像データと同一人物判定とを統合的に解析し、映像解析エンジンＶＥ１～ＶＥｎからの映像データに含まれる複数のフレームから人物画像を検出する（ＳＴ４）。例えば、フレームの数は膨大であり、検出される人物画像も膨大になる。画像検出部１２は、検出された人物画像を含む検出画像情報をカメラ情報テーブル３１のデータベースへ登録する（ＳＴ５）。 The communication I / F5 of the Web server device SV receives the video data and the same person determination from the video analysis engines VE1 to VEn. The information acquisition unit 11 acquires video data from the video analysis engines VE1 to VEn and determination of the same person (ST3). The image detection unit 12 comprehensively analyzes the video data from the video analysis engines VE1 to VEn and the same person determination, and detects the person image from a plurality of frames included in the video data from the video analysis engines VE1 to VEn. (ST4). For example, the number of frames is enormous, and the number of detected human images is enormous. The image detection unit 12 registers the detected image information including the detected person image in the database of the camera information table 31 (ST5).
　検索条件設定部１３は、管理者端末ＯＴからの指示に従い設定情報テーブル３２から検索条件を読み出し、画像検索部１４に対して検索条件を設定する。例えば、検索条件は、開始日時Ｄ１及び終了日時Ｄ２を含む。また、検索条件設定部１３は、管理者端末ＯＴからの指示に従い設定情報テーブル３２から第１又は第２の出力指定を読み出し、画像選択部１５に対して第１又は第２の出力指定を設定する。 The search condition setting unit 13 reads the search condition from the setting information table 32 according to the instruction from the administrator terminal OT, and sets the search condition in the image search unit 14. For example, the search condition includes a start date and time D1 and an end date and time D2. Further, the search condition setting unit 13 reads the first or second output designation from the setting information table 32 according to the instruction from the administrator terminal OT, and sets the first or second output designation to the image selection unit 15. do.
　画像検索部１４は、カメラ情報テーブル３１に登録されたデータベースから、設定された検索条件に応じた対象者画像を検索する（ＳＴ６）。例えば、画像検索部１４は、データベースに登録された検出画像情報に含まれる属性情報を参照し、開始日時Ｄ１及び終了日時Ｄ２の期間に含まれる対象者画像を検索する。 The image search unit 14 searches the database registered in the camera information table 31 for the target person image according to the set search condition (ST6). For example, the image search unit 14 refers to the attribute information included in the detected image information registered in the database, and searches for the target person image included in the period of the start date / time D1 and the end date / time D2.
　画像選択部１５は、第１又は第２の出力指定に基づき画像検索部１４により検索された全部又は一部の対象者画像を選択する（ＳＴ７）。出力部１６は、入出力Ｉ／Ｆ４等を介して、モニタ装置ＭＴに対して、画像選択部１５により選択される対象者画像を出力する（ＳＴ８）。 The image selection unit 15 selects all or part of the target person images searched by the image search unit 14 based on the first or second output designation (ST7). The output unit 16 outputs the target person image selected by the image selection unit 15 to the monitoring device MT via the input / output I / F4 or the like (ST8).
　図５は、この発明の一実施形態に係るシステムによる検索処理のうちの対象画像の選択処理の一例を示すフローチャートである。図５は、図４に示すＳＴ７の詳細を説明するフローチャートである。 FIG. 5 is a flowchart showing an example of a target image selection process in the search process by the system according to the embodiment of the present invention. FIG. 5 is a flowchart illustrating the details of ST7 shown in FIG.
　第１の出力指定が設定されている場合（フィルタ機能ＯＦＦの場合）（ＳＴ７０１、ＹＥＳ）、画像選択部１５は、第１の出力指定に基づき画像検索部１４により検索された全ての対象者画像を選択する（ＳＴ７０２）。 When the first output designation is set (when the filter function is OFF) (ST701, YES), the image selection unit 15 is the all target image searched by the image search unit 14 based on the first output designation. Is selected (ST702).
　また、第２の出力指定が設定されている場合（フィルタ機能ＯＮの場合）（ＳＴ７０１、ＮＯ）（ＳＴ７０３）、画像選択部１５は、第２の出力指定に基づき画像検索部１４により検索された全ての対象者画像のうち出力条件を満たし且つ同一人物と見なされる第１の所定数の対象者画像を選択する（ＳＴ７０４）。 Further, when the second output designation is set (when the filter function is ON) (ST701, NO) (ST703), the image selection unit 15 is searched by the image search unit 14 based on the second output designation. A first predetermined number of subject images that satisfy the output conditions and are considered to be the same person are selected from all the subject images (ST704).
　例えば、管理者端末ＯＴ等を介して入力された第１の所定数が「１」であり、出力条件が「人物らしさのスコア」であれば、画像選択部１５は、第２の出力指定に基づき出力条件を満たし且つ同一人物と見なされる１つの対象者画像を選択し、出力部１６は、選択された１つの対象者画像を出力する。この場合、最も人物らしさのスコアが高い対象者画像が出力条件を満たす。 For example, if the first predetermined number input via the administrator terminal OT or the like is "1" and the output condition is "personality score", the image selection unit 15 determines the second output designation. Based on this, one target person image that satisfies the output conditions and is regarded as the same person is selected, and the output unit 16 outputs one selected target person image. In this case, the subject image having the highest score of humanity satisfies the output condition.
　管理者端末ＯＴ等を介して入力された第１の所定数が「２」であり、出力条件が「人物らしさのスコア」及び「顔画像」であれば、画像選択部１５は、第２の出力指定に基づき出力条件を満たし且つ同一人物と見なされる２つの対象者画像を選択し、出力部１６は、選択された２つの対象者画像を出力する。この場合、顔画像のうち、人物らしさのスコアが高い上位２つの対象者画像が出力条件を満たす。 If the first predetermined number input via the administrator terminal OT or the like is "2" and the output conditions are "personality score" and "face image", the image selection unit 15 is the second. Based on the output designation, two target person images that satisfy the output conditions and are regarded as the same person are selected, and the output unit 16 outputs the two selected target person images. In this case, among the facial images, the top two subject images having a high score of humanity satisfy the output condition.
　出力部１６は、第１の出力指定に基づき画像検索部１４により検索された全ての対象者画像が選択された場合には、選択された全ての対象者画像をモニタ装置ＭＴに出力する。また、出力部１６は、第２の出力指定に基づき画像検索部１４により検索された全ての対象者画像のうち出力条件を満たし且つ同一人物と見なされる第１の所定数の対象者画像が選択された場合には、選択された第１の所定数の対象者画像をモニタ装置ＭＴに出力する。 When all the target person images searched by the image search unit 14 are selected based on the first output designation, the output unit 16 outputs all the selected target person images to the monitoring device MT. Further, the output unit 16 selects a first predetermined number of target person images that satisfy the output conditions and are regarded as the same person among all the target person images searched by the image search unit 14 based on the second output designation. If so, the selected first predetermined number of target images are output to the monitoring device MT.
　或いは、画像選択部１５は、第２の出力指定に基づき、１台のカメラからの映像データに対して同一人物とみなされる第１の所定数又は第１の所定数を上限として対象者画像を選択するようにしてもよい。この場合、出力部１６は、カメラ毎に、同一人物とみなされる第１の所定数又は第１の所定数を上限とした対象者画像を出力し、モニタ装置ＭＴが、出力される対象者画像を表示する。 Alternatively, the image selection unit 15 sets the target person image up to a first predetermined number or a first predetermined number considered to be the same person for the video data from one camera based on the second output designation. You may choose. In this case, the output unit 16 outputs the target person image up to the first predetermined number or the first predetermined number regarded as the same person for each camera, and the monitoring device MT outputs the target person image. Is displayed.
　或いは、画像選択部１５は、第２の出力指定に基づき、１台のカメラからの映像データに含まれる、連続する所定時間分の複数のフレームに対して同一人物とみなされる第１の所定数又は第１の所定数を上限とする対象者画像を選択するようにしてもよい。この場合、出力部１６は、同一人物とみなされる第１の所定数又は第１の所定数を上限とした対象者画像を出力し、モニタ装置ＭＴが、出力される対象者画像を表示する。所定時間毎に第１の所定数の同一人物の表示が可能となり、視認性に優れるとともに詳細確認も可能となる。 Alternatively, based on the second output designation, the image selection unit 15 has a first predetermined number that is regarded as the same person for a plurality of frames for a continuous predetermined time included in the video data from one camera. Alternatively, a target person image whose upper limit is the first predetermined number may be selected. In this case, the output unit 16 outputs the target person image up to the first predetermined number or the first predetermined number regarded as the same person, and the monitoring device MT displays the output target person image. It is possible to display a first predetermined number of the same person at predetermined time intervals, which is excellent in visibility and enables detailed confirmation.
　図６は、この発明の一実施形態に係るシステムによる検索処理のうちの画角内追跡ＩＤ毎の対象画像の選択処理の一例を示すフローチャートである。図６は、図４に示すＳＴ７の詳細を説明するフローチャートである。 FIG. 6 is a flowchart showing an example of a target image selection process for each tracking ID within the angle of view in the search process by the system according to the embodiment of the present invention. FIG. 6 is a flowchart illustrating the details of ST7 shown in FIG.
　画像選択部１５は、画像検索部１４による検索結果を受信し（ＳＴ７１１）、検索結果ファイルを読み込む（ＳＴ７１２）。例えば、検索結果ファイルは、対象者画像情報を含み、対象者画像情報は、対象者画像及び属性情報を含む。属性情報は、カメラＩＤ、画角内追跡ＩＤ、人物らしさのスコア、画像種別（顔画像又は全身画像）、撮影日時等を含む。 The image selection unit 15 receives the search result from the image search unit 14 (ST711) and reads the search result file (ST712). For example, the search result file includes the target person image information, and the target person image information includes the target person image and the attribute information. The attribute information includes a camera ID, a tracking ID within the angle of view, a score of personality, an image type (face image or whole body image), a shooting date and time, and the like.
　画像選択部１５は、画角内追跡ＩＤ別に、対象者画像を並び替える（ＳＴ７１３）。例えば、画像選択部１５は、第１優先度を日時、第２優先度を人物らしさのスコアとして、対象者画像を並び替える。顔画像が指定されている場合又は全身画像が指定されている場合は、画像選択部１５は、類似度の高い順に並び替える。顔画像と全身画像の両方が指定されている場合は、画像選択部１５は、顔画像類似度が閾値を超える画像については、顔画像類似度の高い順に画像を並び替え、顔画像類似度が閾値を超えない画像については、全身画像類似度の高い順に画像を並び替える。 The image selection unit 15 sorts the target person images according to the tracking ID within the angle of view (ST713). For example, the image selection unit 15 rearranges the target person images with the first priority as the date and time and the second priority as the score of person-likeness. When a face image is specified or a whole body image is specified, the image selection unit 15 sorts the images in descending order of similarity. When both the face image and the whole body image are specified, the image selection unit 15 sorts the images in descending order of the face image similarity for the images whose face image similarity exceeds the threshold, and the face image similarity is high. For images that do not exceed the threshold, the images are sorted in descending order of similarity to the whole body image.
　画角内追跡ＩＤフィルタフラグがTrueでなければ（フィルタ機能ＯＦＦ）（ＳＴ７１４、ＮＯ）、画像選択部１５は、検索された全ての対象者画像を選択する。画角内追跡ＩＤフィルタフラグがTrueであれば（フィルタ機能ＯＮ）（ＳＴ７１４、ＹＥＳ）、画像選択部１５は、検索された全ての対象者画像のうちの一部の対象者画像を選択する（ＳＴ７１５～ＳＴ７２０）。 If the tracking ID filter flag in the angle of view is not True (filter function OFF) (ST714, NO), the image selection unit 15 selects all the searched target image. If the tracking ID filter flag in the angle of view is True (filter function ON) (ST714, YES), the image selection unit 15 selects a part of the target person images among all the searched target person images (ST714, YES). ST715 to ST720).
　画像選択部１５は、各画角内追跡ＩＤを順に調べる。画像選択部１５は、Ｎ＝１を設定し（ＳＴ７１５）、Ｎ番目の画角内追跡ＩＤの対象者画像を取得する（ＳＴ７１６）。表示リストに同一画角内追跡ＩＤの対象者画像が存在しなければ（ＳＴ７１７、ＮＯ）、画像選択部１５は、取得した対象者画像を表示リストに追加する（ＳＴ７１８）。最後の画角内追跡ＩＤまで確認していなければ（ＳＴ７１９、ＮＯ）、画像選択部１５は、Ｎ＝Ｎ＋１を設定し（ＳＴ７２０）、ＳＴ７１６～ＳＴ７１９を実行する。 The image selection unit 15 examines the tracking ID in each angle of view in order. The image selection unit 15 sets N = 1 (ST715) and acquires the target person image of the N-th angle of view tracking ID (ST716). If the target person image having the same angle of view tracking ID does not exist in the display list (ST717, NO), the image selection unit 15 adds the acquired target person image to the display list (ST718). If the tracking ID within the last angle of view has not been confirmed (ST719, NO), the image selection unit 15 sets N = N + 1 (ST720) and executes ST716 to ST719.
　その結果、表示リストには、画角内追跡ＩＤ毎に対象者画像が登録される。出力部１６は、表示リストを出力し、モニタ装置ＭＴは、表示リストに基づき対象者画像を表示する。 As a result, the target person image is registered in the display list for each tracking ID within the angle of view. The output unit 16 outputs a display list, and the monitoring device MT displays a target person image based on the display list.
　本実施形態によれば、検出された同一人物の画像の視認性の向上を図るシステム、装置、方法、プログラムを提供することができる。一つのカメラで同一人物が長時間撮影された場合にはこのカメラからの映像データの複数のフレームには同一人物の画像が含まれる。また、複数のカメラで同一人物が撮影された場合にはこれら複数のカメラからの映像データの複数のフレームには同一人物の画像が含まれる。Ｗｅｂサーバ装置ＳＶは、第１の出力指定の設定により、同一人物の多数の画像を表示し、第２の出力指定の設定により、同一人物の絞り込まれた少数の画像を表示する。仮に、第２の出力指定の設定で表示される画像が目的に合致しない画像であれば、第１の出力指定の設定に変更すればよい。例えば、第１の出力指定を利用すれば、多くの画像から人物を詳細に確認することができ、第２の出力指定を利用すれば、少ない画像から人物を短時間で確認することができる。さらに、第２の出力指定を利用すれば、表示処理にかかる時間も短縮できる。 According to the present embodiment, it is possible to provide a system, an apparatus, a method, and a program for improving the visibility of a detected image of the same person. When the same person is photographed for a long time with one camera, the images of the same person are included in a plurality of frames of the video data from this camera. Further, when the same person is photographed by a plurality of cameras, the images of the same person are included in the plurality of frames of the video data from the plurality of cameras. The Web server device SV displays a large number of images of the same person by the setting of the first output designation, and displays a small number of narrowed-down images of the same person by the setting of the second output designation. If the image displayed by the second output designation setting is an image that does not match the purpose, it may be changed to the first output designation setting. For example, if the first output designation is used, a person can be confirmed in detail from many images, and if the second output designation is used, a person can be confirmed from a small number of images in a short time. Further, if the second output designation is used, the time required for the display process can be shortened.
　本実施形態に係るプログラムは、電子機器に記憶された状態で譲渡されてよいし、記憶媒体に記憶された状態で譲渡されてもよいし、ネットワーク等を介したダウンロードにより譲渡されてもよい。記録媒体は、磁気ディスク、光ディスク、又はフラッシュメモリ等の非一時的なコンピュータ可読記憶媒体である。　
　なお、本発明は、上記実施形態に限定されるものではなく、実施段階ではその要旨を逸脱しない範囲で種々に変形することが可能である。また、各実施形態は適宜組み合わせて実施してもよく、その場合組み合わせた効果が得られる。更に、上記実施形態には種々の発明が含まれており、開示される複数の構成要件から選択された組み合わせにより種々の発明が抽出され得る。例えば、実施形態に示される全構成要件からいくつかの構成要件が削除されても、課題が解決でき、効果が得られる場合には、この構成要件が削除された構成が発明として抽出され得る。 The program according to the present embodiment may be transferred in a state of being stored in an electronic device, may be transferred in a state of being stored in a storage medium, or may be transferred by downloading via a network or the like. The recording medium is a non-temporary computer-readable storage medium such as a magnetic disk, an optical disk, or a flash memory.
The present invention is not limited to the above embodiment, and can be variously modified at the implementation stage without departing from the gist thereof. In addition, each embodiment may be carried out in combination as appropriate, in which case the combined effect can be obtained. Further, the above-described embodiment includes various inventions, and various inventions can be extracted by a combination selected from a plurality of disclosed constituent requirements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, if the problem can be solved and the effect is obtained, the configuration in which the constituent elements are deleted can be extracted as an invention.
１…制御部
２…プログラム記憶部
３…データ記憶部
４…入出力インタフェース（入出力Ｉ／Ｆ）
５…通信インタフェース（通信Ｉ／Ｆ）
６…バス
１１…情報取得部
１２…画像検出部
１３…検索条件設定
１４…画像検索部
１５…画像選択部
１６…出力部
３１…カメラ情報テーブル
３２…設定情報テーブル
Ｃ１、Ｃ２、Ｃｎ…監視カメラ
ＭＴ…モニタ装置
ＮＷ…ネットワーク
ＯＴ…管理者端末
ＳＶ…サーバ装置
ＶＥ１、ＶＥｎ…映像解析エンジン　　
  1 ... Control unit 2 ... Program storage unit 3 ... Data storage unit 4 ... Input / output interface (input / output I / F)
5 ... Communication interface (communication I / F)
6 ... Bus 11 ... Information acquisition unit 12 ... Image detection unit 13 ... Search condition setting 14 ... Image search unit 15 ... Image selection unit 16 ... Output unit 31 ... Camera information table 32 ... Setting information tables C1, C2, Cn ... Surveillance camera MT ... Monitor device NW ... Network OT ... Administrator terminal SV ... Server device VE1, VEn ... Video analysis engine

Claims (7)

  1. 　１以上のカメラからの映像データに含まれる複数のフレームから検出された検出画像が登録されたデータベースから、検索条件に応じた対象者画像を検索する検索部と、
    　第１の出力指定に基づき前記検索部により検索された全ての対象者画像を選択し、第２の出力指定に基づき前記検索部により検索された全ての対象者画像のうち出力条件を満たし且つ同一人物と見なされる第１の所定数の対象者画像を選択する選択部と、
    　前記選択部により選択される対象者画像を出力する出力部と、
    　を備える情報検索装置。     A search unit that searches for target person images according to search conditions from a database in which detected images detected from multiple frames included in video data from one or more cameras are registered.
    All the target image searched by the search unit based on the first output designation is selected, and the output conditions are satisfied and the same among all the target image searched by the search unit based on the second output designation. A selection unit that selects a first predetermined number of target images that are considered to be people, and
    An output unit that outputs a target person image selected by the selection unit, and an output unit.
    An information retrieval device.
  2. 　前記検索部は、複数のカメラからの映像データに含まれる複数のフレームから検出された検出画像が登録されたデータベースから、検索条件に応じた対象者画像を検索する請求項１の情報検索装置。 The search unit is the information retrieval device according to claim 1, which searches for a target person image according to a search condition from a database in which detected images detected from a plurality of frames included in video data from a plurality of cameras are registered.
  3. 　前記選択部は、前記第２の出力指定に基づき、１台のカメラからの映像データに対して同一人物と見なされる第１の所定数の対象者画像を選択する、請求項２の情報検索装置。 The information retrieval device according to claim 2, wherein the selection unit selects a first predetermined number of target person images regarded as the same person for video data from one camera based on the second output designation. ..
  4. 　前記選択部は、前記第２の出力指定に基づき、１台のカメラからの連続する所定時間分の複数のフレームに対して同一人物と見なされる第１の所定数の対象者画像を選択する、請求項２の情報検索装置。 Based on the second output designation, the selection unit selects a first predetermined number of target image images considered to be the same person for a plurality of frames for a predetermined time in a row from one camera. The information retrieval device according to claim 2.
  5. 　前記選択部は、前記第２の出力指定に基づき人物らしさのスコアの高い順に上位から第１の所定数の対象者画像を選択する、請求項１の情報検索装置。 The information retrieval device according to claim 1, wherein the selection unit selects a predetermined number of target person images from the top in descending order of personality score based on the second output designation.
  6. 　１以上のカメラからの映像データに含まれる複数のフレームから検出された検出画像が登録されたデータベースから、対象者の検索条件に応じた対象者画像を検索し、
    　第１の出力指定に基づき、検索された全ての対象者画像を選択し、第２の出力指定に基づき、検索された全ての対象者画像のうち出力条件を満たす第１の所定数の同一人物と見なされる対象者画像を選択し、
    　選択される対象者画像を出力する情報検索方法。     From the database in which the detected images detected from a plurality of frames included in the video data from one or more cameras are registered, the target person image according to the target person's search condition is searched, and the target person image is searched.
    Based on the first output specification, all the searched target person images are selected, and based on the second output specification, the first predetermined number of the same person satisfying the output condition among all the searched target person images. Select the subject image that is considered to be
    An information retrieval method that outputs the selected target person image.
  7. 　請求項１乃至６の何れかの情報検索装置が備える各部による処理をプロセッサに実行させるプログラム。
          A program for causing a processor to execute processing by each part included in the information retrieval apparatus according to any one of claims 1 to 6.
PCT/JP2021/028964 2020-08-07 2021-08-04 Information retrieval device, information retrieval method, and program WO2022030549A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020135168A JP2022030865A (en) 2020-08-07 2020-08-07 Information retrieval device, information retrieval method, and program
JP2020-135168 2020-08-07

Publications (1)

Publication Number Publication Date
WO2022030549A1 true WO2022030549A1 (en) 2022-02-10

Family

ID=80117521

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/028964 WO2022030549A1 (en) 2020-08-07 2021-08-04 Information retrieval device, information retrieval method, and program

Country Status (2)

Country Link
JP (1) JP2022030865A (en)
WO (1) WO2022030549A1 (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018163398A1 (en) * 2017-03-10 2018-09-13 株式会社日立国際電気 Similar image search system

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2018163398A1 (en) * 2017-03-10 2018-09-13 株式会社日立国際電気 Similar image search system

Also Published As

Publication number Publication date
JP2022030865A (en) 2022-02-18

Similar Documents

Publication Publication Date Title
US11210504B2 (en) Emotion detection enabled video redaction
US9560323B2 (en) Method and system for metadata extraction from master-slave cameras tracking system
KR101417548B1 (en) Method and system for generating and labeling events in photo collections
CN111131902B (en) Method for determining target object information and video playing equipment
TWI747341B (en) File application method, device and storage medium
JP2022518459A (en) Information processing methods and devices, storage media
KR101777238B1 (en) Method and system for image trend detection and curation of image
JP2008165701A (en) Image processing device, electronics equipment, image processing method, and program
JP2022022298A (en) Investigation support system, investigation support method, and computer program
CN106844654A (en) Towards the massive video distributed search method of police service practical
JP2015228564A (en) Monitoring camera system
WO2022030549A1 (en) Information retrieval device, information retrieval method, and program
JP2017049733A (en) Case Evidence Management System and Case Evidence Management Method
JP2007213183A (en) Device, method, and program for classifying digital image data
US10599928B2 (en) Method and system for enabling information in augmented reality applications
JP5523122B2 (en) Image management apparatus, method and program
JP6341843B2 (en) Image search apparatus and image search system
JP7497853B2 (en) Face Detection System
JP7389955B2 (en) Information processing device, information processing method and program
JP7235612B2 (en) Person search system and person search method
US20110085696A1 (en) Image data management apparatus, method and program
WO2022030546A1 (en) Information processing device, information processing method, and program
JP2020078030A (en) System, information processing device, information processing method, and program
US20230396741A1 (en) Computer-implemented method, computer program and apparatus for video processing and for generating a thumbnail from a video sequence, and video surveillance system comprising such an apparatus
JP2019212068A (en) Information processing apparatus, information processing method, and program

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21852173

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21852173

Country of ref document: EP

Kind code of ref document: A1