JP2008171434A - Image retrieval device and image retrieval system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image retrieval device and an image retrieval system having a light load to an operator and capable of rapidly retrieving an imaged part of a photographed object whose position can be specified. <P>SOLUTION: Image data 1 is composed of image raw data 5 which is a time series set of image frames, and image attribute data 6 including imaging range data 10 showing an imaging range of each image frame. In an image retrieval section 3, an input means 7 accepts a retrieval request including position data of the photographed object and a position relationship between the photographed object and the imaging range, and a retrieval processing means 8 calculates a mutual position relationship from the position data and the imaging range data in the image storage section 2, and determines the image frame which meets the accepted retrieval request. An output means 9 outputs the retrieval result to a retrieval application executing section 4. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video search device that searches video data or a video data portion showing a subject such as a desired object or space from among a large number of video data stored in a recording device, using the subject position data as a key, The present invention relates to a video search system that acquires position data of a designated subject and sends a search request to the video search device. For example, in the fields of security monitoring, plant operation control, facility management, production of movies and broadcast programs, etc. It is suitable.

  In security monitoring and plant operation control, video monitoring is performed in which a situation where a building or facility is located is imaged by a plurality of monitoring cameras, and a necessary video is switched and displayed at a monitoring center remote from the site. FIG. 22 is a block diagram showing an example of the configuration of a general video surveillance system. The situation in the field is captured by a plurality of cameras including a pan head camera and a camera robot, and the video data is transmitted from a local controller to a network. The video is monitored at a monitoring station of a monitoring center remote from the site connected by the network. As a result, when the occurrence of a failure or the like is detected, the monitoring station instructs the equipment controller connected via the network as needed to control the failure recovery of the equipment on the site.

  In this monitoring center, the video to be displayed is searched for and displayed on a video tape recorder (hereinafter referred to as VTR) or a hard disk device, in addition to confirming the current situation with live video, Work such as confirming the situation before the incident occurs and repeatedly analyzing the phenomenon for detailed analysis. Since the amount of information in video data is enormous, VTRs were mainly used for its storage. Recently, with the progress and popularization of video compression / storage / transmission technology, video has been digitally compressed. Video surveillance devices that record on a hard disk device or the like have also been commercialized.

In this way, by digitizing and accumulating surveillance video, the accumulated video can be displayed without stopping recording, past video can be quickly read and displayed, and search and display in association with numerical data such as sensor data and operation data Since there is an effect such as being able to do so, in the video monitoring system shown in FIG. 22 and the like, it is certain that a large amount of video data will be stored temporarily or for a long time in the future. At that time, a video search apparatus having a function of searching for necessary video data at high speed when necessary from a large amount of video data captured by a large number of cameras is important.

  Here, FIG. 23 is a block diagram showing a configuration of a conventional video search apparatus disclosed in Patent Document 1, for example. In the figure, 1 is video data, 2 is a video storage unit, 3 is a video search unit, and 4 is a search application. Reference numeral 5 denotes raw video data of the video data 1, reference numeral 6 denotes the video attribute data thereof, reference numeral 7 denotes input means of the video search unit 3, reference numeral 8 denotes search processing means, and reference numeral 9 denotes output means.

Next, the operation will be described.
The video data 1 stored in the video storage unit 2 is, for example, an endless recording of the latest 10 minutes of video for each camera, or alarms from various sensors installed at the site. There is one that records a one-minute video. Each video data 1 includes video raw data 5 in which video frames as one image are temporally continuous, video attribute data describing an ID of a camera that has been imaged, an ID of an alarm that triggers recording, an imaging time, and the like. It is comprised from 6.

  In response to an operator's instruction, the search application 4 sends a search request to the video search unit 3 so as to search for the corresponding video data using the alarm ID or camera ID as a key. When the video search unit 3 accepts this search request by the input unit 7, the video processing unit 8 reads the video attribute data 6 from the video storage unit 2 and searches the video data 1 that matches the specified alarm ID or camera ID. To do. Next, the video search unit 3 answers the search application 4 from the output means 9 with the ID list of the matched video data 1 as a search result. Based on the search result, the search application 4 takes out the corresponding video raw data 5 from the video storage unit 2 and displays it automatically or in accordance with an instruction from the operator.

“Digital image recording apparatus for security monitoring”, Sanyo Technical Review Vol. 28, no. 2, pp. 50-57 0ct. 1996

  Since the conventional video search device is configured as described above, for example, a request to view only the video portion in which the subject is shown, such as the valve B of the device A or the X-chome and Y-addresses, is used. It is only possible to search for possible video data 1 from the ID of the camera in charge and the alarm ID associated with the corresponding part, and the operator himself looks through all the search results and searches for the corresponding video part. In particular, in the case of a camera with a pan head that can pan, tilt, and zoom, or a camera robot that moves the camera itself, only the subject determined by a fixed angle is not captured. There are cases where the number of parts is small, and there is a problem that wasteful work increases and the load on the operator increases.

  The present invention has been made in order to solve the above-described problems, and a video search apparatus capable of quickly searching a video data portion in which a subject whose position can be specified is shown from a large amount of stored video data, and a subject An object of the present invention is to obtain a video search system that acquires position data of a designated subject from a subsystem that manages the video and sends a search request to the video search device.

  The video search device according to the present invention searches for video data satisfying a search condition of a search request in response to a search request from a video storage unit that stores video data and the query source, and returns the result to the query source In the video search device including the video search unit, the video data stored in the video storage unit includes video raw data that is a time-series set of video frames and imaging range data indicating an imaging range of each video frame. An input means configured by video attribute data, wherein the video search unit accepts a search request including a condition relating to a direction in which the subject is imaged as position data of the subject and a positional relationship between the subject and the imaging range, position data, and imaging Calculates the direction in which the subject is imaged from the range data, and satisfies the search request including the conditions related to the specified imaging direction. And search processing means for determining, and has an output means for outputting the inquiry source search result of the search processing means.

  In the video search device according to the present invention, the input unit of the video search unit accepts a search request including a condition relating to the relationship of a plurality of subjects on the projection plane, and the search processing unit of the video search unit includes a plurality of subjects. The relationship between a plurality of subjects on the projection plane is calculated from the position data and the photographing range data, and a video frame satisfying a search request including a condition regarding the relationship on the designated projection plane is obtained.

  In the video search device according to the present invention, the video attribute data includes imaging range data indicating the imaging range of each video frame and imaging time data indicating the time when each video frame was captured. A search request including the positional relationship between the data and the subject and the imaging range, and an imaging time interval indicating the imaging time range is received by the input means, and the search processing means accepts each other from the position data and the imaging range data. By calculating the positional relationship, a video frame that satisfies the accepted search request is obtained.

  In the video search device according to the present invention, the video attribute data includes imaging time data for calculating the imaging time of each video frame together with imaging range data indicating the imaging range of each video frame. A search request including the positional data and the positional relationship between the subject and the imaging range, and the imaging time interval indicating the imaging time range is received by the input means, and the search processing means determines from the position data and the imaging range data. In addition to calculating the positional relationship with each other, the imaging time is calculated from the imaging time data, and a video frame satisfying the accepted search request is obtained.

  The video search device according to the present invention includes, for each of a plurality of temporally continuous video frame sets, first imaging range data indicating an imaging range of an entire video frame included in the set, and each video frame or video frame set. The imaging range data of the video attribute data is configured by the second imaging range data indicating the imaging range of the divided video frame sub-set, and in the video search unit, the search processing means first converts the first imaging range data And calculating a positional relationship with respect to the video frame set to obtain a video frame set in which a video frame satisfying the search request may exist, and then, for each video frame or video frame sub-set belonging to the video frame set, By calculating the positional relationship based on the second imaging range data, a video frame that satisfies the search request is obtained. .

  The video search apparatus according to the present invention includes, for each of a plurality of temporally continuous video frame sets, first imaging range data indicating an imaging range of the entire video frame included in the set, and each video frame or video frame set The imaging range data of the video attribute data is constituted by the third imaging range data for calculating the imaging range of the video frame sub-set obtained by dividing the video frame sub-set. By calculating the positional relationship with the video frame set based on the imaging range data, a video frame set in which a video frame satisfying the search request can be obtained is obtained, and then imaging of the video frame or video frame sub-set belonging to the video frame set The range is calculated based on the third imaging range data, the positional relationship is calculated, and a video frame that satisfies the search request is obtained. Those were Unishi.

  The video search device according to the present invention is the fourth imaging range indicating the imaging range of the entire video frame belonging to the video data captured by the camera for each camera in the first imaging range data and the second imaging range data. The imaging range data is configured by adding data, and in the video search unit, the search processing means first calculates the positional relationship with respect to the entire video frame belonging to each camera based on the fourth imaging range data, and issues a search request. A camera in which a satisfactory video frame can exist is obtained, and then a positional relationship with respect to a set of video frames captured by the camera is calculated based on the first imaging range data, and a video frame that satisfies the search request can exist. A video frame set is obtained, and for each video frame or video frame sub-set belonging to the video frame set, a second imaging range data is obtained. The is obtained so as to obtain an image frame that satisfies the search request to calculate the positional relationship based.

  The video search device according to the present invention provides a fourth imaging range indicating the imaging range of the entire video frame belonging to the video data captured by the camera for each camera in the first imaging range data and the third imaging range data. The imaging range data is configured by adding data, and in the video search unit, the search processing means first calculates the positional relationship with respect to the entire video frame belonging to each camera based on the fourth imaging range data, and issues a search request. A camera in which a satisfactory video frame can exist is obtained, and then a positional relationship with respect to a set of video frames captured by the camera is calculated based on the first imaging range data, and a video frame that satisfies the search request can exist. A video frame set is obtained, and an imaging range of a video frame or a video frame sub-set belonging to the video frame set is determined as a third imaging range set. Seeking based on data to calculate the positional relationship is obtained by to obtain the video frame that satisfies the search request.

  The video search system according to the present invention includes a video search device according to the above invention, a search application execution unit as an inquiry source for sending a search request to the video search device, and a subsystem that manages at least the position data of the subject. The search application execution unit acquires the position data of the designated subject from the subsystem and sends a search request including the acquired position data to the video search device. It is a thing.

  According to the present invention, in response to a search request from a query source, a video search unit that stores video data, searches for video data that satisfies a search request search condition, and returns the result to the query source. Image data including video raw data, which is a time-series set of video frames, and imaging range data indicating an imaging range of each video frame. An input means for receiving a search request including a condition relating to a direction in which the subject is imaged as a positional relationship between the subject position data and the subject and the imaging range, and the position data and the imaging range data. The direction in which the subject is imaged is calculated, and a search for obtaining a video frame that satisfies the search request including the condition relating to the specified direction in which the image is captured is performed. Since the processing means and the output means for outputting the search result of the search processing means to the inquiry source are provided, only the video data portion satisfying the positional relationship with the imaging range is automatically searched for the subject whose position can be specified. Therefore, the operator does not need to see the video portion that does not satisfy the positional relationship, so that the work can be performed in a short time and the video search device that can reduce the burden on the operator can be obtained. is there. In addition, for example, when it is desired to view only a video portion where a certain subject is reflected in the center of the projection surface, it becomes possible to perform a search with higher accuracy, and the operator views an unnecessary video portion from the search result. There is an effect that it becomes unnecessary.

  According to the present invention, the input means of the video search unit accepts a search request including a condition relating to the relationship of the plurality of subjects on the projection plane, and the search processing means of the video search unit receives the position data of the plurality of subjects. And the shooting range data are used to calculate the relationship between the projection planes of multiple subjects and to obtain a video frame that satisfies the search request including the conditions related to the specified projection plane. The search request can be easily expanded not only for the subject but also for a plurality of subjects including a condition that combines the positional data of each region and the relationship between the position and size of each other on the projection plane.

  According to the present invention, the video attribute data includes the imaging range data indicating the imaging range of each video frame and the imaging time data indicating the imaging time. In the video search unit, the input means includes the subject position data and the subject. A search request including the positional relationship between the imaging ranges and the imaging time interval indicating the imaging time range is received, and the search processing unit calculates the positional relationship between the position data and the imaging range data, and satisfies the received search request. Since it is configured to obtain a video frame, if you want to search only video of a certain subject in a certain time zone, it is possible to perform a more accurate search, and the operator views unnecessary video parts from the search results There is an effect that it becomes unnecessary.

  According to the present invention, the video attribute data includes imaging range data indicating the imaging range of each video frame, and imaging time data for calculating the imaging time. A search request including a positional relationship between the subject and the imaging range and an imaging time interval indicating a range of the imaging time, and the search processing means calculates the mutual positional relationship from the position data and the imaging range data, and from the imaging time data Since it is configured to calculate the imaging time and obtain a video frame that satisfies the received search request, it is possible to perform a more accurate search if you only want to search for video in a certain time zone of a subject. The operator does not need to see unnecessary video parts from the search results, and the memory for storing imaging time data The amount can be reduced, there is an effect such as to allow shortening of the search process time.

  According to the present invention, the imaging range data includes the first imaging range data indicating the imaging range of the entire video frame included in the set for each of a plurality of temporally continuous video frame sets, and the imaging range of each video frame. The video frame satisfying the search request by the search processing means first calculating the positional relationship with respect to the video frame set based on the first imaging range data in the video search unit. To obtain a video frame satisfying the search request by calculating a positional relationship based on the second imaging range data for each video frame belonging to the video frame set. Since it is configured, it is necessary to read the imaging range data, calculate the positional relationship, and determine all the video frames in all the stored video data. Is eliminated, there is an effect that can shorten the search process time.

  According to the present invention, the imaging range data includes the first imaging range data indicating the imaging range of the entire video frame included in the set for each of a plurality of temporally continuous video frame sets, and the imaging range of each video frame. The image search unit first calculates the positional relationship with respect to the video frame set based on the first image pickup range data to satisfy the search request. A video frame satisfying a search request by obtaining a video frame set in which a video frame can exist, calculating an imaging range of video frames belonging to the video frame set based on third imaging range data, and calculating a positional relationship Therefore, for all the video frames in all the video data, the imaging range data is read and the positional relationship is calculated and determined. It is not necessary to perform, it becomes possible to shorten the search processing time, and can be described with a small amount of data an imaging range data for each video frame, there are effects such as may reduce the capacity of the recording apparatus.

  According to the present invention, the imaging range data is converted into the first imaging range data and the second imaging range data, and the fourth imaging range indicating the imaging range of the entire video frame belonging to the video data captured by the camera for each camera. Video that is configured by adding imaging range data, and in the video search unit, the search processing means first calculates the positional relationship with respect to the entire video frame belonging to each camera based on the fourth imaging range data, and satisfies the search request A video that can satisfy a search request by calculating a positional relationship with respect to a set of video frames captured by the camera based on the first imaging range data after obtaining a camera that can have a frame. Find the frame set and calculate the positional relationship based on the second imaging range data for each video frame belonging to the video frame set to satisfy the search request Since it is configured to determine a video frame that, for accumulating all the image data, it is not necessary to perform the calculation and determination of the positional relationship, there is an effect of speeding up the Search process.

  According to this invention, the imaging range data is converted into the first imaging range data and the third imaging range data, and the fourth imaging range indicating the imaging range of the entire video frame belonging to the video data captured by the camera for each camera. Video that is configured by adding imaging range data, and in the video search unit, the search processing stage first calculates the positional relationship with respect to the entire video frame belonging to each camera based on the fourth imaging range data, and satisfies the search request A video that can satisfy a search request by calculating a positional relationship with respect to a set of video frames captured by the camera based on the first imaging range data after obtaining a camera that can have a frame. Obtain a frame set, calculate the image range of video frames belonging to the video frame set based on the third image range data, calculate the positional relationship, and request a search Since it is configured to obtain a satisfactory video frame, it is not necessary to calculate and determine the positional relationship for all accumulated video data, the search process can be speeded up, and the imaging range data of each video frame can be obtained. This is advantageous in that it can be described with a small amount of data and the capacity of the recording apparatus can be reduced.

  According to the present invention, the video data stored in the video storage unit is composed of video raw data that is a time-series set of video frames and video attribute data including imaging range data indicating the imaging range of each video frame. In the video search unit, the input unit accepts a search request including the position data of the subject and the positional relationship between the subject and the imaging range from the search application execution unit, and the search processing unit receives the location data and the imaging range data in the video storage unit. From each other, the video frame satisfying the search request including the received positional relationship is obtained, and the output means outputs the search result to the search application execution unit, which is designated by the operator or the like in the search application execution unit. For each subject, at least from the subsystem that manages that location data. Since the search request including the acquired position data is sent to the video search device, the operator does not need to search for the position data of the subject by himself / herself and input it with a keyboard or the like. By selecting an object visually from a table or a map, it is possible to search and display video data in which the object is reflected, and there is an effect that a video search system with good operability can be obtained.

An embodiment of the present invention will be described below.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing the configuration of a video search apparatus according to Embodiment 1 of the present invention. In the figure, 1 is video data searched by the video search device, and 2 is a video storage unit for storing the video data 1. 3 is a video search unit that searches the video storage unit 2 for video data 1 that satisfies the search condition in response to a search request from an inquiry source, and 4 returns a result. 4 is a video search unit according to an instruction from an operator or the like. 3 is a search application execution unit serving as an inquiry source.

  Further, 5 is video raw data by a time series set in which video frames as one image are temporally continuous, and 6 is the ID of the camera that captured the video raw data 5 and the alarm that triggered the recording. This is video attribute data describing attribute information of the video raw data 5 including an ID and the like. The video data 1 is composed of this raw video data 5 and video attribute data 6. Reference numeral 7 denotes an input means for receiving a search request from the search application execution unit 4 as an inquiry source, and reference numeral 8 denotes a search processing means for obtaining video data satisfying the search request from the video data 1 stored in the video storage unit 2. , 9 are output means for outputting the search result to the search application execution unit 4. The video search unit 3 has this input means 7, search processing means 8, and output means 9. These are the parts corresponding to those of the related art shown with the same reference numerals in FIG.

  Reference numeral 10 denotes imaging range data indicating the imaging range of each video frame. The camera position (X, Y, Z), the horizontal imaging angle (HL, HR), and the vertical imaging angle (VU, VD). The imaging range defined by the imaging distance (DN, DF) is stored for each video frame. The video attribute data 6 is different from the conventional one shown in FIG. 23 with the same reference numerals in that it includes the imaging range data 10 together with the camera ID and alarm ID.

Next, the operation will be described.
When it is desired to search for and display an image showing a desired subject, the operator gives subject position data (x, y, z) to the search application execution unit 4 to instruct search. Upon receiving this search instruction, the search application execution unit 4 sends a search request to the video search unit 3 using the position data, the positional relationship “include” between the subject and the imaging range as parameters. When receiving the search request from the search application execution unit 4, the input unit 7 of the video search unit 3 instructs the search processing unit 8 to execute the search process of the video data 1.

  Next, search processing by the search processing means 8 will be described. FIG. 2 is a flowchart showing the search processing procedure in the first embodiment, and the search processing means 8 searches for a video frame satisfying the search request from the search application execution unit 4 according to the procedure shown in FIG. Register to the search result list. That is, first, in step ST101, the video data counter i is initially set to "1", and in step ST102, it is checked whether or not all of # 1 to #M video data 1 have been examined. In this case, since it has not been completed yet, the process branches to step ST103, the video frame counter j is initialized to “1”, and the video data 1 is set to #i (# 1 in this case) in step ST104. It is checked whether all the included video frames # 1 to #Fi have been examined. In this case, since it has not been completed yet, the process branches to step ST105, and the imaging range data 10 of #j (# 1 in this case) video frame of #i video data 1 is read.

  Next, the process proceeds to step ST106, where the subject and the imaging range in the #j video frame of #i video data 1 are determined based on the given position data of the subject such as an object or space and the imaging range data 10 read in step ST105. In step ST107, it is determined whether or not the positional relationships in the #j video frame of the #i video data 1 satisfy the specified positional relationship included in the search request. Do. Specifically, in step ST106, an inclusion relationship is calculated from whether the subject is included in the imaging range from the position data and the imaging range data 10, and in step ST107, the #j video of the #i video data 1 is calculated. It is determined whether the inclusion relationship in the frame satisfies the specified inclusion relationship.

  As a result of the determination, if the specified positional relationship is satisfied, the #j video frame is registered in the search result list in step ST108, and then the video frame counter j is incremented in step ST109. On the other hand, if not satisfied, the process directly branches to step ST109 and only increments the counter j. Thereafter, the process returns to step ST104, and the series of processes of step ST104 to step ST109 is repeated. For the #i video data 1, when the above processing is completed for all of the # 1 to #Fi video frames, the flow branches to step ST110 to increment the video data counter i. Thereafter, the process returns to step ST102, and the series of processes of steps ST102 to ST110 is repeated. When the above processing is completed for all of the M pieces of video data 1 of # 1 to #M, the search processing by the search processing unit 8 is finished, and a search result in which video frames satisfying the search condition are registered in the output unit 9 Pass the list.

  The output means 9 replies to the search application execution unit 4 as a search result with a search result list by the video ID for identifying the video data 1 and the start number and end number of the corresponding video frame as shown in FIG. The application execution unit 4 displays the search result list to the operator. If the operator instructs the display of the corresponding video data portion based on it, the search application execution unit 4 extracts the video raw data 5 of the corresponding video data portion from the video storage unit 2 and displays it.

  Here, the video data 1 has a data structure as shown in FIG. 3, for example. FIG. 3 is an explanatory diagram showing the data structure of the video data 1 according to the first embodiment. In the figure, reference numeral 11 denotes video data management including a pointer for specifying the raw video data 5 and a pointer for specifying the video attribute data 6. It is a table. As described above, the video data management table 11 can access the video raw data 5 and the video attribute data 6, and the video attribute data 6 and the imaging range data 10 can be accessed using the address pointer. The imaging range data 10 is described for each video frame in the example of FIG. 1 and for each set of 10 consecutive video frames in the example of FIG. The data items to be described include the camera position (X, Y, Z), the horizontal imaging angle (HL, HR), the vertical imaging angle (VU, VD), and the imaging distance (DN, DF). Yes, the camera position is expressed in the same global coordinate system as the subject position data, and the horizontal and vertical imaging angles and imaging distances are expressed in a polar coordinate system centered on the camera position.

FIG. 4 is an explanatory diagram showing the relationship between the subject and the imaging range, and 12 in the figure is the subject. FIG. 4 shows the relationship between the position data of the subject and the imaging range data of the video frame on the XY two-dimensional plane. The camera positions in the # 1 video frame and the # 10 video frame are (x1, y1) = (x10, y10). The imaging range of the video frame corresponds to a hatched donut-shaped portion defined by a horizontal imaging angle (HL, HR) and an imaging distance (DN, DF) from the camera. Here, the subject 12 indicated by the asterisk is not included in the imaging range of the # 1 video frame, but is included in the imaging range of the # 10 video frame. The inclusion relationship can be easily determined from the following expressions (1) and (2).
HR ≦ θ = tan ⁻¹ {(y−Y1) / (x−X1)} ≦ HL (1)
DN ≦ d = {(x−X1) ² + (y−Y1) ² } ^1/2 ≦ DF (2)

  The distance of the imaging range is determined from the zoom value (focal length) of the camera and the arrangement of buildings and equipment to be monitored. For example, a camera installed in a room cannot capture the outside of the room, and a correct search result can be returned if the distance of the imaging range is set in the room. FIG. 5 is an explanatory diagram showing the relationship between the distance of the imaging range and the focal length of the camera. In FIG. 5, 12 is a subject, and 13 is a projection plane on which the subject 12 of the camera is projected. As shown in FIG. 5, when the zoom value, that is, the focal length is changed, the size of the subject 12 on the projection plane 13 changes. For example, the far subject 12 cannot be observed in detail at a close focal length. Accordingly, it is effective to dynamically change the distance value of the imaging range by changing the zoom value.

  As described above, according to the first embodiment, the video data 1 stored in the video storage unit 2 includes the video raw data 5 that is a time-series set of video frames and the imaging range indicating the imaging range of each video frame. In the video search unit 3, the video search unit 3 accepts a search request including the positional data of the subject 12 such as an object or space, and the positional relationship between the subject 12 and the imaging range. The search processing unit 8 calculates the positional relationship between the position data and the imaging range data 10 in the video storage unit 2 to obtain a video frame satisfying the search request including the specified positional relationship, and the output unit 9 Since the search result is output to the inquiry source, it is possible to search and automatically extract only the video data portion in which the subject 12 whose position can be specified is shown. Because rater is there is no need to look up the image portion not reflected the object 12, can conduct business in a short time, the effect is obtained that fatigue is also reduced due to the work.

  In the above description, the point (x, y, z) is given as the position data of the subject. However, the position data may be given as a two-dimensional or three-dimensional area, or as a temporally changing locus. May be. In this way, when given by areas and trajectories, it can be easily extended to variations in which the whole is completely included or part is included. In addition, it is possible to deal with a combination of a plurality of elements such as including both areas A and B as position data.

  Furthermore, the horizontal and vertical imaging range data 10 is given by the minimum angle and the maximum angle, but it may be given by an angle representing the camera orientation and the horizontal and vertical spread, and the spread is also determined from the focal length and the zoom value. Various polar coordinate system parameters can be considered.

  Further, the data structure of the video data 1 is not limited to that shown in FIG. 3. For example, as shown in FIG. 6, the imaging range data 10 is included in the header portion of each video frame of the video raw data 5. It may be a structure as described.

Embodiment 2. FIG.
In the first embodiment, the case where the imaging range data is defined in the polar coordinate system of the camera has been described. However, it may be defined in the coordinate system of the subject. The second embodiment relates to such a video search apparatus, and the configuration of the apparatus and the procedure of search processing are the same as those of the first embodiment, except that the coordinate system and data structure of the imaging range data are different. FIG. 7 is an explanatory diagram showing the coordinate system and data structure of the imaging range data in the second embodiment. In the figure, 10a is imaging range data based on a circumscribed trapezoid, 10b is imaging range data based on a circumscribed rectangle, and 12 is a subject.

  Hereinafter, the coordinate system and data structure of the imaging range data will be described. The coordinate system of the imaging range data in the second embodiment is a global coordinate system that represents the position of the subject 12, and two examples are shown in FIG. One of them is imaging range data 10a based on a circumscribed trapezoid, which is defined by a trapezoid circumscribing an area on the donut that is the imaging range of the polar coordinate system. In the case of the three-dimensional case, the imaging range data is defined by a circumscribed pyramid. The determination as to whether or not the subject 12 is included in this area is the vertical relationship between the position (x, y) of the subject 12 and the four straight lines constituting the circumscribed trapezoid in the case of two dimensions, and the subject position in the case of three dimensions. What is necessary is just to calculate the vertical relationship between (x, y, z) and the six surfaces constituting the circumscribed frustum, and since the calculation method may use the basic formula of geometric mathematics, its description is omitted here.

  The other is imaging range data 10b based on a circumscribed rectangle defined by a rectangle circumscribing an area on the donut that is an imaging range of the polar coordinate system. In the case of the three-dimensional case, the imaging range data is defined by a circumscribed rectangular parallelepiped. In this case, whether or not the subject 12 is included in the region is determined by calculating the vertical relationship with the straight line that forms the circumscribed rectangle if it is two-dimensional, and the surface that forms the circumscribed cuboid if it is three-dimensional. Since the calculation of the upper and lower relations is simply a comparison of the components of the position (x, y, z), it is even easier.

  As described above, according to the second embodiment, since the imaging range data 10 of the video attribute data 6 is described in the coordinate system representing the position of the subject 12, the position of the subject 12 is determined each time as in the polar coordinate system. Because the process of converting to a polar coordinate value is no longer necessary, the positional relationship determination process can be performed easily and at high speed.Especially, when defining a circumscribed rectangle or circumscribed cuboid, it is an error from the true imaging area. However, the positional relationship determination process can be performed more easily and at a higher speed.

Embodiment 3 FIG.
In each of the above embodiments, the case where the search is performed based on the inclusion relationship between the subject and the imaging range has been described. However, the search may be performed based on the size or position of the subject on the projection plane. FIG. 8 is an explanatory diagram showing the positional relationship of the subject on the projection plane of the video search apparatus according to the third embodiment, in which 12 is the subject and 13 is the projection plane. 2A shows a case where the subject 12 is at the center of the projection plane 13, and FIG. 2B shows a case where the subject 12 is displaced from the center of the projection plane 13. FIG.

  In the third embodiment, the input unit 7 of the video search unit 3 sets conditions regarding the relationship on the projection surface 13 such as the size and position of the subject 12 on the projection surface 13 as the positional relationship between the subject 12 and the imaging range. The first embodiment is received except that the search processing unit 8 receives the included search request and calculates the relationship of the subject 12 on the projection plane 13 from the position data and the imaging range data 10 in step ST106 shown in FIG. It is the same as the case of.

  For example, as shown in FIG. 8A, when it is desired to search only the video portion where the subject 12 “A” is shown in the center, the position (x, y, z), size ( [Delta] x, [Delta] y, [Delta] z) is given, and "include & center" is designated that the area specified by the position data is combined and located in the center. In step ST106 in FIG. 2, the search processing means 8 first calculates how the designated area is projected on the projection plane of the camera.

As shown in FIG. 5, for example, when the subject 12 and the camera lens are facing each other, if the focal length of the camera lens is F and the distance from the lens to the subject 12 is L, the coordinate system X of the projection plane 13 is shown in FIG. The position (X0, Y0) at which the point (x0, y0) is projected on the projection plane 13 in the local coordinate system xy with the origin of the intersection of the center line of the lens and the subject 12 as opposed to -Y is It is given by the following equation (3).
(X0, Y0) = (x0, y0) × F ÷ L (3)

  If the subject 12 is not directly facing the camera lens, it may be obtained by calculation according to the above equation (3) after mapping the subject 12 to the local coordinate system xy facing the subject.

  By this calculation, it is determined whether the region specified in the search request is at the center of the projection surface 13 as shown in FIG. 8A, for example, or at the end of the projection surface 13 as shown in FIG. Can be sought.

  As described above, according to the third embodiment, the input unit 7 of the video search unit 3 uses the projection surface such as the size and position of the subject 12 on the projection surface 13 as the positional relationship between the subject 12 and the imaging range. A search request including a relational condition is received, and the search processing means 8 calculates the relationship of the subject 12 on the projection plane 13 from the position data and the imaging range data, and satisfies the search request including the relationship on the designated projection plane 13. For example, when it is desired to view only the video portion in which a certain subject 12 is reflected in the center of the projection plane 13, the operator can perform a more accurate search. There is an effect that it is not necessary to see an unnecessary video portion (for example, a video portion in which only a part of the subject 12 is shown) from the search result.

  In the above description, the positional relationship of whether or not the region of the subject 12 is in the center of the projection surface 13 is specified as the relationship on the projection surface 13, but the specified region occupies more than the area of the projection surface 13. It is also possible to specify a size relationship such as a case (that is, a case where the subject 12 looks larger than a certain size) or a combination of position and size.

  Further, the search condition includes not only those relating to the single subject 12 but also a plurality of subjects 12 including a condition that combines the position data of each region and the relationship between the position and size of each other on the projection plane 13. The search can be easily extended to a request, for example, a search request in which the subject 12 “A” is displayed on the projection plane 13 on the right side of the subject 12 “B”.

Embodiment 4 FIG.
In the first embodiment and the second embodiment, the search is performed based on the inclusion relationship between the subject and the imaging range, and in the third embodiment, the search is performed based on the size and position of the subject on the projection plane. The search may be performed according to the imaging direction. FIG. 9 is an explanatory diagram showing the relationship between the subject and the search direction in the video search apparatus according to the fourth embodiment, and shows a case where the subject 12 is imaged by four cameras # 1 to # 4. .

  In the fourth embodiment, the input unit 7 of the video search unit 3 accepts a search request including a condition regarding the direction in which the subject 12 is imaged as the positional relationship between the subject 12 and the imaging range, and the search processing unit 8 Except for calculating and determining the imaging direction of the subject 12 from the position data and the imaging range data 10 in Steps ST106 and ST107 of 2, the same as in the case of the first embodiment.

For example, as shown in FIG. 9, the subject 12 is captured from four cameras # 1 to # 4, and an angle of ± θ with respect to one point (designated point) on the N plane of the subject 12 around the vector V1 Suppose you want to search for the video part that is shown inside. In this case, as a search request, a direction designation “direct” which is a related condition, coordinates (x, y, z) of a designated point, a vector V1 of an imaging direction, and an angle range ± θ are given. In step ST106 and ST107 shown in FIG. 2, the search processing means 8 first calculates whether or not each video frame includes the designated point. If included, the search processing means 8 captures the vector V2 from the designated point toward the camera and the imaging. The angle φ between the vectors is obtained from the direction vector V1, and compared with the angle range ± θ. That is, it can be determined by the following equation (4).
θ ≧ φ ≧ −θ (4)

The angle φ between the two vectors can be obtained by the following equation (5), where the inner product of both vectors is V1 · V2.
φ = arccos (V1 · V2 ÷ (| V1 | × | V2 |)) (5)

  As described above, according to the fourth embodiment, the input unit 7 of the video search unit 3 accepts a search request including a condition regarding the direction in which the subject 12 is imaged as the positional relationship between the subject 12 and the imaging range. Since the processing means 8 calculates the imaging direction of the subject 12 from the position data and the imaging range data, and obtains a video frame that satisfies the search request including the designated imaging direction, for example, a certain subject 12 is detected within a certain angular range. If you only want to see the video part that is shown on the screen, you can search more accurately, and the operator will not have to see the unnecessary video part from the search results. .

  In the above description, a search for a single subject 12 has been shown as a search condition. However, the search can be easily extended to a search request for a combination in which imaging directions for a plurality of subjects 12 are specified.

Embodiment 5. FIG.
In each of the above embodiments, the search is performed based on the inclusion relationship between the subject and the imaging range, the size and position of the subject on the projection plane, and the imaging direction, but in addition to the positional relationship between the subject and the imaging range. Thus, the search may be performed using the imaging time as a condition. FIG. 10 is an explanatory diagram showing the data structure of the video data in the video search apparatus according to the fifth embodiment, in which 11 is a video data management table and 14 is an imaging range / imaging time data. The imaging range / imaging time data 14 includes imaging range data 10 indicating the imaging range of each video frame, and imaging time data 15 indicating the time when the video frame was captured.

  In the fifth embodiment, in addition to the positional relationship between the subject and the imaging range, the imaging time can be specified by a search request. The apparatus configuration is the same as that of the first embodiment shown in FIG. Although the same, the data structure of the video data 1 and the processing procedure of the search processing means 8 are different, and will be mainly described below.

  As shown in FIG. 10, in the video attribute data 6 of the video data 1, in addition to the imaging range data 10 indicating the imaging range of each video frame, imaging time data 15 indicating when the video frame is captured is described. The imaging range / imaging time data 14 can be accessed using an address pointer. Here, the relationship between the video data 1 and the imaging time interval specified by the search request is shown in FIG. For example, as shown in FIG. 11, from ts = 13: 20: 30 on January 6, 1997 to te = 13: 30 as well as the relationship condition between the subject and the imaging range that each video frame is in the search request. For a search that also takes an imaging time interval, such as video data captured at a time of 20 seconds, each video frame is acquired using the imaging range data 10 and the imaging time data 15 of the imaging range / imaging time data 14. Judgment is made.

Next, the processing procedure will be described.
Here, FIG. 12 is a flowchart showing the processing procedure of the search processing means 8 in the fifth embodiment. The search processing means 8 takes the imaging range / imaging time data 14 for each video frame of each video data 1. Thus, the imaging range data 10 and the imaging time data 15 are read out to make a determination. Accordingly, only steps ST205 to ST209 in FIG. 12 for reading and determining data are different from steps ST105 to ST108 in the first embodiment shown in FIG. 2, and the other steps are the same as those in the first embodiment. Since this is the same, only the operations of steps ST205 to ST209 will be described here.

  In step ST205, the search processing means 8 first reads the imaging range data 10 and the imaging time data 15 of the #j video frame of the #i video data 1, and then in step ST206, the search time interval [ ts, te] is checked. If it is included, the process proceeds to step ST207, and the positional relationship between the subject and the imaging range in the #j video frame of #i video data 1 is calculated in the same manner as in the first embodiment. It is determined whether or not the mutual positional relationship in the #j video frame of the i video data 1 satisfies the designated positional relationship included in the search request. As a result of the determination, if satisfied, the video frame is registered in the search result list in step ST209, and then the counter j is incremented in step ST210. If not satisfied, the video frame is not registered and remains as it is. The process branches to step ST210.

  As described above, according to the fifth embodiment, the video attribute data 6 includes the imaging range data 10 indicating the imaging range of each video frame and the imaging time data 15 indicating the imaging time. The input means 7 accepts a search request including the position data of the subject such as an object or space, the positional relationship between the subject and the imaging range, and the imaging time interval indicating the imaging time range, and the search processing means 8 receives the position data and the imaging range data. Since the positional relationship is calculated from 10 and the video frame satisfying the search request including the designated positional relationship and the imaging time interval is obtained, it is desired to search only the video in a certain time zone of a certain subject. In this case, the search can be performed with higher accuracy, and the operator can avoid the unnecessary video portion from the search result. .

  In the above description, the data structure of the video data 1 has been described as having the imaging time data 15 for each video frame. However, the present invention is not limited to this, and for example, as shown in FIG. The data structure may be As shown in FIG. 13, for each video data 1, the start and end times of imaging and the period that is the time interval of video frames are described in the video attribute data 6 as imaging time data 15a. The imaging time of each video frame of the video data 1 can be obtained from these imaging time data 15a by simple calculation.

  Thus, by describing the imaging time data 15a in units of video data, the amount of data to be accumulated can be reduced as compared to the case where the imaging time data 15 is described for each video frame as shown in FIG. Further, by comparing the start time and the end time of the video data 1 in the search request with the start time and the end time of the video data 1, the video frames of the video data 1 that are not in the image pickup time section are shown in steps ST205 to ST210 in FIG. When it is not necessary to perform the loop processing and a part of the video frames of the video data 1 is in the designated imaging time section, the processing of steps ST205 to ST210 only needs to be performed on the video frame 1, so the search is performed. Speed can also be improved.

  Furthermore, the imaging time for each video data is described in, for example, the document “The Advanced Video Information System: data structures and query processing” (ACM Multimedia Systems, Vol. 4, No. 4, pp. 172-18, pp. 172-18). If managed with a simple tree structure, there is no need to search for unnecessary video data at the time of searching, especially when the number of video data is large and the time interval to be searched is small compared to the entire recorded imaging time interval. Can greatly improve the search time. Note that a specific management method using a tree structure is not the essence of the present invention, and therefore the description thereof is omitted here.

  In the monitoring video and the recorded video, the imaging range is physically determined by the position, direction, focal length, and the like of the camera, and the imaging time is also real time, but is not necessarily limited thereto. Even in the case of drama, animation, etc., the same effect can be obtained when searching for a video scene to be viewed as long as the imaging range and virtual imaging time in the virtual space are defined and described in the video data.

Embodiment 6 FIG.
In each of the above-described embodiments, the description has been made of the one-stage search by the video frame. However, the search may be performed by two stages of the video frame from the video frame set. FIG. 14 is an explanatory view showing the data structure of the video data in the video search apparatus according to the sixth embodiment, and the corresponding parts are denoted by the same reference numerals as those in FIG. In the figure, 16 is the first imaging range data indicating the imaging range of the entire video frame included in the video frame set for each temporally continuous video frame set, and 17 is the imaging range of each video frame. Is the second imaging range data.

  The sixth embodiment is aimed at speeding up the search, and the configuration of the apparatus is the same as that of the first embodiment shown in FIG. 1, but the data structure of the video data 1 and the processing procedure of the search processing means 8 are the same. However, it is different from that.

  As shown in FIG. 14, the imaging range data includes, for each of a plurality of temporally continuous video frame sets, an imaging range of the entire video frame included in the video frame set, that is, an area including all the imaging ranges of each video frame. The first imaging range data 16 indicating the imaging range and the second imaging range data 17 indicating the imaging range of each video frame in the video frame set. The video attribute data 6 of the video data 1 can access the first imaging range data 16 and the second imaging range data 17 in two stages using an address pointer.

Next, the search processing procedure will be described.
FIG. 15 is a flowchart showing the processing procedure of the search processing means 8 in the sixth embodiment. As shown in FIG. 15, the search processing means 8 performs determination processing for all the video data 1, but since one video data 1 is divided into a plurality of video frame sets that continue in time, In ST306, first imaging range data 16 for #j video frame set of #i video data 1 is read. Next, in step ST307, the positional relationship between the subject and the imaging range is calculated based on the first imaging data 16, and it is determined in step ST308 whether the positional relationship specified by the search request is satisfied.

  If it is not satisfied as a result of the determination, any video frame included in the video frame set does not satisfy the search request condition, and the process proceeds to the check of the next video frame set through steps ST315, ST304, and ST305. On the other hand, if the condition is satisfied, the second imaging range data 17 is read for the #k video frame included in the #j video frame set in step ST310. Next, in step ST311, calculation of the positional relationship based on the second imaging range data 17, determination of whether or not the positional relationship specified in step ST312 is satisfied, registration in the search result list in step ST313, etc. Perform the process.

  If it is detected in steps ST314 and ST309 that the processing for all the video frames in the video frame set has been completed, the process proceeds to steps ST315, ST304, and ST305 to check the next video frame set. Thereafter, when it is detected in steps ST315 and ST304 that the processing for all video frame sets of the #j video data 1 has been completed, the process proceeds to the check of the next video data 1 through steps ST316, ST302 and ST303. Further, when it is detected in steps ST316 and ST302 that the processing for all the video data 1 of # 1 to #M has been completed, the search processing by the search processing means 8 is ended, and the obtained search result list is displayed. It passes to the output means 9.

  As described above, according to the sixth embodiment, the first imaging range data indicating the imaging range of the entire video frame included in the set of the plurality of temporally continuous video frame sets. 16 and second imaging range data 17 indicating the imaging range of each video frame, and the search processing means 8 of the video search unit 3 first has a positional relationship with the video frame set based on the first imaging range data 16. To obtain a set of video frames in which there can exist video frames satisfying the search request, and further calculate the positional relationship for each video frame belonging to the video frame set based on the second imaging range data 17 Since the video frames satisfying the search request are obtained, the imaging range data is read for all the video frames in all the stored video data 1. There is no need to perform calculations and determine the positional relationship, there is an advantage that it reduces the search processing time.

  In the above description, the continuous video frame set, which is the unit of the first imaging range data 16, is obtained by dividing one video data 1 into a plurality of pieces, but it is one video data 1 itself. May be. In this case, the data structure of the video data 1 is as shown in FIG. 16, for example. That is, the imaging range of the entire video frame included in the video data 1 may be described as the first imaging range data 16 in the video attribute data 6 of each video data 1. Further, the first imaging range data 16 is defined for one video data 1, and the first imaging range data 16 is also defined for a set of video frames obtained by dividing the video data 1. A data structure such as 16 is also conceivable.

  Further, the interval between video frame sets does not need to be equal, and for example, arbitrary settings such as a section where the camera is moving and a section where the camera is stationary are possible.

  Furthermore, if the first imaging range data 16 is separately managed with a tree structure as shown in the document “Image Database” (Masao Sakauchi, Hiroaki Osawa, 1987), an unnecessary set of video frames at the time of retrieval is obtained. The search time is greatly improved, especially when the amount of video data is large, the entire imaging area (for all cameras) is wide, and the imaging range to be searched is small compared to the whole. Figured. Although a specific management method using a tree structure is not the essence of the present invention and will not be described in detail, for example, for each of the first imaging range data 16, a circumscribed rectangle and a center point of the imaging range are defined. In addition, a tree such as a kd tree or a BD tree is generated at the center point, and a circumscribed rectangle including a circumscribed rectangle of a child node is defined for each node. A circumscribed rectangle and corresponding first video data are defined in a leaf node. It is possible to write a pointer value that can be referenced.

  In the above description, the unit of the second imaging range data 17 is a video frame. However, it is a video frame sub-set obtained by further subdividing the video frame set that is the unit of the first imaging range data 16. There may be.

Embodiment 7 FIG.
In the sixth embodiment, the case where the imaging range of the video frame is obtained directly from the second imaging range data has been described. However, the imaging range of the video frame may be obtained by calculation. FIG. 17 is an explanatory view showing the data structure of the video data in the video search apparatus according to the seventh embodiment. The same reference numerals are assigned to the corresponding parts, and the description thereof is omitted. In the figure, reference numeral 18 denotes third imaging range data for calculating the imaging range of each video frame.

  As in the case of the sixth embodiment, the seventh embodiment aims to speed up the search, and the processing procedure of the search processing means 8 is similar to that. The description will focus on the structure and the parts different from the sixth embodiment in the processing procedure of the search processing means 8.

  As shown in FIG. 17, the imaging range data is used to calculate first imaging range data 16 indicating an area including all the imaging ranges of each video frame included in the video frame set, and the imaging range of each video frame. Of the third imaging range data 18. The third imaging range data 18 represents the initial value and change rate of the imaging range data in the video frame set. For example, the video frame set of video frames # 1 to # 10 indicates that the horizontal angle in the direction of the camera has moved from the initial value of 20 degrees to 2 degrees / frame. The data items of the third imaging range data 18 are the camera position, the camera direction by the horizontal angle and the vertical angle, and the zoom value. As described in the first embodiment, the camera position, direction, and zoom From the value, the imaging range for the video frame of FIG. 4 can be easily calculated.

Next, the search processing procedure will be described.
FIG. 18 is a flowchart showing the processing procedure of the search processing means 8 in the seventh embodiment. As shown in FIG. 18, in the search processing procedure of the seventh embodiment, instead of reading the second imaging range data 17, the third imaging range data 18 is read and the third imaging range is read. This is different from the search processing procedure of the sixth embodiment shown in FIG. 15 in that processing for calculating the imaging range of the video frame based on the data 18 is added. That is, when the positional relationship of the #j video frame set calculated based on the first imaging range data 16 is determined to satisfy the search request in step ST408, the third # of the #j video frame set is determined in step ST409. In step ST411, the imaging range data 18 of the #k video frame is calculated.

  As described above, according to the seventh embodiment, the first imaging range data indicating the imaging range of the entire video frame included in the set of the plurality of temporally continuous video frame sets. 16 and third imaging range data 18 for calculating the imaging range of each video frame, and the search processing means 8 of the video search unit 3 first selects the video frame based on the first imaging range data 16. By calculating the positional relationship with the set, a video frame set in which a video frame satisfying the search request can be obtained is obtained, and the imaging range of the video frame belonging to the video frame set is determined based on the third imaging range data 18. Thus, the positional relationship is calculated and a video frame satisfying the search request is obtained. Therefore, the imaging range data of each video frame is reduced as compared with the case of the sixth embodiment. Can be written in data amount, there is an effect of reducing the capacity of the recording apparatus.

  In the seventh embodiment, as in the case of the sixth embodiment, the unit of the first imaging range data 16 is very good as one video data itself, and for one video data. The first imaging range data 16 may be defined, or the first imaging range data 16 may be defined for a set of video frames obtained by dividing video data. Further, the search speed is improved by separately managing the tree structure. Expansion such as making it possible is conceivable.

  The unit of the third imaging range data 18 may also be a video frame sub-set obtained by further finely dividing the video frame set that is the unit of the first imaging range data 16.

Embodiment 8 FIG.
In the sixth embodiment and the seventh embodiment described above, the search is performed in two stages based on the video frame from the video frame set. However, the search is performed in the three stages of the video frame set and the video frame from the camera. Also good. FIG. 19 is an explanatory view showing the data structure of the video data in the video search apparatus according to the eighth embodiment. The same reference numerals are assigned to the corresponding parts, and the description thereof is omitted. In the figure, 19 is the fourth imaging range data showing the imaging range of the entire video frame belonging to the video data captured by the camera for each camera in the form of a circumscribed rectangle, and 20 is the fourth imaging range data. 4 is a camera management table that associates imaging range data 19 with camera IDs and video data IDs.

  As in the case of the sixth embodiment, the eighth embodiment aims to speed up the search, and the processing procedure is similar to that. Therefore, the video data 1 different from the sixth embodiment is described below. The difference between the data structure and the sixth embodiment in the processing procedure of the search processing means 8 will be mainly described.

  As shown in FIG. 19, the video data 1 of the eighth embodiment includes a camera management table 20 in addition to the video data 1 of the sixth embodiment shown in FIG. The camera management table 20 includes a camera ID for each camera, fourth imaging range data 19 indicating the imaging range of the entire video frame belonging to the video data imaged and stored by the camera, and the ID of the video data. Are described, and can be mutually referred to the video data management table 11. The camera management table 20 is used for search processing described below.

Next, the search processing procedure will be described.
FIG. 20 is a flowchart showing the processing procedure of the search processing means 8 in the eighth embodiment. As shown in FIG. 20, in the search processing procedure of the eighth embodiment, first, in step ST501, a counter 1 for camera discrimination is initially set to “1”, and whether or not the camera is a corresponding camera in steps ST502 to ST505. Determine. That is, in step ST503, the fourth video data 19 is read into the camera # 1, and the positional relationship in the search request is calculated in step ST504, and it is determined whether or not the positional relationship is satisfied in step ST505. .

  As a result, if satisfied, there is a possibility that a video frame satisfying the search request exists in the video data imaged and stored by the #l camera. Therefore, in steps ST506 to ST519, the #l camera It is determined whether or not the video data captured in (1) satisfies the positional relationship. If the video data is satisfied, a search is performed for a video frame that satisfies the positional relationship in the video data.

  On the other hand, if it is not satisfied as a result of the determination in step ST505, the process branches to step ST522 to increment the counter l, and the process returns to step ST502 to search video data captured by the next camera. If it is detected in step ST507 that the search for all frame sets of video data captured by the #l camera is completed, the process branches to step ST522 to increment the counter l, and the process is stepped. Returning to ST502, the video data captured by the next camera is searched. When it is detected in step ST502 that the search of the video data captured by all the cameras # 1 to #C is completed, the search process by the search processing unit 8 is ended and the obtained search result list is output. Passed to means 9.

  As described above, according to the eighth embodiment, in addition to the first imaging range data 16 and the second imaging range data 17, the imaging range data is converted into video data captured by the camera for each camera. The fourth imaging range data 19 indicating the imaging range of the entire video frame to which the video frame belongs belongs, and the search processing means 8 of the video search unit 3 first selects the video frame set belonging to each camera based on the fourth imaging range data 19. The positional relationship is calculated to find a camera that can have a video frame that satisfies the search request. Next, based on the first imaging range data 16, the positional relationship with respect to the set of video frames captured by the camera is calculated. Thus, a video frame set in which video frames satisfying the search request can be obtained is obtained, and the second imaging range data 17 is obtained for each video frame belonging to the video frame set. Since the positional relationship is calculated based on the video frame satisfying the search request, it is not necessary to calculate and determine the positional relationship for all the stored video data 1, thereby speeding up the search process. In particular, when the imaging range of each camera does not overlap each other or there is little overlap, the number of video data to be searched can be significantly reduced, and the search processing can be speeded up. Is obtained.

  In the above description, the fourth imaging range data 19 is data indicating the imaging range of the entire video frame belonging to the video data imaged and stored by the camera. This is the default for each camera. It may be a fixed value defined in (1), or may be dynamically updated in response to accumulation / deletion of video data.

  In the above description, the fourth imaging range data 19 is shown as a circumscribed rectangle. However, other circumscribed polygons may be used, and a polar coordinate system can be described if the camera has a fixed position. is there. Furthermore, if the fourth imaging range data 19 is separately managed in the above-described tree structure, the search process can be further speeded up.

  Further, in the above description, the video data 1 is obtained by using the second imaging range data 17 as the data indicating the imaging range of the video frame. However, the third imaging range data 18 described in the seventh embodiment is used. You may make it use. By doing so, the imaging range data of each video frame can be described with a small amount of data, and the capacity of the recording apparatus can be reduced.

  In addition, when the camera position changes like a camera robot and moves in a wide area, the fourth imaging range data 19 is described by dividing it into a plurality of sub-cameras based on the imaging position and imaging time. Thus, it is possible to narrow down the video data to be checked.

Embodiment 9 FIG.
In each of the above-described embodiments, the video search device that searches video data has been described. However, the video search device can be applied to a video search system that searches for the position data of a subject from a subsystem. FIG. 21 is a block diagram showing the configuration of such a video search system according to the ninth embodiment of the present invention. In the figure, 21 is an equipment database system as the above-mentioned subsystem, and 22 is also a geographic information system as a subsystem, and is connected to the search application execution unit 4 via a network such as a local area network (LAN). Other parts are denoted by the same reference numerals as those in FIG. 1, and the description thereof is omitted.

  As shown in FIG. 21, the video search system according to the ninth embodiment includes the video search apparatus described in each of the above embodiments and subject position data such as the equipment database system 21 and the geographic information system 22. It is a system that integrates a sub-system that manages various information about the subject.

Next, the operation will be described.
The search application execution unit 4 displays the state and position of the equipment to be monitored in a table format or a graphics format to the operator. For example, the name of the equipment, the operation parameter, and the measured value are displayed in a table format or a graph format, or the equipment object is displayed on the monitoring area map. In addition, what is displayed may be displayed independently of subsystems such as the equipment database system 21 and the geographic information system 22 integrated by the search application execution unit 4 itself. It may be displayed on the facility database system 21 or the geographic information system 22 by instructing whether to display it.

  In the former case, when the operator selects an object displayed on the table or map with a mouse click or the like, the search application execution unit 4 identifies the object, obtains the name or ID of the object, Is used as a parameter to inquire about the position where the object exists in the equipment database system 21 or the geographic information system 22, and the position data is obtained. In the latter case, when an operator selects an object displayed on a table or a map with a mouse click or the like, the equipment database system 21 or the geographic information system 22 identifies the object, and only the object name and ID are used. The position data is also obtained and the search application execution unit 4 is notified of the position data where the object exists together with the object name and ID. Such a linkage method between the search application execution unit 4 and the subsystems such as the facility database system 21 and the geographic information system 22 can be realized by a known technique and is not the essence of the present invention. Description is omitted.

  When an operator wants to see a monitoring image of a certain facility or area, the operator does not search for the position data of the subject and input the keyboard in the form of coordinates or the like. Specify by mouse click. By the linkage function between the search application execution unit 4 and the subsystems such as the equipment database system 21 and the geographic information system 22 described above, the search application execution unit 4 uses the equipment database system 21 and the geography as the position data of the subject object selected by the operator. The information is acquired from the information system 22 and the like, and is output as a search request to the video search unit 3 of the video search device together with the specified positional relationship condition. The video search unit 3 registers video data (video frames) showing the designated subject in the search result list according to the search procedure described in the above embodiments, and the search result obtained at the end of the search procedure The list is output to the search application execution unit 4 as a search result.

  After that, as in the other embodiments, the search application execution unit 4 displays the search result to the operator, and if the operator instructs display of the corresponding video data portion based on the search result, the search application execution unit 4 The video raw data 5 corresponding to the video data 1 is extracted from the video storage unit 2 and displayed.

  As described above, according to the ninth embodiment, the video data 1 stored in the video storage unit 2 is converted into the video raw data 5 that is a time series set of video frames, and the imaging range indicating the imaging range of each video frame. In the video search unit 3 of the video search device, the input means 7 accepts a search request including position data of a subject such as an object or space and a positional relationship between the subject and the imaging range. The search processing means 8 calculates the mutual positional relationship from the position data and the imaging range data 10 in the video storage unit 2 to obtain a video frame satisfying the search request including the positional relationship specified in the search request. The search result obtained by the output means 9 is output to the search application execution unit 4, and the search application execution unit 4 detects the subject specified by the operator or the like. The position data is acquired from the subsystems 21 and 22 that manage at least the position data of the subject, such as buildings and devices, and a search request including the acquired position data is sent to the video search unit 3 of the video search device. Therefore, the operator can search the video data in which the object is reflected by selecting the corresponding object by directly operating from the equipment operation table or the map, and display the video raw data 5. The effect of realizing a good video search system can be obtained.

  In the above description, the video data is searched by direct operation from the equipment operation table or map. However, the present invention is not limited to this. For example, the operator can select the type of equipment or the type of alarm event. The search application execution unit 4 inquires about the equipment corresponding to the subsystems such as the equipment database system 21 and the geographic information system 22, acquires the position data based on the equipment ID list of the answer result, and inquires the video search apparatus. Various user interfaces are possible.

  In the description of the sixth to ninth embodiments, the search request uses the relationship between the position of the subject and the imaging range as a key. However, as in the case of the fifth embodiment, the search request includes the imaging time. It may be a condition.

  In all of the above embodiments, the relationship between the position of the subject and the imaging range and the conditions related to the imaging time are each one. However, a search request based on a logical expression in which a plurality of conditions are combined by AND or OR. Good.

  Further, in all of the above embodiments, when there are a plurality of corresponding video data, the search result list is returned with priorities assigned according to the designated conditions, or only a predetermined number of video data is returned. It is also possible to make it. For example, when the same subject is imaged at different angles with a plurality of cameras, it is possible to automatically select the best captured image from the corresponding video data, or to give a reply with priority information.

It is a block diagram which shows the structure of the video search device by Embodiment 1 of this invention. It is a flowchart which shows the procedure of the search process in the said Embodiment 1. It is explanatory drawing which shows the data structure of the video data in the said Embodiment 1. FIG. It is explanatory drawing which shows the relationship between the to-be-photographed object in the said Embodiment 1, and an imaging range. It is explanatory drawing which shows the relationship between the distance of the imaging range in the said Embodiment 1, and the focal distance of a camera. It is explanatory drawing which shows the other example of the video data structure in the said Embodiment 1. FIG. It is explanatory drawing which shows the coordinate system and data structure of imaging range data in the image | video search device by Embodiment 2 of this invention. It is explanatory drawing which shows the positional relationship on the projection surface of a to-be-photographed object in the image | video search device by Embodiment 3 of this invention. It is explanatory drawing which shows the relationship between the to-be-photographed object and the search direction in the video search device by Embodiment 4 of this invention. It is explanatory drawing which shows the data structure of the video data in the video search device by Embodiment 5 of this invention. It is explanatory drawing which shows the relationship between the video data in the said Embodiment 5, and the imaging time area designated by the search request. It is a flowchart which shows the procedure of the search process in the said Embodiment 5. It is explanatory drawing which shows the other example of the data structure of the video data in the said Embodiment 5. It is explanatory drawing which shows the data structure of the video data in the video search device by Embodiment 6 of this invention. It is a flowchart which shows the procedure of the search process in the said Embodiment 6. It is explanatory drawing which shows the other example of the data structure of the video data in the said Embodiment 6. FIG. It is explanatory drawing which shows the data structure of the video data in the video search device by Embodiment 7 of this invention. It is a flowchart which shows the procedure of the search process in the said Embodiment 7. It is explanatory drawing which shows the data structure of the video data in the video search device by Embodiment 8 of this invention. It is a flowchart which shows the procedure of the search process in the said Embodiment 8. It is a block diagram which shows the structure of the video search system by Embodiment 9 of this invention. 1 is a system configuration diagram illustrating a configuration example of a general video monitoring system. It is a block diagram which shows the structure of the conventional video search apparatus.

Explanation of symbols

  1 video data, 2 video storage unit, 3 video search unit, 4 search application execution unit (inquiry source), 5 video raw data, 6 video attribute data, 7 input means, 8 search processing means, 9 output means, 10 and 10a , 10b imaging range data, 12 subjects, 13 projection planes, 15, 15a imaging time data, 16 first imaging range data, 17 second imaging range data, 18 third imaging range data, 19 fourth imaging range Data, 21 Equipment database system (subsystem), 22 Geographic information system (subsystem).

Claims (9)

A video storage unit that stores video data, and a video search unit that searches for video data that satisfies the search conditions of the search request in response to a search request from the inquiry source and answers the result to the inquiry source. Video search device
The video data stored in the video storage unit is composed of video raw data that is a time-series set of video frames, and video attribute data including imaging range data indicating the imaging range of each video frame,
The video search unit
An input means for receiving a search request including a condition relating to a direction in which the subject is imaged as position data of the subject and a positional relationship between the subject and the imaging range;
Search processing means for calculating a direction in which the subject is imaged from the position data and imaging range data, and for obtaining a video frame satisfying a search request including a condition relating to the designated imaging direction;
A video search apparatus comprising output means for outputting a search result of the search processing means to the inquiry source. The input means of the video search unit accepts a search request including a condition regarding a direction in which a plurality of subjects are imaged,
The search processing means of the video search unit calculates a direction in which the plurality of subjects are imaged from a plurality of subject position data and shooting range data, and satisfies a search request including a condition regarding the designated imaging direction. 2. The video search apparatus according to claim 1, wherein a video frame to be obtained is obtained. The video attribute data of the video data stored in the video storage unit includes imaging range data indicating the imaging range of each video frame, and imaging time data indicating the imaging time of each video frame,
The input means of the video search unit accepts a search request including a subject position data, a positional relationship between the subject and the imaging range, and an imaging time interval indicating the imaging time range,
The search processing means of the video search unit calculates a positional relationship between the position data and the imaging range data, and obtains a video frame satisfying a search request including the specified positional relationship and imaging time interval. The video search apparatus according to claim 1, wherein the video search apparatus is provided. Video attribute data of video data stored in the video storage unit includes imaging range data indicating an imaging range of each video frame, and imaging time data for calculating an imaging time of each video frame,
The input means of the video search unit accepts a search request including a subject position data, a positional relationship between the subject and the imaging range, and an imaging time interval indicating the imaging time range,
A search request in which the search processing means of the video search unit calculates a positional relationship between the position data and the imaging range data and an imaging time from the imaging time data, and includes the specified positional relationship and an imaging time interval 3. A video search apparatus according to claim 1, wherein a video frame satisfying the above is obtained. The imaging range data in the video attribute data of the video data includes, for each of a plurality of temporally continuous video frame sets, first imaging range data indicating an imaging range of the entire video frame included in the set, and each video frame or Each video frame sub-set obtained by dividing the video frame set is composed of second imaging range data indicating the imaging range,
The search processing means of the video search unit first calculates a positional relationship with respect to the video frame set based on the first imaging range data to obtain a video frame set in which a video frame satisfying the search request can exist, A video frame satisfying the search request is obtained by calculating a positional relationship for each video frame or video frame sub-set belonging to the video frame set based on the second imaging range data. The video search device according to claim 1 or 2.   The imaging range data in the video attribute data of the video data includes, for each of a plurality of temporally continuous video frame sets, first imaging range data indicating an imaging range of the entire video frame included in the set, and each video frame or For each video frame sub-set obtained by dividing the video frame set, it is composed of third imaging range data for calculating the imaging range, and the search processing means of the video search unit first extracts the first imaging range data. And calculating a positional relationship with respect to the video frame set to obtain a video frame set in which a video frame satisfying the search request may exist, and further determining an imaging range of each video frame or video frame sub-set belonging to the video frame set. And calculating the positional relationship based on the third imaging range data to obtain a video frame satisfying the search request. Video searching apparatus according to claim 1 or claim 2 wherein Rukoto. In addition to the first imaging range data and the second imaging range data, the imaging range data in the video attribute data of the video data is captured by the camera for each camera in addition to the first imaging range data and the second imaging range data. Composed of the fourth imaging range data indicating the imaging range of the entire video frame belonging to the video data,
The search processing means of the video search unit first calculates a positional relationship with respect to the entire video frame belonging to each camera based on the fourth imaging range data, and obtains a camera in which a video frame that satisfies the search request can exist. Next, based on the first imaging range data, a positional relationship with respect to a set of video frames captured by the camera is calculated to obtain a video frame set in which a video frame satisfying the search request can exist, and further For each video frame or video frame sub-set belonging to the video frame set, a positional relationship is calculated based on the second imaging range data, and a video frame satisfying the search request is obtained. The video search device according to claim 5. In addition to the first imaging range data and the third imaging range data, the imaging range data in the video attribute data of the video data is captured by the camera for each camera in addition to the first imaging range data and the third imaging range data. Composed of the fourth imaging range data indicating the imaging range of the entire video frame belonging to the video data,
The search processing means of the video search unit first calculates a positional relationship with respect to all video frames belonging to each camera based on the fourth imaging range data, and obtains a camera in which a video frame satisfying the search request can exist. Next, a positional relationship with respect to a set of video frames captured by the camera based on the first imaging range data is calculated to obtain a video frame set in which a video frame satisfying the search request can exist, and Based on the third imaging range data, the imaging relationship of each video frame or video frame sub-set belonging to the video frame set is obtained, the positional relationship is calculated, and the video frame satisfying the search request is obtained. The video search device according to claim 6. The video search device according to any one of claims 1 to 8,
A search application execution unit as an inquiry source for sending a search request to the video search device;
A video search system composed of a subsystem that manages at least the subject's position data for the subject,
The video search characterized in that the search application execution unit acquires position data of a specified subject from the subsystem and sends a search request including the acquired position data to the video search device. system.

Images