JP4929102B2 - Video search system and video search device - Google Patents

Description

  The present invention relates to a video search system and a video search apparatus that search video data shot by a plurality of cameras.

  In recent years, for example, a video search system that stores video data (hereinafter simply referred to as video) captured by a camera in a medium such as a hard disk drive (HDD) and searches for the stored video is known. It has been.

  In such a video search system, for example, when searching for videos taken by a plurality of cameras installed in a store or the like for security purposes, a plurality of same objects (for example, the same person) taken by different cameras are searched. It is necessary to search in association with each other video.

Therefore, for example, a technology (hereinafter referred to as prior art) that enables autonomous tracking of a moving body in the entire system and can efficiently collect and manage information distributed to a plurality of cameras is disclosed. (For example, refer to Patent Document 1). According to this prior art, for example, based on a feature amount (for example, color information) of a specific moving body, a moving body that is distributed for each of the plurality of video cameras is searched by searching for the moving body identified by the feature amount. Enables comprehensive management of tracking information.
JP 2004-72628 A

  However, in the video search system as described above, for example, a method of processing a real-time video in which video flows in time series and a video at a future time cannot be referred to is employed. For this reason, for example, when video is stored, it is possible to refer to all scenes of related video when processing a specific video. It may cause a decrease.

  Further, as in the above-described prior art, a case is assumed where videos taken by a plurality of cameras are searched based on color information, for example. In this case, for example, color information influenced by the light source is calculated according to the position where the camera is installed. For this reason, depending on the camera that captured the video, for example, a video having a color different from the color perceived by a person may be searched. Similarly, when the color expressed on the video differs depending on the characteristics of the camera, for example, a video having a color different from the color perceived by a person may be searched.

  In addition, when executing a video search, the process is generally executed by the user specifying a keyword or a sample image as a search condition. In this case, normally, the user cannot specify the conditions regarding the search speed or the search accuracy. For this reason, for example, only the search depending on the specifications in the video search system can be executed.

  An object of the present invention is to provide a video search system that efficiently searches videos taken by a plurality of cameras.

  According to one aspect of the present invention, a plurality of cameras that are installed within a certain range and that are configured to capture images composed of images that include objects within the range and the plurality of cameras are used. Retrieving an image including a search object designated by a user from among a receiving unit for receiving a video, a storage unit for storing the received video, and an image constituting the video stored in the storage unit Search condition input means for inputting the search conditions, analysis means for analyzing the image constituting the video stored in the storage means, and generating metadata indicating attributes characterizing the object included in the analyzed image And a first search unit that searches the storage unit for an image including the search object based on the input search condition and the generated metadata. And on the basis of the generated metadata, concealment occurrence detection means for detecting occurrence of concealment with respect to the search object included in the image searched by the first search means, and occurrence of the concealment is detected. In the case where the image stored in the storage means is searched for an image that does not conceal the search target object based on the input search condition and the generated metadata. 2 search means, acquisition means for acquiring metadata indicating an attribute characterizing the search object included in the image searched by the second search means, and the search based on the acquired metadata There is provided a video search system comprising: a third search means for searching for an image including an object from the storage means.

  According to the present invention, it is possible to efficiently search for videos taken by a plurality of cameras.

  Embodiments of the present invention will be described below with reference to the drawings.

  With reference to FIG.1 and FIG.2, the outline | summary of the video search system which concerns on this embodiment is demonstrated. FIG. 1 is a block diagram showing a configuration of a video search system according to this embodiment. As shown in FIG. 1, the video search system includes a camera 1, an image processing device 10, and a video search device 50.

  The camera 1 has a function of shooting various videos. Here, FIG. 2 is a diagram showing, for example, a store floor where the camera 1 shown in FIG. 1 is installed. As shown in FIG. 2, a plurality of cameras 1 are installed within a certain range such as a store floor 2. Further, the camera 1 captures an image including an object (for example, a person) 4 existing in the monitoring range 3 within the monitoring range 3 of the camera 1.

  In the video search system according to this embodiment, for example, a search process for the object 4 that moves (through) between a plurality of cameras 1 installed on the store floor 2 is executed as shown in FIG.

  The image processing apparatus 10 is connected to each of the cameras 1. The image processing apparatus 10 inputs video data captured by the camera 1 (hereinafter simply referred to as video). This video is composed of, for example, a plurality of image data (hereinafter simply referred to as images). The video includes audio data, for example. The image processing device 10 transmits the input video to the video search device 50 via the network 20, for example.

  The video search device 50 is connected to each of the image processing devices 10 via, for example, the network 20. The video search device 50 receives the video transmitted by the image processing device 10 via the network 20. The video search device 50 stores the received video. In addition, the video search device 50 searches, for example, a video (a constituent image) including a specific object that has been moved (through) between the plurality of cameras 1 described above, from the stored video.

  FIG. 3 is a block diagram mainly showing a functional configuration of the image processing apparatus 10 shown in FIG. Since a plurality of image processing devices 10 included in the video search system have the same configuration, only one image processing device 10 will be described below. As illustrated in FIG. 3, the image processing apparatus 10 includes an input unit 11, an image cutout unit 12, a metadata generation unit 13, and a transmission unit 14.

  The input unit 11 inputs video (data) taken by the camera 1 connected to the image processing apparatus 10. Note that the image captured by the input unit 11 includes an object (for example, a person) 4 (shown).

  The image cutout unit 12 cuts out a region (image) of the object 4 included in the image in the image constituting the video input by the input unit 11.

  The metadata generation unit 13 has a function of generating metadata by performing image processing on the image cut out by the image cutout unit 12. The metadata generation unit 13 analyzes the image cut out by the image cutout unit 12. The metadata generation unit 13 generates metadata indicating attributes that characterize the target object 4 included in the analyzed image (hereinafter referred to as metadata related to the target object 4). The metadata related to the object 4 includes, for example, color information or luminance information related to the object 4. The metadata generated by the metadata generation unit 55 is given to the image used for generating the metadata. Further, when a plurality of metadata is generated for one object 4, the plurality of metadata are collected into one (by coding) and handled as an information pattern.

  Here, a specific example of metadata generated by the metadata generation unit 13 will be described. For example, when the object 4 included in the analyzed image is a person, the metadata generated by the metadata generation unit 13 includes, for example, facial features, hair color, hairstyle, hair length, hair knitting , Head shape, clothing color / type, shoe type / color, bag type / color / shape of bag / paper bag, etc., system, posture, height or ear shape.

  For example, when the object 4 included in the analyzed image is a car, the metadata generated by the metadata generation unit 13 includes color, car type, manufacturer emblem, body shape, tire shape, wheel Includes type, light shape, glass shape or engine sound etc.

  For example, when the object 4 included in the analyzed image is an animal, the metadata generated by the metadata generation unit 13 includes face shape, ear shape, body color, hair growth, and number of legs. , Body shape or crying.

  For example, when the object 4 included in the analyzed image is a landscape, the metadata generated by the metadata generation unit 13 includes the color arrangement pattern, sky, sea, river, soil, clouds, flowers, trees in the image. Or the color and arrangement of the forest.

  For example, when the object 4 included in the analyzed image is a vehicle, the metadata generated by the metadata generation unit 13 includes color, body shape, number of tires, glass shape, handle shape, The shape of the muffler is included.

  The transmission unit 14 transmits a video composed of an image to which metadata is added to the video search device 50 via the network 20.

  FIG. 4 is a block diagram mainly showing a functional configuration of the video search device 50 shown in FIG. The video search device 50 includes a receiving unit 51, a video storage processing unit 52, a video storage unit 53, a search condition input unit 54, a metadata generation unit 55, a search processing unit 56, a search result output unit 57, and a search option setting unit 58. Including.

  The receiving unit 51 receives the video (image) transmitted by the image processing apparatus 10 (the transmitting unit 14 included therein). The receiving unit 51 receives video transmitted from each of the plurality of image processing apparatuses 10 included in the video search system.

  The video accumulation processing unit 52 executes a process of accumulating (storing) the video received by the reception unit 51 in the video accumulation unit 53. At this time, the video storage processing unit 52 stores the video received by the receiving unit 51 in the video storage unit 53 for each camera 1 that captured the video (for each image processing apparatus 10 that transmitted the video). The video accumulation processing unit 52 accumulates the video (image) received by the reception unit 51 in time series.

  The search condition input unit 54 inputs search conditions for searching for an image including an object 4 (hereinafter referred to as a search object) 4 designated by a user who uses the video search device 50, for example. In addition, for example, when tracking a specific object 4 in real time, the search condition may be automatically input by specifying the object 4. This search condition includes, for example, attribute information (metadata) indicating an attribute characterizing the search object 4 or a sample image.

  The metadata generation unit 55 has a function of generating metadata by performing image processing on images constituting the video stored in the video storage unit 53. The metadata generation unit 55 analyzes the images constituting the video stored in the video storage unit 53. The metadata generation unit 55 generates metadata indicating attributes that characterize the object (for example, person) 4 included in the analyzed image. The metadata generated by the metadata generation unit 55 is added to an image used for generating the metadata, for example.

  As described above, the image processing apparatus 10 includes the metadata generation unit 13 having the same function as the metadata generation unit 55. In the present embodiment, for example, a case where metadata is given to a video in advance on the image processing apparatus 10 side and a case where metadata is generated and searched on the video search apparatus 50 side are conceivable. For this reason, the configuration may be such that the metadata is generated by the metadata generation unit 13 of the image processing device 10 as described above, or the metadata is generated by the metadata generation unit 55 of the video search device 50.

  Hereinafter, in the present embodiment, a case where metadata is generated by the metadata generation unit 55 of the video search device 50 and search processing for the video storage unit 53 is executed will be described.

  The search processing unit 56 executes processing for searching for an image (video) that matches the search condition based on the search condition input by the search condition input unit 54 and the metadata generated by the metadata generation unit 55. . That is, the search processing unit 56 searches the video storage unit 53 for an image (video) including the search target 4 designated by the user.

  The search processing unit 56 collects metadata related to the search target 4 from the entire searched image (video). That is, the search processing unit 56 collects metadata related to the search target object 4 assigned to each searched image. The search processing unit 56 searches for an image including the search target object 4 based on the collected metadata. That is, the search processing unit 56 performs identification (collation) processing of the search target 4 designated by the user by comparing the collected metadata and the metadata generated by the metadata generation unit 55. In this way, metadata is collected from the entire searched video, and search performance is improved by executing search processing based on the metadata.

  The search processing unit 56 detects occurrence of concealment (overlap) with respect to the search object 4 included in the searched image in the searched image. That is, the search processing unit 56 detects that the search target 4 included in the searched image overlaps with another target included in the image. Hereinafter, an image in which concealment with respect to the search object 4 has occurred is referred to as a concealment occurrence image. The search processing unit 56 detects the occurrence of concealment based on the metadata related to the search target object 4.

  When the occurrence of concealment with respect to the search object 4 is detected, the search processing unit 56 searches for images before and after the concealment occurrence image in time series, and without occurrence of concealment with respect to the search object 4. .

  The search processing unit 56 passes the searched image (video) to the search result output unit 57 as a search result for the search condition input by the search condition input unit 54.

  The search result output unit 57 outputs the search result passed from the search processing unit 56 to the user, for example. In addition, the search result output unit 57 outputs (displays) images between images before and after the concealment occurrence image searched by the search processing unit 56, for example, to a user.

  The search option setting unit 58 sets (specifies) a condition related to the search when searching for an image constituting a video stored in the video storage unit 53, for example, according to a user operation. Conditions relating to this search include, for example, conditions relating to search speed and search accuracy. By setting the conditions related to this search, for example, the search speed of the search by the search processing unit 56 can be set.

  Next, with reference to the flowchart of FIG. 5, the processing procedure of the search process in the video search apparatus 50 will be described. Note that the metadata regarding the object 4 included in the image will be described as being generated on the video search device 50 side (the metadata generation unit 55) as described above.

  First, the receiving unit 51 receives the video (data) transmitted by the transmitting unit 14 of the image processing apparatus 10 (step S11).

  The video accumulation processing unit 52 accumulates the video received by the reception unit 51 in the video accumulation unit 53 (Step S12).

  The search condition input unit 54 inputs a search condition for searching for an image including the target object (search target object) 4 designated by the user, for example (step S13). This search condition includes, for example, metadata about the search object 4 or a sample image.

  When the search condition is input by the search condition input unit 54, the metadata generation unit 55 generates the metadata of the object 4 included in the image constituting the video stored in the video storage unit 53 (step S14). . The metadata generated by the metadata generation unit 55 is given to the image (image including the target object 4) used for generating the metadata. This metadata generation process is executed for each of the images constituting the video stored in the video storage unit 53.

  The search processing unit 56 searches for an image (video) including the search target 4 based on the search condition input by the search condition input unit 54 and the metadata generated by the metadata generation unit 55 (step S15). ).

  In the search processing, for example, based on the metadata (feature amount) included in the search condition and the metadata (feature amount) generated by the metadata generation unit 55, the similarity between the images (the object 4 included in the image) An operation is performed. Here, the feature amount is a numerical value of metadata. In other words, metadata is a feature quantity that humans can handle with words, and when the feature quantity is not treated as metadata but is used only as a numerical value, it is expressed in terms of feature quantities. The search processing unit 56 searches for an image including the search object 4 according to, for example, the calculated similarity.

  Next, the search processing unit 56 collects metadata about the search object 4 from the entire searched image (video) (step S14). The metadata related to the search object 4 is given to the searched image (image including the search object 4).

  The search processing unit 56 searches the video storage unit 53 for an image (video) including the search target 4 based on the collected metadata (step S17). Thereby, for example, the search performance can be improved by using more information (metadata) than the metadata (attribute information) regarding the search object 4 included in the search condition.

  Next, a processing procedure in the case where concealment occurs in the image searched by the video search device 50 will be described with reference to the flowchart of FIG.

  First, the process of step S21-step S25 equivalent to the process of step S11-step S15 of FIG. 5 mentioned above is performed.

  Next, the search processing unit 56 detects occurrence of concealment with respect to the search object 4 included in the searched image (step S26). At this time, the search processing unit 56 detects the occurrence of concealment based on the metadata related to the search object 4 given to the searched image. The search processing unit 56 compares the shape of the search object 4 included in the searched image with the entire image (whole body image) of the search object 4 obtained at a time before, for example, concealment occurs. Detect the occurrence of concealment. The shape or the like of the search target 4 is included in the metadata related to the search target 4 generated by the metadata generation unit 55. In addition, the search processing unit 56 detects occurrence of concealment based on luminance information or color information included in the metadata related to the search target 4. Specifically, for example, only a part of a person (search object 4) wearing a specific color clothes that was shown in the previous frame (image) of the searched image is the next frame (that is, the search). In the case where the search object 4 is visible, occurrence of concealment with respect to the search object 4 is detected.

  When the occurrence of concealment with respect to the search object 4 is detected, the search processing unit 56 searches for images (videos) before and after the image (concealment occurrence image) in which the occurrence of concealment is detected in time series. (Step S27). In this case, the search processing unit 56 searches for an image that includes the search target object 4 and that does not conceal the search target object 4.

  The search processing unit 56 acquires metadata about the search target 4 included in the images before and after the search (step S28). Since the images before and after this are images in which the search object 4 is not concealed, more metadata about the search object 4 can be acquired.

  The search processing unit 56 searches the video storage unit 53 for an image (video) including the search target 4 based on the acquired metadata about the search target 4 (step S29).

  The search result output unit 57 outputs the searched image (video) to the user as a search result for the search condition input by the search condition input unit 54 (step S30). In addition, the search result output unit 57 outputs the images between the images before and after the search in step S27 described above to the user as continuous videos. Thereby, for example, even when the information (metadata) of the search target 4 cannot be obtained from the concealment occurrence image, the search continuity can be ensured and the search performance can be improved.

  Next, a processing procedure for setting a search option will be described with reference to the flowchart of FIG.

  First, the search option setting unit 58 sets conditions (search options) relating to the search when searching for the video (the image constituting the video) stored in the video storage unit 53 according to, for example, a user operation ( Step S31). This search option includes, for example, conditions relating to search speed or search accuracy.

  Here, when the search process as described above is executed, the information used for the process differs depending on the scene (image) to be searched. Furthermore, the search speed decreases when the search accuracy is increased. Therefore, in the search option described above, a combination of a plurality of feature amounts (metadata) prepared for use in search processing and the order of information to be referred to are set according to a user operation. That is, the user can specify a search target, select information (metadata) used for search, specify a combination, specify search accuracy, and the like. Thereby, the user can designate the search speed and the search accuracy individually.

  When a search option is set by the search option setting unit 58, the search processing unit 56 executes a search process according to the search option (step S32).

  As described above, in the present embodiment, for example, when searching for an image (video) including the search target object 4, metadata about the search target object 4 is collected from the entire video stored in the video storage unit 53, By performing a search process using the collected metadata, for example, a search can be performed using more information than the information included in the search condition input by the search condition input unit 54. Thereby, search performance can be improved.

  In the present embodiment, in the search processing for the video storage unit 53, for example, the search object 4 included in the image is temporarily concealed by another object (object), and the search object is searched from an image at a certain moment. When metadata about the object 4 cannot be obtained, an image (video) at a time before the concealment of the search target object 4 occurs or an image (video) at a time after the concealment stops occurring is searched and processed. Thus, continuous search of the search object can be realized. Thereby, information (metadata) in the image can be efficiently acquired without performing an excessive search on the image (video) in which concealment has occurred. Further, by continuously outputting images between the preceding and succeeding images, continuity can be ensured and search performance can be improved.

  In the present embodiment, the user can specify a search condition (search option). As a result, for example, a double speed function is realized. Thereby, the search according to a user's intention can be performed.

  That is, as described above, in the present embodiment, it is possible to efficiently search for images (videos) taken by the plurality of cameras 1.

  In addition, by introducing a mechanism capable of processing not only stored video but also real-time video, it is possible to continuously search for people who are shown between a plurality of cameras 1 installed in a store, for example. .

[First Modification]
Next, a first modification of the present embodiment will be described with reference to FIG. The configuration of the video search system according to this modification is the same as the configuration of this embodiment described above, and will be described with reference to FIGS.

  The search processing unit 56 included in the video search device 50 according to the present modification uses the search condition based on the search condition input by the search condition input unit 54 and the metadata generated by the metadata generation unit 55, for example. Search for matching images (video). That is, the search processing unit 56 searches the video storage unit 53 for an image (video) including the search target 4 designated by the user.

  The search processing unit 56 detects occurrence of concealment (overlap) with respect to the search target object 4 included in the searched image. The search processing unit 56 detects the occurrence of concealment based on, for example, metadata related to the search object 4 given to the searched image.

  When the occurrence of concealment for the search object 4 is detected, the search processing unit 56 skips the image (the concealment occurrence image) where the occurrence of the concealment is detected. The search processing unit 56 searches for an image in which no concealment occurs with respect to the search target object 4 among the images constituting the video stored in the video storage unit 53. The search processing unit 56 acquires metadata related to the search target 4 included in an image in which no concealment occurs for the searched search target 4 (the occurrence of concealment is not detected). The search processing unit 56 searches for an image including the search target 4 based on the acquired metadata about the search target 4. That is, the search processing unit 56 executes the above-described identification (collation) process of the search target object 4.

  Next, a processing procedure of the video search device 50 according to this modification will be described with reference to the flowchart of FIG.

  First, the process of step S41-step S46 equivalent to the process of step S21-step S26 of FIG. 6 mentioned above is performed.

  Next, when the occurrence of concealment for the search target 4 is detected, the search processing unit 56 skips the image (the concealment occurrence image) where the occurrence of concealment for the search target 4 is detected (step S47).

  When the concealment occurrence image is skipped, the search processing unit 56 searches for an image in which no concealment occurs for the search target object 4 (step S48). The search processing unit 56 searches for an image from the video storage unit 53 based on the search condition input by the search condition input unit 54 and the metadata generated by the metadata generation unit 55. The image without occurrence of concealment with respect to the search target 4 is, for example, an image before or after the concealment occurrence image in time series, and an image in which the entire search target 4 (whole body) can be seen. is there.

  The search processing unit 56 acquires metadata about the search target 4 added to an image in which the search target 4 is not concealed (step S49). In this case, since the acquired metadata is metadata related to the search target object 4 in which concealment has not occurred, the search processing unit 56 compares the metadata with respect to the search target object 4 assigned to the concealment occurrence image, for example. More information (metadata).

  The search processing unit 56 searches the image including the search target object 4 by executing the identification process of the search target object 4 based on the acquired metadata (step S50).

  The image searched by the search processing unit 56 is output to the user, for example, by the search result output 57.

  As described above, in the present modification, when occurrence of concealment (overlap) with respect to the search object 4 is detected in the search process, the image (concealment occurrence image) in which the occurrence of concealment is detected is skipped. Metadata about the search object 4 attached to an image in which no concealment occurs with respect to the search object 4 (for example, an image in which the entire search object 4 before or after the concealment occurrence image can be seen) is acquired. In this modification, the identification of the search target 4 is executed based on metadata acquired from an image that does not cause concealment.

  As described above, in the present modification, by performing a search after actively searching for an image with less concealment (overlap) with respect to the search target 4, for example, a high-speed search based on skipping useless video can be realized. Is possible. Therefore, in this modification, for example, it is possible to efficiently search for videos (images) taken by a plurality of cameras 1.

[Second Modification]
Next, a second modification of the present embodiment will be described with reference to FIG. FIG. 9 is a block diagram mainly illustrating a functional configuration of the image processing apparatus 100 according to the present modification. The same parts as those in FIG. 3 described above are denoted by the same reference numerals, and detailed description thereof is omitted. Here, parts different from FIG. 3 will be mainly described.

  As shown in FIG. 9, the image processing apparatus 100 includes a correction information storage unit 101 and a color image processing unit 102.

  In the correction information storage unit 101, for example, correction information for correcting individual differences in colors (colors) caused by the characteristics of each of the plurality of cameras 1 is stored in advance. This correction information is generated using, for example, a checkerboard of a reference color for each of the plurality of cameras 1 in advance and using the captured checkerboard (image thereof).

  The color image processing unit 102 performs color image processing on the image cut out by the image cutout unit 12. The color image processing unit 102 executes processing for removing the influence of the light source on the image (target object 4) cut out by the image cutout unit 12, for example.

  The color image processing unit 102 has a function of generating the correction information described above. The color image processing unit 102 stores the generated correction information in the correction information storage unit 14.

  Based on the correction information stored in the correction information storage unit 14, the color image processing unit 102 performs color calibration that absorbs (corrects) individual differences that occur in the image cut out by the image cutout unit 12.

  Next, the color image processing procedure will be described with reference to the flowchart of FIG.

  First, the input unit 11 of the image processing apparatus 100 inputs video (data) captured by the camera 1 connected to the image processing apparatus 100 (step S51).

  The image cutout unit 12 cuts out a region (image) of the target object 4 included in the image included in the image input from the input unit 11 (step S52).

  Here, for example, a specific object (for example, a person) 4 included in an image photographed by each of the plurality of cameras 1 is searched by moving (through) a range photographed by each of the plurality of cameras 1. As metadata used in this case, for example, color information such as clothes of the specific object 4 or hair color is important. However, for example, when video (image) taken by each of the plurality of cameras 1 is simply image-processed, color information influenced by the light source is calculated according to the location where the camera 1 is installed. Is done. Thereby, for example, the color of clothes or hair of the same object (same person) is calculated as a different color between the surrounding environment, time zone, and camera, and the collation performance of the object 4 based on color information may deteriorate. .

  Therefore, the color image processing unit 102 performs color image processing on the image cut out by the image cutout unit 12 (step S53). The color image processing unit 102 removes the influence of the light source on the image (target object 4) cut out by the image cutout unit 12, for example.

  The video constituted by the image subjected to the color image processing by the color image processing unit 102 is transmitted to the video search device 50 by the transmission unit 14, for example. The video transmitted by the transmission unit 14 is stored in the video storage unit 53 of the video search device 50.

  Next, it is assumed that the video search device 50 executes a search process for the video stored in the video storage unit 53. In this case, the metadata generation unit 55 of the video search device 50 extracts the color information of the object 4 included in the image by analyzing the image constituting the video stored in the video storage unit 53 (step S54). Here, since the image to be processed is subjected to color image processing as described above, the extracted color information is not affected by, for example, a light source, and an object-specific color is calculated.

  The metadata generation unit 55 generates metadata regarding the object 4 including the extracted color information (step S55). Here, as described above, for example, the metadata (attribute information) related to the specific object 4 includes the color of clothes or hair. In this case, for example, when a plurality of pieces of metadata relating to one specific object 4 are generated, the plurality of pieces of metadata are combined into one (by coding) and handled as an information pattern.

  In the present modification, as in the above-described embodiment, it has been described that metadata is generated by the metadata generation unit 55 of the video search device 50. However, as described above, the metadata of the image processing apparatus 100 is also described. The configuration may be generated by the data generation unit 13.

  Based on the metadata generated by the metadata generation unit 55, the search processing unit 56 executes identification processing of the target object (search target object) 4 in an image photographed by each of the plurality of cameras 1, for example (step). S56).

  Next, the color calibration process will be described with reference to the flowchart of FIG.

  Each of the plurality of cameras 1 captures (captures), for example, a checker board having a reference color prepared in advance (step S61). An image (video) of the checkerboard imaged by each of the plurality of cameras 1 is input by the input unit 11 of the image processing apparatus 100.

  The color image processing unit 102 performs color image processing on the checkerboard image input by the input unit 11. As a result, the color image processing unit 102 calculates the color (information) of the checkerboard (step S62). The color information calculated here is unique color information of the checker board. The calculated unique color information of the checkerboard is stored in the correction information storage unit 101 as correction information.

  Here, for example, a case is assumed in which video captured by each of the plurality of cameras 1 is input by the input unit 11. In this case, the color image processing unit 102 performs color calibration (correction processing) on the video (the image constituting it) input by the input unit 11 based on the correction information stored in the correction information storage unit 101. Is executed (step S63). Thereby, each color of the video image | photographed with each of the some camera 1 is correct | amended to the same color at the level of a specific color.

  The color image processing unit 102 compares the colors of images (corrected images) for which color calibration has been executed among the plurality of cameras 1 (step S64).

  As a result of the comparison, the color image processing unit 102 confirms that the color error (color error) between the plurality of cameras 1 falls within, for example, a predetermined range (fixed range) (step S65). When the color image processing unit 102 confirms that the color error between the plurality of cameras 1 is within a certain range, the color calibration process is ended.

  As described above, in this modification, color image processing for removing the influence of the light source on the video (image) taken by each of the plurality of cameras 1 is performed, and the unique color of the object 4 included in the image is calculated. The search performance can be improved by executing identification (collation) processing of the target object (search target object) 4 included in videos (images) captured by different cameras 1, for example, using the main color as metadata. It becomes possible.

  In this modification, for example, based on correction information generated in advance using a reference color checker board, color calibration is performed to absorb individual differences in color caused by the characteristics of each of the plurality of cameras 1. Is done. In this way, the video captured by each of the plurality of cameras 1 is converted into a video that has been subjected to color calibration, and search performance is improved by, for example, searching for a specific target (search target) 4. Can be improved.

  Further, in this modification, for example, a plurality of metadata (attribute information) related to the target object 4 is collectively encoded and handled as an information pattern, so that the metadata can be compared in the search process for the target object 4. Calculation is possible, and identification (collation) of the same person is facilitated even in videos (images) taken by each of the plurality of cameras 1.

  As described above, in the present modification, for example, it is possible to efficiently search for videos (images) taken by a plurality of cameras 1.

[Third Modification]
Next, a third modification of the present embodiment will be described with reference to FIG. FIG. 12 is a block diagram mainly showing a functional configuration of the image processing apparatus 110 according to the present modification. The same parts as those in FIG. 9 described above are denoted by the same reference numerals and detailed description thereof is omitted. Here, a different part from FIG. 9 is described.

  As illustrated in FIG. 12, the image processing apparatus 110 includes a composite image generation unit 111. The composite image generation unit 111 collects a plurality of images of the target object 4 cut out by the image cutout unit 12. In this case, the composite image generation unit 111 collects images of the target object 4 in time series, for example. Each of the collected images includes, for example, the same object 4 from different viewpoints. Further, the collected plurality of images are images taken by the same camera 1.

  The composite image generation unit 111 generates a composite image that represents the object 4 included in the image from multiple viewpoints based on the collected images.

  Next, a processing procedure of the video search system according to this modification will be described with reference to the flowchart of FIG. Note that the processing of the color image processing unit 102 included in the image processing apparatus 110 (for example, the processing in step S53 in FIG. 10 described above) is the same as that in the second modification described above, and is omitted here for convenience. To do.

  First, the process of step S71 and step S72 equivalent to the process of step S51 and step S52 shown in FIG. 10 mentioned above is performed.

  Here, in this modification, for example, when searching for images (images) taken by a plurality of cameras 1 installed in a store, a specific object (for example, a person) 4 between the plurality of cameras 1. It is assumed that the search is tracked. In this case, the object looks different depending on the positional relationship between the camera 1 and the object 4. Therefore, it is difficult to simply collate (identify) whether the object 4 is the same among the plurality of cameras 1. That is, since a plurality of cameras 1 do not always obtain images (images) that look the same, it is necessary to collate the objects 4 that look different.

  Therefore, the composite image generation unit 111 collects a plurality of images including the target object 4 cut out by the image cutout unit 12 (step S73). The composite image generation unit 111 collects, for example, images taken by the same camera 1 among the plurality of cameras 1 in time series. That is, the composite image generation unit 111 collects images of the same object 4 from different viewpoints, for example.

  Next, the composite image generation unit 111 generates a composite image that represents the object 4 included in the image from multiple viewpoints based on the collected plurality of images (step S74). For example, when the target 4 included in the collected image is a person, the composite image generation unit 111 generates a human model image (composite image) when viewed from multiple viewpoints. Further, this composite image is generated for each camera 1 that has taken a plurality of collected images, for example.

  The composite image generated by the composite image generation unit 111 is transmitted to the video search device 50 by the transmission unit 14, for example.

  The metadata generation unit 55 of the video search device 50 analyzes the composite image transmitted by the transmission unit 14 of the image processing device 110. Based on the analysis result, the metadata generation unit 55 generates metadata related to the object 4 represented by, for example, a composite image generated for each camera 1 (step S75).

  In the present modification, as in the above-described embodiment, it has been described that metadata is generated by the metadata generation unit 55 of the video search device 50. However, as described above, the metadata of the image processing device 110 is also described. The configuration may be generated by the data generation unit 13.

  Based on the metadata generated by the metadata generation unit 55, the search processing unit 56 executes identification processing of the target object (search target object) 4 in an image photographed by each of the plurality of cameras 1, for example (step). S76).

  Here, it is assumed that the plurality of cameras 1 includes a first camera 1 and a second camera 1 different from the first camera 1. Further, for example, a composite image generated from an image (video) photographed by the first camera 1 is converted into a first composite image, for example, a composite image generated from an image (video) captured by the second camera 1. A second composite image is assumed. In this case, the search processing unit 56 analyzes the metadata (first metadata) generated by analyzing the first composite image and the metadata (second metadata) generated by analyzing the second composite image. Are compared, the object 4 represented by the first composite image and the object 4 represented by the second composite image are collated. Thereby, for example, even when the object 4 (image of the object) that looks different depending on each of the first camera 1 and the second camera 1 is photographed, the first camera 1 and the second camera 1 are captured. It becomes possible to collate the objects 4 included in the video imaged by the camera 1.

  As described above, in this modified example, a composite image (for example, a human image) representing the object 4 included in the image from multiple viewpoints is generated from the time-series images of each camera 1 and photographed by different cameras 1. The images (images included) included in the compared images (images) can be compared. For this reason, for example, even when videos that look different from each other are captured by the plurality of cameras 1, the objects 4 included in the videos can be collated. Therefore, in this modification, it is possible to improve the continuous search performance of the target object 4 photographed over the plurality of cameras 1. Thereby, in this modification, it becomes possible to search efficiently the image (video) image | photographed with the some camera 1. FIG.

  Note that the present invention is not limited to the above-described embodiment or its modifications, and can be embodied by modifying the components without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiment or its modifications. For example, you may delete a some component from all the components shown by embodiment or each modification. Furthermore, you may combine suitably the component covering different embodiment or its modification.

1 is a block diagram showing a configuration of a video search system according to an embodiment of the present invention. The figure which shows the shop floor in which the camera 1 shown in FIG. 1 is installed. FIG. 2 is a block diagram mainly showing a functional configuration of the image processing apparatus 10 shown in FIG. 1. FIG. 2 is a block diagram mainly showing a functional configuration of the video search device 50 shown in FIG. 1. 5 is a flowchart showing a processing procedure of search processing in the video search device 50. The flowchart which shows the process sequence when concealment generate | occur | produces in the image searched in the video search device. The flowchart which shows the process sequence which sets a search option. The flowchart which shows the process sequence of the video search apparatus 50 which concerns on the 1st modification of this embodiment. The block diagram which mainly shows a function structure of the image processing apparatus 100 which concerns on the 2nd modification of this embodiment. 5 is a flowchart showing a processing procedure of color image processing. 5 is a flowchart showing a color calibration processing procedure. The block diagram which mainly shows a function structure of the image processing apparatus 110 which concerns on the 3rd modification of this embodiment. The flowchart which shows the process sequence of the video | video search system which concerns on this modification.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Camera, 2 ... Store floor, 3 ... Monitoring range, 4 ... Object, 10, 100, 110 ... Image processing apparatus, 11 ... Input part, 12 ... Image clipping part, 13 ... Metadata production | generation part, 14 ... Transmission , 20 ... Network, 50 ... Video search device, 51 ... Receiving part, 52 ... Video storage processing part, 53 ... Video storage part (storage means), 54 ... Search condition input part, 55 ... Metadata generation part, 56 ... Search processing unit, 57... Search result output unit, 58... Search option setting unit, 101... Correction information storage unit, 102.

Claims (2)

A plurality of cameras that are installed within a certain range and shoot a video composed of an image including an object existing within the range;
Receiving means for receiving images taken by the plurality of cameras;
Storage means for storing the received video;
Search condition input means for inputting a search condition for searching for an image including a search target specified by a user among images constituting the video stored in the storage means;
Analyzing means for analyzing an image constituting the video stored in the storage means;
Metadata generation means for generating metadata indicating attributes characterizing the object included in the analyzed image;
First search means for searching the storage means for an image including the search object based on the input search condition and the generated metadata;
Concealment occurrence detection means for detecting occurrence of concealment for the search object included in the image searched by the first search means based on the generated metadata;
When the occurrence of concealment is detected, among the images constituting the video stored in the storage means, an image that does not cause concealment with respect to the search object is selected as the input search condition and the generated meta data. A second search means for searching based on the data;
Obtaining means for obtaining metadata indicating an attribute characterizing the search object included in the image searched by the second search means;
A video search system comprising: third search means for searching the storage means for an image including the search object based on the acquired metadata. Receiving means for receiving a video composed of an image including an object present in the range, which is taken by a plurality of cameras installed in a certain range;
Storage means for storing the received video;
Search condition input means for inputting a search condition for searching for an image including a search target specified by a user among images constituting the video stored in the storage means;
Analyzing means for analyzing an image constituting the video stored in the storage means;
Metadata generation means for generating metadata indicating attributes characterizing the object included in the analyzed image;
First search means for searching the storage means for an image including the search object based on the input search condition and the generated metadata;
Concealment occurrence detection means for detecting occurrence of concealment for the search object included in the image searched by the first search means based on the generated metadata;
When the occurrence of concealment is detected, among the images constituting the video stored in the storage means, an image that does not cause concealment with respect to the search object is selected as the input search condition and the generated meta data. A second search means for searching based on the data;
Obtaining means for obtaining metadata indicating an attribute characterizing the search object included in the image searched by the second search means;
3. A video search apparatus comprising: a third search unit that searches the storage unit for an image including the search object based on the acquired metadata.

Images