JP2007087150A - Image reproduction method, machine, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To acquire a stable result without respect to search precision, which is reduced when it is greatly affected by influence of fluctuation in direction of a face and lighting in determination of a distance of face feature information, in person search using the face feature information. <P>SOLUTION: This image reproduction machine is provided with a predetermined image extraction part 102 extracting a predetermined image inside a picture, an identifier generation part 113 generating a predetermined identifier to be attached to the predetermined image inside a second picture, which is extracted by the predetermined image extraction part and satisfies a predetermined condition in relationship with the first predetermined image inside the first picture extracted by the predetermined image extraction part, and an authentication image flag generation part 1191 generating a predetermined flag to be attached to the first predetermined image inside the first picture and one of the predetermined image inside the second picture according to the predetermined identifier. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a person search method and an action locus correction method for reproducing an action locus of a target person as a video.

  Conventionally, in order to reproduce the entire video of the monitoring target person, the feature information of the person shown in the monitoring video is extracted, and the feature information of the person is stored in a database together with the image.

  Then, using the person characteristic information of a specific monitoring target person to be searched, the above database is searched, images searched as the same person are arranged in time series, and this is reproduced as time series video of the specific monitoring target person. It was done.

In the person search in the conventional technique, the person search is performed by the face image search using the face feature information which is the feature of the face image stored in the database (Patent Document 1).
JP 2002-183205 A

  However, the conventional person search has the following problems.

  In human search using facial feature information, the determination of the proximity of facial feature information is greatly affected by face orientation and lighting fluctuations, and the search accuracy is lowered and stable results cannot be obtained. is there.

  The invention of the present application solves the above-described problems, and assigns an identifier to an image satisfying a certain condition, manages the image, and replaces the image according to the identifier, so that the person who is a search target can be surely obtained. The present invention relates to an image reproducing method, an image reproducing device, and the like that can reproduce a series video.

  The present invention is an image that satisfies a predetermined condition in a relationship between a predetermined image extracting unit that extracts a predetermined image in a picture and a first predetermined image in a first picture extracted by the predetermined image extracting unit. An identifier generating unit that generates a predetermined identifier to be attached to a predetermined image in a second picture extracted by the predetermined image extracting unit; and a first predetermined image in the first picture according to the predetermined identifier; And a legitimate image flag generation unit that generates a predetermined flag attached to one of the predetermined images in the second picture.

  As a result, it is possible to manage by assigning an identifier to an image that may be a search target, and to replace the image according to the identifier.

  Since the image playback device of the present invention can reduce the number of images to be searched, the amount of calculation can be reduced.

  In the first embodiment or the second embodiment, an image playback device, an image playback method, and an image playback program according to the present invention are described.

  The present invention is an image reproducing method and an image reproducing apparatus that manages an image (for example, a person image) that satisfies a certain condition by assigning an identifier and replacing the image according to the identifier.

  As a result, there is an error that an image other than the search target is mixed in the middle of the time-series image of the search target arranged in time series, which is recognized as a search target using arbitrary image recognition. Even at times, it is possible to reliably obtain a time-series image of a correct search object.

  In the first embodiment, when an image is recognized as an object to be searched using arbitrary image recognition, and after that, images other than the object to be searched are mixed in the middle of the time-series images of the objects to be searched arranged in time series. The image player is described.

  Thereby, it is possible to reliably obtain a time-series image of a correct search object.

  In the second embodiment, an update method of an identifier attached to the target image will be described for the image playback device described in the first embodiment.

  Thereby, when searching for the time series image regarding the same target object next, the calculation amount and the calculation time can be reduced.

  The invention described in the present embodiment arranges images recognized as search objects using arbitrary image recognition in time series, and images other than the search object are mixed in the middle of this time series. The present invention relates to an image player or the like.

  For example, as an example of the present embodiment, in order to track an untrustworthy person at a place where a plurality of surveillance cameras are installed, a playback machine that plays back images of the target person in time series can be considered.

  Since the above example is an example, the image reproducing device of the present invention can be used in a situation where a time-series image of a search target is acquired without being restricted thereto.

  Therefore, the present invention can be implemented as long as it has a playback device such as a television, a desktop video, a DVD (Digital Versatile Disc), a digital camera, a video camera, a mobile device, and a mobile phone that can play back images.

  In the present embodiment, an image reproducing device related to an image taken by a surveillance camera will be described below with reference to FIGS.

  The search used in the specification of the present application is used in the meaning of searching for a monitoring target reflected in a monitoring camera from a storage unit in which an image is stored.

  FIG. 1 is a schematic block diagram showing the overall configuration of an image playback device 100 according to the first embodiment of the present invention.

  In FIG. 1, an image playback device 100 includes a monitoring camera 101, a person feature extraction unit 102 that extracts person feature information from a monitoring video input from the monitoring camera 101, and a person information storage unit 106 that stores person feature information. A video storage unit 107 for accumulating surveillance video, a person search unit 114 for performing a person search using the surveillance video and the person feature information, a display unit 111, and a user search request and locus correction information are input. And an identifier generating unit 113 that generates an identifier attached to an image that can be a search target.

  The image photographed by the surveillance camera is, for example, a JPEG (Joint Photographic Experts Group) format image.

  Images captured by the surveillance camera include still images in a compressed form such as JPEG and JPEG2000, still images in an uncompressed form (for example, pictures, frames, etc.), and moving images in a compressed form such as MPEG1, 2, 4 An uncompressed moving image may be used. Furthermore, a still image or a moving image of a unique format may be used.

  Hereinafter, these are collectively referred to as an image.

  The person feature extracting unit 102 includes a moving region extracting unit 103 that extracts a moving region in the monitoring video, an action locus extracting unit 104 that creates a person's locus based on the moving region information, and a face for extracting facial features. And a feature extraction unit 105.

  The person search unit 114 needs to reconstruct the video reproduction unit 108 for reproducing the monitoring video accumulated in the image storage unit 107 and the video reproduced by the video reproduction unit 108 based on information from the input unit. Based on the determination result of the video reconstruction determination unit 110 that determines whether or not there is an image and the identifier generated by the identifier generation unit 113, the image to be reproduced is reconstructed based on the determination result of the video reconstruction determination unit 110 And a video reconstruction unit 109.

  Note that the configuration, name, and the like of the person search unit 114 are merely examples, and are not limited to the above configuration as long as they exhibit the same operations and effects as the components of the person search unit 114.

  Further, in this embodiment, “video” is a continuous reproduction of the “image” as described above. In the following description, the term “video” is used in the above definition, unless otherwise specified.

  In the image reproducing apparatus 100 configured as described above, first, a user interface displayed on the display unit 111 will be described with reference to FIGS.

  Reference numeral 301 in FIG. 3 represents a monitoring video reproduction screen displayed by the display unit 111.

  The display unit 111 also serves as a user interface for searching for people.

  In FIG. 3, what is indicated by 302 is a person who appears in the monitoring video, and is a person whose feature information has been extracted by the person feature extraction unit 102.

  In FIG. 3, reference numeral 303 denotes a pointing device that can be operated with a mouse or the like for selecting a person to be searched.

  In the above configuration, the pointing device 303 is operated by the input unit 112.

  When someone pointing to the person 302 displayed on the display unit 111 is selected by the pointing device 303, a person search operation is performed.

  Since the above operation is an example, for example, when a person of the person in the display unit 111 is selected, a message such as “Do you want to search for a person?” Is displayed, and only when you agree with this, the operation moves to the person search operation. You may do it.

  Hereinafter, the operation of the image player when searching for a person selected by the pointing device 303 will be described with reference to FIG.

  In FIG. 4, 401 represents an image correction screen displayed on the display unit 111.

  Reference numeral 402 denotes an image of a search target person selected by the pointing device 303.

  The time when the image was taken is displayed above the image 402 of the person to be searched.

  Here, the search target person image 420 does not necessarily have to be the one specified by the pointing device 303, and may be any image as long as it is the search target person image.

  Reference numeral 404 denotes a time axis of time when the search target person is photographed by the surveillance camera.

  The area indicated by the mesh pattern on the time axis represents a place where the search target person may be replaced with another person during the time from visiting the store to leaving the store.

  This possibility of switching may occur when a plurality of persons or objects cross, approach, hide, or the like from a search target person.

  As shown in FIG. 4, both ends of the time axis 404 indicate images when it is determined that the image player has visited or left the corresponding search target person. Time is shown.

  Reference numeral 403 indicates the meaning of the mark on the time axis 404 in the user interface indicated by 401.

  406 indicates a list of persons that may be misidentified as a search target person after the time of the mesh pattern due to a mesh pattern time in the time axis 404, including the search target person. Represents.

  The monitor inputs whether or not there is a search target person in each line in the person list 406 by selecting a person image with a pointing device or the like (not shown).

  The pointing device is operated by the input unit 112 having the configuration of the present embodiment.

  Reference numeral 407 denotes a re-search button. By selecting an image of a target person to be searched by the supervisor from the person list 406 from the person list 406 through the input unit 112, the user presses the re-search button 407. Video reconstruction is performed.

  At this time, the video reconstruction determination unit 110 in the image player 100 determines whether or not a signal for selecting a search target person in the person list 406 is input from the input unit 112.

  As a result of the determination, if a signal for selecting a search target person is input, the video reconstruction unit 109 reconstructs the video. This video reconstruction will be described later.

  Regarding the determination method, any determination method that is commonly applied by those skilled in the art may be used.

  In this embodiment, the video reconstruction determination unit 110 determines whether or not the search target person in the person list 406 has been selected from the input unit 112 by detecting the input signal. Without being constrained, for example, based on data in a database in which past search target person selection results are stored, the image player may automatically determine without input.

  Reference numeral 408 denotes a person video playback button. By pressing the person video playback button 408, it is possible to play back all videos of the search target person.

  Here, description will be given of how the replacement of the search target indicated by the mesh pattern on the time axis 404 in FIG. 4 described above, and the reliability (identifier) of the image attached at this time will be described with reference to FIG. I will explain.

  In FIG. 5, reference numeral 501 schematically represents person information in the present invention.

  In this case, as shown in FIG. 5, Mr. A and Mr. B are photographed by the camera.

  FIG. 5A shows an image at time t, and FIG. 5B shows an image at time t + 2, which is an image after Mr. A and Mr. B intersect.

  The image at time t + 2 is an example, and an image at another time may be used as long as it is an image after A and B are switched.

  Reference numerals 502 to 505 schematically represent specific individuals that can be extracted by inter-frame differences or the like.

  In FIG. 5, 502 and 505 schematically represent Mr. A, and 503 and 504 schematically represent Mr. B.

  506 indicates that Mr. A and Mr. B crossed. As a result, it may become difficult to know whether Mr. A of 502 is Mr. A's image 505 or Mr. B's image 504. There is.

  This is because in the person search using the face feature information, the determination of the proximity of the face feature information is greatly affected by the face orientation and the illumination variation.

  In such a case, it is possible to set the reliability high in relation to the image of Mr. A immediately before intersecting the image recognized as being closest to Mr. A among the images immediately after the intersection, and other replacements are possible. The reliability is set to be low in relation to the image of Mr. A immediately before crossing the image of the other party (Mr. B image in this case), and is stored in the person information storage unit 106 as person characteristic information.

  The recognition method described above may be any recognition method that allows a person skilled in the art to recognize a person such as face recognition that is normally used for image recognition.

  The operation of the image playback device 100 configured as described above will be described below with reference to FIGS.

  Here, let us consider an image playback device that plays back images from a plurality of surveillance cameras that shoot from a customer's visit to a store exit in a certain store.

  It is assumed that the plurality of cameras do not shoot a place where they overlap each other, and that the inside of the store monitored by the plurality of cameras is photographed thoroughly at any time by any camera.

  Note that there is no problem in the present invention even if an overlapping place is photographed by a plurality of cameras.

  In addition, a single camera may be used instead of a plurality of cameras as long as the inside of the store can be photographed without any problems.

  Although not shown in the block diagram of the invention of the present embodiment shown in FIG. 1, the following overall operations are performed by the control unit, and optimal control is performed by this control unit. It shall be.

  The control unit (not shown) that performs the above optimal control performs control that can be naturally applied in carrying out the present embodiment, among the control methods that are usually applied by those skilled in the art in consideration of common technical knowledge. It is a control unit.

  First, in step 201 in FIG. 2, an image is taken by the monitoring camera 101.

  In step 202, the monitoring video imaged by the monitoring camera 101 in step 201 is input to the person feature extraction unit 102.

  The person feature extraction performed in step 202 is performed in the following steps 203, 204, 205, and 218.

  In step 203, first, the moving region extraction unit 103 extracts a moving region as a place where a person exists by a technique such as inter-frame difference or background difference.

  In step 204, the action trajectory extraction unit 104 extracts the trajectory of the same person from the elapsed time and the proximity of the motion area with respect to the motion area obtained by the motion area extraction unit 103.

  In step 205, the face feature extraction unit 105 searches for a human face region with respect to the motion region obtained by the motion region extraction unit 103, and extracts face feature information for the obtained face region. To do.

  In step 218, the identifier generation unit 113 associates the person with the trajectory information based on the information in the above step 202, the connection relationship between the trajectories, and if there is a possibility that the trajectories are different, The connection partner of the possibility of connection and the certainty of the connection relation between the trajectories are generated as reliability (identifier).

  Specifically, the reliability is generated by the identifier generation unit 113 and attached to the corresponding image as shown in FIG.

  FIG. 16 shows an example of reliability in the case of FIG.

  For example, after Mr. A and Mr. B intersect, if the person image 505 is recognized as Mr. A by image recognition or the like, the person image 505 is set to high reliability.

  On the other hand, the person image 504 that may be replaced with the Mr. A image is set to low reliability.

  In step 206, information such as reliability as shown in FIG. 16 is accumulated in the person information storage unit 106 as person feature information.

  In addition, person images such as the person image 502 to the person image 505 shown in FIG. 5 are stored in the video storage unit 107.

  In the present embodiment, the storage units are explicitly divided as the person information storage unit 106 and the video storage unit 107, but one storage unit may be used without division.

  In step 207, an end determination is made. If the end is not completed (Yes in S207), the process returns to the monitoring video capturing step (S201), and then the steps described above are sequentially performed.

  These processes are basically continued unless the observer ends, but such processes are performed on all the images obtained from the surveillance camera in consideration of the resources of the image player. Also good.

  Each block 102 is shown in FIG. 1 in the present embodiment, but is not limited to the block of the present application, and any block that can obtain the same operation, action, and effect as the human feature extraction unit 102 of the present embodiment. Other configurations that can be considered by those skilled in the art may be used.

  The information to which the identifier is added by the identifier generation unit 113 includes the trajectory information, the certainty (reliability) of the connection relationship between the trajectories, target person information whose trajectory may be changed depending on the processing accuracy, and the trajectory. It suffices to have information that can be calculated by calculating whether there is a match / mismatch between them, or whether they are similar or not.

  Next, a person search operation performed by the video player in FIG. 1 for the video imaged as described above will be described below with reference to FIG.

  First, in step 208, the video playback unit 108 in the person search unit 114 obtains a monitoring video stored in the video storage unit 107.

  In step 209, the display unit 111 displays the video reproduced by the video reproduction unit 108 as shown in FIG.

  In step 210, a search target person is selected via the input unit 112.

  At this time, as described above, the search target person is selected by the pointing device 303 of FIG.

  In step 211, when a search target person is selected in step 210, a control unit (not shown) focuses on a predetermined time before and after the image shooting time of the search target person when the search target person is selected. Then, a person who is likely to be replaced with the person to be searched is found from the person information storage unit 106.

  Here, the person who may be replaced with the person who wants to search is a person to whom reliability (identifier) is given by the identifier generation unit 113.

  In step 212, a control unit (not shown) creates and displays a trajectory correction screen as shown in FIG.

  The flow of creating the locus correction screen will be described in detail later.

  In step 213, the user selects a search target person from the person list 406 in FIG. 4 via the input unit 112.

  In step 214, one or more search target images among the person images appearing in FIG. 4 are selected from the person list 406.

  This person list 406 is an image of images that are currently recognized as the same person by image recognition or the like on the horizontal axis.

  For example, the person list 406 includes four person lists 406 in which images of persons who may be replaced with the selected person 402 are arranged in the downward direction on the screen. The uppermost image 406 is recognized as A and the second image from the uppermost is recognized as Mr.B.

  In step 217, when the re-search button 407 in FIG. 4 is selected (YES in s214), the video reconstruction unit 109 corrects the video.

  This video correction will be described in detail below.

  In this case, if the video reconstruction determination unit 110 determines that no search target person is selected from the person list 406 shown in FIG.

  Thereafter, in the video reconstruction unit 109, a correction screen is created again (S212) and displayed, and the steps of S212, S213, S214, and S217 are sequentially executed.

  At this time, the time range displayed in FIG. 4 may be gradually expanded to be operated.

  Thereafter, in step 216, when the person playback button 408 in FIG. 4 is selected, the video of all the searched trajectories is played back as the search target person, and the process ends.

  Next, the processing content of step 217 performed in the video reconstruction unit 109 will be described.

  First, FIG. 6 schematically shows video reconstruction. In FIG. 6, the locus of Mr. A is indicated by 609.

  Reference numerals 601 and 602 indicate respective images of Mr. A and Mr. B before Mr. A and Mr. B intersect, for example.

  Reference numerals 603 and 604 denote images of Mr. A and Mr. B after Mr. A and Mr. B intersect, for example.

  In the image reproducing device of the present invention, as shown at 609, for example, after Mr. A and Mr. B intersect, the identifier generating unit 113 determines the reliability (hereinafter referred to as Mr. A). The identifier generated by the identifier generation unit 113 is referred to as “reliability”).

  The reliability is the certainty of whether or not the person is a search target person, and is expressed by binary values of high (1) and low (0), for example.

  For this reason, for example, a certain degree of reliability is given to a person image that may cross with Mr. A (for example, Mr. B image in FIG. 6).

  In the case of FIG. 6, for example, after Mr. A and Mr. B intersect, the image reproducing device of the present invention recognizes Mr. B's image 603 as Mr. A's image.

  Then, Mr. B's image 603 is associated with high reliability.

  Further, for example, the Mr. A image 604 after crossing is not recognized as the Mr. A image, but is given some reliability (for example, low reliability) as described above.

  As a recognition method for recognizing Mr. A out of Mr. A's image 603 and Mr. B's image 604, those skilled in the art naturally apply in addition to face recognition by extracting and recognizing a facial feature. Any method may be used as long as it is a recognition method.

  In addition, although not shown in the figure, there is an extraction unit that extracts a person who may cross Mr. A from the motion vector, etc. An extraction unit that extracts persons in the range of m as persons who intersect with Mr. A can be considered.

  Furthermore, an extraction unit that combines the above extraction methods and extracts a person who crosses Mr. A can be considered.

  Even if it is not the said extraction method, regarding this person extraction method, if it is an extraction part which performs the person extraction method applied naturally by those skilled in the art, what kind of extraction part may be used.

  Next, the processing operation of step 217 performed in the video reconstruction unit 109 will be described more specifically with reference to FIGS. 7 to 13, 19, and 20.

  It should be noted that the solid line connecting the person images indicated by the alphabets (A, B, C, etc.) from FIG. 8 to FIG. 13 is the person image connected by the solid line is the previous person image (in FIG. 6, The image 601) with respect to the image 603 indicates an image with the highest reliability, and the broken line indicates an image with a low reliability with respect to the previous person image.

  8 to 13 show images of Mr. A, Mr. B, and Mr. C taken by the camera in time series. In the specification including this embodiment, “trajectory” is used. The expression is used.

  First, the components of the video reconstruction unit 109 shown in FIG. 19 will be described.

  The inter-image connection relationship determination unit 1192 determines whether there is any unrelated relationship between images in the trajectory of the search target person.

  For example, in FIG. 10, the upper image of 1001, the middle image of Mr. A's trajectory and the rightmost image are not connected by a solid line or a dotted line.

  For this reason, the inter-image connection relationship determination unit 1192 determines that there is no relationship between the two images.

  The above is a schematic explanation. Actually, the inter-image connection relation determination unit determines whether there is any relationship between adjacent time-series images based on the stored data as shown in FIG. To do.

  The image determined as described above is actually provided with a valid image flag by the valid image flag generation unit 1991 and managed as shown in FIG.

  In this specification, “replace the image” means that the legitimate image flag generation unit 1991 actually adds a flag to the image after a predetermined determination is made. The drawings schematically show how images are exchanged.

  For example, as shown in FIG. 20A, the locus of Mr. A and Mr. B in FIG. 6 is determined.

  By the image reproducing apparatus of the present embodiment, the Mr. A trajectory as shown in FIG. 6 is changed from the trajectory composed of the Mr. A image 601 and the Mr. B image 603 to the trajectory composed of the Mr. A image 601 and the Mr. A image 604. Is replaced.

  At this time, as shown in FIG. 20B, a legitimate image flag is attached to the image in the locus of Mr. A. In FIG. 20B, it is assumed that Mr. A's image 601 and Mr. A's image 604 have Mr. A's legitimate image flag.

  On the other hand, Mr. A's legitimate image flag is not included in Mr. B's image 603.

  In FIG. 20 (b), only the flag is provided and the flag is not shown, but in actuality, a distinction is made between Mr. A's legitimate image flag and Mr. B's legitimate image flag.

  Next, the operation of the video reconstruction unit 109 will be described.

  As shown in FIG. 9, three persons, Mr. A, Mr. B, and Mr. C, were extracted by an extraction unit (not shown) that extracts a person who may intersect with the search target person A. In this case, the user inputs two images indicated by reference numeral 901 in FIG.

  The input signal is stored in the input signal storage unit 1993 shown in FIG.

  The above description using FIG. 9 is a schematic diagram. In the actual user interface, as shown in FIG. 4, each locus (trajectory connected with high reliability) is represented by a representative still image. .

  Therefore, the act of inputting two trajectories 901 in FIG. 9 corresponds to an operation of selecting a correct search target person (Mr. A in FIG. 9) in the person list 406.

  For example, this is not an appropriate example because the arrangement of persons is different from that in FIG. 9, but in FIG. 4, the upper image of the second person list 406 from the left and the fourth person list 406 from the left on the screen. It feels like choosing a third-tier image.

  When the search target person is selected as described above, the video reconstruction determination unit 110 determines to reconstruct the video.

  In step 701 of FIG. 7, the selected image is replaced with the image of Mr. C as the image of Mr. A as shown in FIG.

  In FIG. 9, as indicated by an arrow 902, the upper right Mr. C image showing Mr. A's trajectory and the lower right Mr. A image showing Mr. C's trajectory are interchanged.

  In step 702, the inter-image connection relationship determination unit 1992 determines whether or not the images before and after the reconstructed trajectory are connected to images of different persons that are not connected with reliability.

  As a result of the replacement by the video reconstruction unit 109 as described above, as shown in FIG. 10, there is no connection relationship between Mr. B and Mr. C as indicated by 1003.

  Here, there is no connection relationship between the image of Mr. B (left) and the image of Mr. C (right) that are recognized as Mr. B in the second-stage B's locus in FIG. It means that there is no connection as shown.

  In other words, the fact that there is no connection due to some degree of reliability means that Mr. C's image (right) in Mr. B's trajectory in Fig. 8 has nothing to do with the B's image of Mr. B's trajectory arranged (left). Means that.

  Even though the connection relationship by reliability is not between the center C image and the right A image of Mr. A's trajectory in the upper part of FIG. Yes (YES in S702).

  In addition, as indicated by 1002, there is a similar contradiction between the image of the center B of Mr. C and the image of Mr. C at the right end (YES in S702).

  In order to eliminate this, in step 703, as indicated by an arrow 1004 in FIG. 10, the center C image of the locus of A and the center B image of the locus of C are switched.

  As a result, as shown in FIG. 11, as shown in 1103, there is a gap between the image of Mr. A at the left end of Mr. A's trajectory and the image of Mr. B at the center of the trajectory of Mr. A. Even though there is no connection relationship, the trajectory is the same person, and there is a contradiction (YES in S702).

  Similarly, as shown in 1102, there is a contradiction between the image of B at the left end of the locus of C shown in the lower part of FIG. 11 and the image of Mr. C at the center of the locus of Mr. C (S702). YES)

  In order to solve this problem, in step 703, as shown at 1104, the left C image of the B's locus shown in the middle of FIG. 11 and the left B image of the C's locus shown in the lower row are switched.

  As a result, as shown in FIG. 12, as shown in 1202, between the A image at the left end of the trajectory of A in the upper part of FIG. Even though there is no connection relationship, the trajectory is the same person, and a contradiction remains (YES in S702).

  In order to solve this problem, in step 703, as shown at 1202, the left B image of the A's locus in the upper part of FIG. 12 and the A's image at the center of the A's locus are switched.

  As a result, as shown in FIG. 12, the trajectories of Mr. A, Mr. B, and Mr. C all settle down to have some connection relationship, and the processing by the video reconstruction unit 109 ends.

  Thereafter, a legitimate image flag generation unit 1991 adds a legitimate image flag to the finally settled image, and the person information storage unit 106 manages the image.

  FIG. 17 and FIG. 18 show what is visually achieved by the operation of the image reproducing apparatus having the above configuration.

  The six screens shown in FIG. 17 and FIG. 18 show images that are reproduced in time series in the numerical order given to the upper left of each screen.

  A plurality of persons are photographed in each screen, and a person surrounded by a thick frame is a monitoring target person.

  In this example, the person to be monitored is surrounded by a thick frame, but without being restricted by this, for example, by marking the upper part of the person to be monitored with an arrow or the like, it can be distinguished from other persons. good.

  For example, Mr. A in FIG. 17 is selected with a pointing device or the like.

  Thereafter, the video of the person recognized by the image player as being Mr. A selected by the pointing device or the like is reproduced.

  FIG. 17 corresponds to this, but in the screen 2 of FIG. 17, when Mr. A overcame Mr. B on the way, the video player misrecognizes Mr. B as Mr. A. It is a video.

  In order to acquire the correct Mr. A video, the above-described operation in the image reproducing apparatus described in the present embodiment is performed.

  Accordingly, the correct Mr. A video as shown in FIG. 18 can be reproduced.

  With the image playback device having the above-described configuration, it is possible to manage an image that may be a search target with an identifier.

  As a result, in the person search using the face feature information, when the proximity of the face feature information is determined, a completely different person is determined as the search target person due to the influence of the face direction and lighting fluctuation. However, since it can be replaced with an image that may be a search target with an identifier as described above, it is possible to suppress a decrease in search accuracy of the search target person, and to obtain a stable search result. Obtainable.

  Further, as described above, since an image that may be a search target is managed with an identifier, when a person search using face feature information determines that a completely different person is a search target person. Even if it is, it is not necessary to search all the stored images, and it is only necessary to search for an image of a person who may be a search target with an identifier. Images can be reduced.

  Thereby, the calculation amount in the image reproducing device can be reduced.

  In addition, since the invention described in the present embodiment can reduce the amount of calculation of the image player, it is more effective for terminals such as mobile phones and mobile devices where the physical memory capacity is limited compared to other devices. It is.

  In addition, as described above, there is an effect that the locus other than the search target person is also corrected.

  Note that the video reconstructing unit 109 may store correct trajectory information for each person, and recalculate the connection relationship of the trajectory of the monitoring target person in the trajectory correction step S217.

  Since there are a plurality of pieces of correct person characteristic information related to the monitoring target person, the accuracy is improved. In addition, there is an effect of improving the accuracy of connection relations other than the time range indicated by the time axis 404 in FIG.

  Similarly, trajectory information that has not been selected as a search target person may be held as incorrect answer information of the monitoring target person.

  Thereby, the connection relation of the trajectory including the person who is not the search target can be recalculated, and the accuracy is improved.

  Further, in this embodiment, the reliability is high and low. For example, the one having a high recognition rate by image recognition is set as a high reliability, and among a plurality of moving objects between adjacent frames, It can be considered that the reliability is set for those that may be close or hidden due to approach or hiding, and the other is set to be low in reliability.

  Since the reliability of the present embodiment is an exemplification, other identifiers may be used as long as they are identifiers for identifying an image without being restricted thereto.

  In the present embodiment, the image reproducing device has been described. However, an image reproducing method capable of obtaining the effects and effects achieved by the cooperation of the components of the image reproducing device described in the present embodiment is also described. I can think.

  In addition, a program that can obtain the functions and effects achieved by the cooperation of the components of the image playback device described in the present embodiment by being incorporated in a conventional general image playback device can be considered. .

  Also, an integrated circuit that can obtain the functions and effects achieved by the cooperation of the components of the image player described in this embodiment by controlling a conventional general image player. I can think.

  In the present embodiment, the image reproducing apparatus for the search target person has been described. However, the image reproducing apparatus of the present embodiment is also used when the search target is an image of an object or another living body, without being limited to a person. Can do.

  In this embodiment, a method for updating an identifier attached to a target image for the image playback device described in the first embodiment will be described.

  As in the first embodiment, the present invention is implemented as long as it has a playback device such as a TV, a desktop video, a DVD (Digital Versatile Disc), a digital camera, a video camera, a mobile device, and a mobile phone that can play back images. can do.

  Since the method other than the method for updating the identifier is the same as that in the first embodiment, it will not be described in detail here.

  In the present embodiment, an image playback device related to an image taken by a surveillance camera will be described below with reference to FIGS.

  As described in the first embodiment, the image reconstruction unit 109 performs image replacement based on the connection relationship based on the reliability, and performs image replacement until there is no contradiction in the connection relationship. This corresponds to 1501 in FIG.

  As a result of the replacement, for example, an image connected by a low-reliability connection as indicated by a dotted line in the upper A's locus of 1501 is Mr. A's trajectory.

  At this time, as shown in 1502, the connection of the image regarded as the locus of Mr. A is updated to a highly reliable connection (solid line) by the identifier replacement unit 1415 shown in FIG. , Updated to the connection (dotted line) in the reliability.

  Actually, data as shown in FIG. 16 is updated.

  Thereby, when searching for the time series image regarding the same target object next, the calculation amount and the calculation time can be reduced.

  In the present embodiment, the image reproducing device has been described. However, an image reproducing method capable of obtaining the effects and effects achieved by the cooperation of the components of the image reproducing device described in the present embodiment is also described. I can think.

  In addition, a program that can obtain the functions and effects achieved by the cooperation of the components of the image playback device described in the present embodiment by being incorporated in a conventional general image playback device can be considered. .

  Also, an integrated circuit that can obtain the functions and effects achieved by the cooperation of the components of the image player described in this embodiment by controlling a conventional general image player. I can think.

  The configuration of the image reproducing apparatus of the present invention shown in the first embodiment or the second embodiment is an example, and the present invention is not limited to the above-described embodiment, and departs from the scope of the present invention. Various modifications or corrections are possible.

  According to the present invention, it is possible to provide an image playback device that can reduce the number of images to be searched in video reconstruction.

Configuration diagram of an image playback device described in the first embodiment (A) Flow chart of video image capturing operation of the image player described in the first embodiment (b) Flow chart of video reconstruction operation of the image player described in the first embodiment User interface diagram of the image player described in the first embodiment (1) User interface diagram of the image player described in the first embodiment (2) (A) Schematic diagram before replacement of search target person described in the first embodiment (b) Schematic diagram after replacement of search target person described in the first embodiment Figure (1) of human trajectory described in the first embodiment Operational diagram of video reconstruction described in the first embodiment Figure (2) of human locus described in the first embodiment Person locus replacement diagram described in the first embodiment (1) Person locus replacement diagram described in the first embodiment (2) Person locus replacement diagram described in the first embodiment (3) Person locus replacement diagram described in the first embodiment (4) Person locus replacement diagram described in the first embodiment (5) Configuration diagram of an image playback device described in the second embodiment Person locus reliability update diagram described in the second embodiment (1) Correspondence diagram between image and reliability described in the first embodiment Playback video diagram before video reconstruction by the image playback device described in the first embodiment Playback video diagram after video reconstruction by the image playback device described in the first embodiment Configuration diagram of a video reconstruction unit in the image playback device described in the first embodiment (A) Correspondence diagram between images and reliability described in the first embodiment (b) Correspondence diagram between images described in the first embodiment and valid image flags

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Surveillance camera 102 Person feature extraction part 103 Motion area extraction part 104 Action locus extraction part 105 Face feature extraction part 106 Person information storage part 107 Video storage part 108 Video reproduction part 109 Video reconstruction part 110 Video reconstruction determination part 111 Display part 112 Input unit 113 Identifier generation unit 114 Person search unit

Claims (11)

A predetermined image extraction unit for extracting a predetermined image in a picture;
A predetermined image in the second picture extracted by the predetermined image extraction unit, which is an image satisfying a predetermined condition in relation to the first predetermined image in the first picture extracted by the predetermined image extraction unit. An identifier generating unit that generates a predetermined identifier to be attached to,
A legitimate image flag generating unit that generates a predetermined flag attached to a first predetermined image in the first picture and one of the predetermined images in the second picture according to the predetermined identifier;
An image playback device comprising: The legitimate image flag generation unit attaches the predetermined flag to an image with the predetermined identifier having the largest value indicated by the predetermined identifier among predetermined images in the second picture. The image reproducing machine according to 1. And a video reconstruction determination unit that determines whether or not a first flag image, which is an image to which the predetermined flag is added, among predetermined images in the second picture satisfies a first predetermined condition. ,
When the video reconstruction determination unit determines that the first flag image does not satisfy the first predetermined condition,
The legitimate image flag generation unit replaces the first flag image with an image satisfying the first predetermined condition, and adds a second flag image that is one image of the predetermined image in the second picture. The image reproducing device according to claim 1 or 2, wherein a predetermined flag to be attached is generated. The information processing apparatus further includes an identifier replacement unit that replaces a value indicated by a first identifier attached to the first flag image with a value indicated by a second identifier attached to the second flag image. 3. The image reproducing machine according to 3. In the first predetermined condition, it is recognized that a first flag image in the first picture and a second flag image that is one image in the second picture are the same object image. The image reproducing apparatus according to claim 3 or 4, wherein the condition is as follows. The image playback device according to claim 3, further comprising an input unit to which the first predetermined condition is input. And an inter-image connection relationship determining unit that determines whether or not the predetermined identifier is attached to an image in the second picture,
When the inter-image connection relation determining unit determines that the predetermined identifier is not attached to a third flag image that is an image in the second picture to which the predetermined flag is attached,
The legitimate image flag generation unit replaces the third flag image with a fourth flag image in the second picture, which is an image with a second predetermined identifier generated by the identifier generation unit. The image reproducing device according to claim 1, wherein a predetermined flag attached to the image generating device is generated. The image playback device according to any one of claims 1 to 7, wherein the second picture is displayed at a time later than the first picture. When the second picture is displayed,
The predetermined image in the second picture is an object image located in the vicinity of the first object image which is the first predetermined image in the first picture. An image playback device according to any one of the above. A predetermined image extraction step for extracting a predetermined image in the picture;
The predetermined image in the second picture extracted by the predetermined image extraction unit, which is a predetermined image that satisfies a predetermined condition in relation to the first predetermined image in the first picture extracted by the predetermined image extraction unit An identifier generating step for generating a predetermined identifier to be attached to
A legitimate image flag generating step of attaching a predetermined flag to the first predetermined image in the first picture and one of the predetermined images in the second picture according to the predetermined identifier;
An image reproduction method comprising: A predetermined image extraction step for extracting a predetermined image in the picture;
The predetermined image in the second picture extracted by the predetermined image extraction unit, which is a predetermined image that satisfies a predetermined condition in relation to the first predetermined image in the first picture extracted by the predetermined image extraction unit An identifier generating step for generating a predetermined identifier to be attached to
A legitimate image flag generating step of attaching a predetermined flag to the first predetermined image in the first picture and one of the predetermined images in the second picture according to the predetermined identifier;
A program that executes

Images