DE10214077B4 - Image detector - Google Patents

Abstract

An image search device including a CPU and a storage medium, the image search device configured to perform the steps of: storing image files with image data from a number of images captured by a camera, wherein an entire image surface represented by the image data is stored in dividing a number of unit image areas into matrix form and respectively storing an average of the luminance component included in the unit image area in the image data, performing an image search for searching a desired image file from the stored image files, comprising the steps of: displaying a reference image whose is divided into the number of unit image areas, wherein a user can select any one of the unit image areas, and this operation can be performed a plurality of times to designate any search area comprising a plurality of unit image areas, performing a search in which the mean values of the luminance components of the unit image areas of the search area of the reference image are compared with the average values of the luminance components of the unit image areas of the search area of another image file by determining the difference of the luminance data, and determining an image file as the desired image file if the result of the comparison meets a predetermined condition.

Description

Background of the invention

THE iNVENTION field

The present invention relates to an image searching apparatus used in a surveillance camera system. More particularly, the present invention relates to an image searching apparatus for searching a desired image among a plurality of images recorded on a recording medium.

Description of the Prior Art

In a conventional surveillance camera system, an arbitrary area of a surveillance image (life image) is set as a surveillance area, and a chronological change of a brightness level at the surveillance area is determined. At a time when the amount of change between frames exceeds a threshold, an alarm is generated. Further, the monitor image is recorded on an image recorder such as a hard disk recorder or a VCR (Video Cassette Recorder) to be subsequently confirmed.

However, in the prior art, if the monitored image is searched for at the time the alarm is generated from a plurality of frames of the recorded surveillance images, there is a need to check each frame of the surveillance image by the naked eye (ie, manually); and therefore it is too tedious.

Furthermore, the alarm will not be triggered even if a suspicious movement occurs in a surface other than the surveillance area and a significant change in the brightness level occurs there. It is more arduous and difficult to look from images in which the alarm has not been generated for a picture on which the suspicious motion occurs. In addition, in the prior art, in the event that the brightness level on the monitor image gradually changes, no alarm could be triggered because the alarm is triggered at a time when the magnitude of the change in brightness level between two adjacent frames exceeds the threshold. The result is the possibility that the suspicious movement that occurs in the surveillance area can escape during playback.

Further prior art is in US 6 031 573 A . DE 36 34 628 A1 . WO 97/20293 A1 . GB 2 158 324 A . EP 1 024 666 A2 and US Pat. No. 5,937,092 disclosed.

Summary of the invention

It is therefore a primary object of the present invention to provide a new image searcher.

Another object of the present invention is to provide an image searching apparatus which can surely and easily search a desired image among a plurality of images recorded on a recording apparatus.

An image searching apparatus for achieving this object has the features of claim 1.

The identifier designates the arbitrary area of the display image based on any one of the plurality of image signals as the search area. The comparator compares the brightness information corresponding to the search area between the reference picture signal and other picture signals, each of which is included in the plurality of picture signals. The determining means determines the image signal in which the comparison result of the comparator satisfies the predetermined condition as the desired image signal. Thus, the desired image can be searched safely and easily.

In the case that the comparator calculates a difference of the brightness information, the predetermined condition includes a condition that the difference exceeds the threshold value. Here, it is possible to change the sensitivity of the search when the threshold is arbitrarily set.

If the reference image signal is arbitrarily selected from the number of image signals, it is possible to improve the flexibility of the search.

Preferably, a number of luminance information, each corresponding to a number of areas constituting the display image, are respectively related to a number of image signals, and the identifier arbitrarily designates the search area from the number of areas. Therefore, it is possible to shorten a time required for the search.

The above-described objects and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.

list of figures

• 1 Fig. 12 is a block diagram schematically showing the configuration of an embodiment of the present invention;
• 2 Fig. 10 is an illustrative view showing a file format of an image file;
• 3 Fig. 10 is an illustrative view showing an example of the operation keys;
• 4 Fig. 10 is an illustrative view showing the recording state of a hard disk; 5 Fig. 10 is an illustrative view showing the relationships between image data and luminance data;
• 6 Fig. 12 is an illustrative diagram showing the relationships between arbitrary image data and luminance data;
• 7 Fig. 12 is an illustrative diagram of a display screen of a display when selecting a reference image;
• 8th Fig. 12 is an illustrative illustration of the display screen of the display after selecting the reference image;
• 9 Fig. 10 is an illustrative illustration of the display screen of the display when selecting a search area;
• 10 Fig. 12 is an illustrative diagram of the display screen of the display in the selection process of the search areas;
• 11 Figure 4 is an illustrative representation of the state that numbers are assigned to the selected search areas;
• 12 Fig. 12 is an illustrative illustration of the display screen of the display after selecting the search areas;
• 13 Fig. 12 is an illustrative view of the state when comparing luminance data between the reference image and another image;
• 14 Fig. 12 is an illustrative diagram of the display screen of the display when no image is found satisfying the search condition;
• 15 Fig. 12 is an illustrative diagram of the display screen of the display when an image satisfying the search condition is found;
• 16 Fig. 10 is a flowchart of an operation of a CPU; and
• 17 Fig. 10 is a flowchart of another part of the operation of the CPU.

Detailed Description of the Preferred Embodiments

Referring to the 1 has a surveillance camera system 10 according to this embodiment, an image searching device 12 , The image finder 12 draws on a hard disk 32 at fixed intervals, a still image of one of a surveillance camera 14 photographed object and reproduces the still image from the hard disk 32 , The image finder 12 also has the function to search a desired still image from the recorded still images.

An analog image signal coming from the surveillance camera 14 is output to the image viewer 12 entered via the input terminal T1. The input image signal is converted into digital image data It (t: an index indicating time) by the A / D converter 16, and the converted image data It becomes a video processor 18 entered. The video processor 18 performs predetermined processings such as filtering and the like on the input image data It to supply the processed image data It at the output terminal T2 to a display through a D / A converter 22 24 issue. Thus, on the display 24 a real-time motion picture of the object is displayed.

Furthermore, those by the video processor 18 processed image data It at predetermined intervals in an image memory 26 and then to an image compression expansion circuit 28 created. The image compression expansion circuit 28 compresses the applied image data It by satisfying a JPEG format to generate an image file Ft stored in the 2 is shown. The generated image file Ft is temporarily stored in a buffer memory 30 accumulated and then on the disk 32 recorded.

Referring to 2 starts the image file Ft of "SOI" of a start mark. After "SOI", "APPn", a mark of an application mark segment, "DQT", a mark of a quantization table, "DHT", a mark of a Huffman coding table, "DRI", a mark of a restart interval, "SOF", follow Mark of a frame head, and "SOS", a mark of a scanning head, in that order. Compressed image data is appended to the scanning head "SOS", and a tail mark "EOI" is appended to the tail end.

Among them, "APPn" is an arbitrary mark, and the time information indicative of the time when the image file Ft is generated, ie, a photographing time t, and luminance data St, which will be described later, become in the application mark segment in this embodiment recorded. It should be noted that a detailed The description is omitted because "APPn" meets a well-known Exif (Exchange Image File Format) or DCF (Design Rule for Camera File System) format. Further, since detailed description is omitted also in the case that marks other than "APPn" and the compressed image data satisfy the JPEG format.

Such a series of operations for inputting the still image signal of the object to recording the image file Ft is performed by a CPU 34 controlled. Specifically, the CPU sets 34 to the video processor 18 , the image memory 26 , the image compression expansion circuit 28 , the cache 30 and the hard drive 32 a timer signal for controlling its operation when the REC button 360 that the operation buttons 36 forms, is pressed. The video processor 18 , the image memory 26 , the image compression expansion circuit 28 , the cache 30 and the hard drive 32 are operated in response to the timing signal, and therefore, a series of recording operations are performed. It should be noted that when the stop button 362 , which is part of the operation buttons 36 is pressed, the CPU 34 stops the recording process.

Furthermore, the CPU 34 with a clock circuit 38 connected to counting a current time. The CPU 34 writes the photographing time in the application mark segment on the basis of the time information supplied by the clock circuit 38 is created.

Furthermore, the CPU 34 the luminance data St entered by the video processor 18 be generated. In detail, the video processor 18 is provided with a Y / C separation circuit 20 to extract a luminance component (Y component) from the image data It. The Y / C separation circuit 20 generates luminance data St corresponding to the extracted luminance component to enter into the CPU 34 to be entered. The CPU 34 writes the input luminance data St into the application mark segment.

Referring to 4 be on the hard drive 32 T + 1 image files Ft (t = 0, 1, 2 .... T) recorded in the order of photographing time t. Further, the image data It and the luminance data St which constitute each image file Ft are as shown in FIG 5 shown, related to each other.

Referring to 6 For example, each luminance data St is formed by N + 1 of the date St (Xn) (n = 0, 1, 2, ... N). An entire image area (an entire surveillance area) through the surveillance camera 14 ) represented by the image data It is divided into N + 1 unit image areas Xn (n = 0, 2, 1, ... N) in matrix form, and the data St (Xn) has an average of the luminance component, is contained in the unit image area Xn. It should be noted that the luminance data St is represented by 8 bits in the present embodiment.

The so on the hard drive 32 recorded image file Ft can be played in a picture playback mode. If a PLAY button 364 that the operation buttons 36 is pressed, the CPU changes 34 in the image display mode and sequentially transfers the image file Ft from the hard disk 32 on the cache 30 , It should be noted that it is possible to arbitrarily determine in the picture display mode from which picture file Ft the reproduction starts. The determination is made by inputting the photographing time t of the first reproducing image file Ft using the 10 keys 366 from "0" to "9", which the operation buttons 36 form, performed.

The image file Ft stored in the cache 30 is stored by the bit compression expansion circuit 28 expanded to the image data It. The expanded image data It is temporarily stored in the image memory 26 saved and then through the video processor 18 applied to the D / A converter 22. The D / A converter 22 converts the input image data into an analog image signal to supply the converted analog image signal to the display via the output terminal T2 24 to apply. Therefore, on the display 24 a playback picture of the object is displayed. It should be noted that if the STOP button 362 is pressed, the CPU 34 the series of playback operations stops to exit the picture display mode.

The image finder as described above 12 According to this embodiment, a function for searching the desired image file Ft from the plurality of image files Ft stored on the hard disk 32 are recorded. The search of the image file Ft is performed in a picture search mode. If a SEARCH button 368 which are part of the operation buttons 36 is pressed, the CPU changes 34 in the image search mode to display 24 to display a screen as in the 7 is shown (more precisely, the video processor 18 controlled to perform such a display).

Like in the 7 is shown in the upper part of the screen is a letter line 240 with "<motion detection search>", which indicates that the CPU 34 changes to image search mode. Below the character line 240 is a character line 242 with "Reference" displayed, and to the right of the character line 242 is a field 244 to enter any time (especially "year", "month", "date", "time", "minute" and "second"). There is also a cursor 246 in form of subscript to indicate either the year, the month, the date, the time, the minute and the second.

The cursor 246 moves within the field 244 right and left in response to pressing a right-arrow key 370 or a left arrow key 372 by the operation buttons 36 are formed. The parts that correspond to "Year", "Month", "Date", "Time", "Minute" and "Second" will be any numerical values by moving the cursor 372 and by pressing an up arrow key 374 , a down arrow key 376 or the ten keys 366 entered. It should be noted that according to the 7 in field 244 the time of 01-03-31, 08: 00.00 is entered.

If any time t so in the field 244 is entered, the CPU shows 34 at a part below the field 244 a reference picture 248 which was photographed at time t. More specifically, the CPU reads 34 from the hard disk 32 an image file Ft corresponding to the time t and controls the buffer memory 30 , the image compression expansion circuit 28 , the image memory 26 and the video processor to display the reference image based on the read image file Ft 248 display. Accordingly, in the 7 as the reference picture 248 An image displayed near a door 250 in a room at the time 01 / 03 / 31 . 08 : 00,00, has been photographed.

If a SET button 378 which are part of the operation buttons 36 is, in a display state according to 7 is pressed, the screen display of the display changes 24 of the 7 to 8th , More specifically, the display of the cursor 246 deleted from the screen, and split lines 252 in grid form divide the entire surface of the reference image 248 in unit image areas X0 ~ XN displayed. It should be noted that the reference image 248 in Horinzontalrichtung and in the vertical direction is divided into 8 parts. Thus, the reference picture 248 divided into 64 unit image areas Xn (n = 0,1, 2, 63). Further, a unit image area X0 is below the respective unit image areas Xn in the upper left with a thick frame 254 displayed in rectangular form.

The thick frame 254 is activated by pressing the arrow keys 370 to 374 which are part of the operation buttons 36 are moved to an arbitrary unit image Xn. When the SET button 378 is pressed in a state in which the thick frame 254 is moved to the arbitrary unit image area Xn, becomes within the thick frame 254 , like in the 9 shown a hatched pattern 256 displayed. The hatched pattern 256 is a flag indicating that the unit image area Xn is designated as a search area.

Referring to 10 can the search area 256 be determined several times in the same process as described above. According to the 10 is the biggest part of the door 250 which is contained in eight unit image areas Xn as a search area 256 designated. At this time, the numbers C0 to C7 are the respective search areas 256 in the order of designation, as in the 11 shown, designated.

When the EXIT button 380 which are part of the operation buttons 36 is pressed after the arbitrary unit image area Xn as a search area 256 has been designated, changes the display of the display 24 of the 10 on the 12 , Specifically, the display of the thick frame 254 deleted from the screen, and between the character line 242 "Reference" and "Reference Image 248" is a character line 258 with "Level ( 1 -5) "and to the right of the character line 258 is a field 260 to enter any value within the range of "1" to "5". The numeric entry in the field 260 is activated by pressing the up arrow key 374 or the down arrow key 376 which are part of the operation buttons 36 are, or the numeric keys 366 from "1" to "5".

If in the in the 12 displayed state, the SET button 378 is pressed, the CPU starts 34 the search processing. In particular, the CPU sets 34 (More specifically, in its own memory, not shown) stores the luminance data St of the entire area of the reference image 248 as reference luminance data Sref and compares the data Sref (C0) to Sref (C7) corresponding to the search area 256 among the reference luminance data Sref with the luminance data St (C0) to St (C7) corresponding to the search area 256 another image file Ft, like this in the 13 is shown. Furthermore, the CPU needs 34 in every search area 256 a difference (= | Sref (Cm) -St (Cm) |, m: index indicative of the search area number) and compares the difference ΔSt (Cm) with a predetermined threshold A.

It should be noted that the threshold A depends on that in the field 260 on the in the 12 shown screen entered value is determined. Specifically, the smaller the value is, which in the field 260 has been entered, the smaller the threshold A, and the larger in the field 260 is entered, the greater the threshold value A. If no image file has been found in which the difference .DELTA.St (Cm) exceeds the threshold value A, the CPU shows 34 on the display 24 a screen on, as in the 14 is shown. Referring to 14 is a line of characters on the top left of the screen 262 with "Reference" displayed, and below the character line 262 is a character line 264 indicating the photographing time t (= ref) of the reference image 248 displays. The reduced reference picture 248 will be on the right side of the character lines 262 and 264 are displayed, and at the bottom left of the screen are the line of characters 266 "Result" and the character line 268 with "nothing found" displayed. An operator can by viewing the character line 268 recognize with "found nothing" that the desired image file Ft has not been found.

On the other hand, if an image file Ft at which the difference ΔSt (Cm) exceeds the threshold value A has been found, the CPU shows 34 on the display 24 like in the 15 displayed screen. Referring to 15 is used instead of the character line 268 with "nothing found" below the character line 266 with "result" a character line 270 indicating the photographing time t of the found image file Ft. Right of the character lines 266 and 270 becomes a reduced picture 272 according to the found image file Ft. Therefore, the operator from the character line 170 and the reduced image 272 recognize an intruder at the time 01 - 03 - 31 09 : 12.53 has appeared.

In such an image search mode, the CPU goes through 34 Flowcharts as they are in the 16 and 17 are shown. It should be noted that a control program corresponding to the flowcharts in a ROM 40 is stored.

I'm in the 16 Step S1 shown becomes the reference image 248 determined and the luminance data St according to the reference data 248 are set as the reference luminance data Sref in response to a key operation of the operator. Subsequently, the index m of the search area number is set to "0" in step S3 as described above, and in step S5, a search area Cm is determined in response to a key operation by the operator. In step S7, it is determined whether or not the selection operation of the search area by Cm is completed by the operator.

The termination depends on which of the buttons, the SET button 378 or the exit button 380 in the in the 9 (or 10 ) is pressed. In detail, when the SET button 378 has been pressed, it determines that the selecting operation of the search area Cm is continued, and then the process returns to step S5 via renewal of the search area number in step S9. If, on the other hand, the exit button 380 has been pressed, it is determined that the dialing operation of the search area Cm has been completed by the operator, and the operation proceeds to step S11.

The index m of a current search area number is set to a maximum value M in step S11, and the threshold value A is determined in response to a key operation by the operator in the following step S13. After completing the setting of the threshold value A, a search operation is performed in step S15.

The search is performed according to one in the 17 Subroutine shown performed. In step S21, "ref + 1", that is, the photographing time of the next frame of the reference frame 248, is set as the indexing t of the photographing time. In step S23, it is determined whether or not the time t exceeds the maximum value T. In the case where the time t exceeds the maximum value T (t> T), the time is set to "0" in step S25, and then the operation proceeds to step S27. However, in the case where the time t is smaller than the maximum value T (t ≦ T), the process directly proceeds to step S27.

In step S27, the index m of the search area number is set to "0", and it is determined in step S29 whether or not the time t is equal to "ref", that is, "0". the photographing time of the reference image. Here, if the condition t = ref is not satisfied, the operation proceeds to step S31, and it is determined whether or not the index m of the search area number exceeds the maximum value M.

In the case that the index m does not exceed the maximum value M (m ≦ M), the difference ΔS (Cm) of the luminance data is compared with the threshold value A in step S33. Then, in the case where the difference ΔS (Cm) exceeds the threshold value A (ΔS (Cm)> A), the process goes to step S35, and on the basis of the image data It corresponding to the currently set time t, the screen ( 272 ), the Indian 15 is shown on the display 24 displayed. Upon completion of the process of step S35, the process is restored to a hierarchically higher level of the routine.

On the other hand, in the case where it has been determined in step S33 that the difference ΔS (Cm) is equal to or lower than the threshold value A (ΔS (Cm) ≤A), a renewal operation of the index m is performed in step S37, and then the process returns to step S31. In the case where it has been determined in step S31 that the index m exceeds the maximum value M (m> M), a renewal operation of the index t is performed in step S39, then the process returns to step S23. In the case where it has further been determined in step S29 that the condition t = ref is satisfied, in step S41, the 14 screen shown (the character line 268 "Found nothing") on the display 24 displayed. Upon completion of the process of step S41, the process is restored to the hierarchically higher level of the routine.

As can be seen from the above description, in searching the desired image, the luminance data Sref (Cm) corresponding to the search area Cm of the reference image 248 and the luminance data St (Cm) corresponding to the search area Cm of another image are compared with each other. This can be the reference image 248 and the search area Cm are arbitrarily selected. Furthermore, the threshold value A, which is compared with the difference Δ St (Cm), can be chosen arbitrarily. Since it is thus possible to arbitrarily determine the state of the image search, the desired image can be searched safely and easily.

Although the image viewer 12 in the present embodiment, in the surveillance camera system 10 is applied, it is needless to say that the image viewer 12 even with a different system than in the image search system 10 can be applied.

Although further in this embodiment, the hard disk 32 is used as the recording destination of the monitor image (the image file Ft), another storage medium such as a rewritable DVD (digital versatile disk), a CD (compact disk) and the like can be used. instead of the hard drive 32 be used.

Further, although the IPEG format is used in this embodiment for compressing the image data It, another compression format such as Graphics Interchange Format (GIS), Portable Network Graphics (PNG), Motion Picture Expert Group (MPEG), and the like may be used. , be used. However, since the MPEG format adopts a GOP structure in which a number of screens of image data It are grouped, in the case that the MPEG format is used in the present invention, there is a need to use a luminance data St GOP unit to form. It should be noted that the image data It is on the hard disk 32 can be recorded in a non-compressed state.

In addition, in the present invention, although the luminance data St is written into the application mark segment of the image file Ft, the image data It and the luminance data St may be recorded in different regions of the same recording medium or on separate recording media as long as parallelism between image data It and luminance data St is maintained becomes.

Although the present invention has been described and illustrated in detail, it is to be understood that this is intended to be illustrative and exemplary only and not limiting, the spirit and scope of the present invention being limited solely by the terms of the claims.

Claims (4)

An image viewer having a CPU and a storage medium, the image finder being configured to perform the following steps: Storing image files with image data from a number of images captured by a camera, wherein an entire image area represented by the image data is divided into a number of unit image areas in matrix form, and an average of the luminance component included in the unit image area is stored in the image data, Perform an image search to find a desired image file from the saved image files with the following steps: Displaying a reference image whose entire image area is divided into the number of unit image areas, wherein a user can select any one of the unit image areas, and this process can be performed multiple times to designate any search area comprising a plurality of unit image areas, Performing a search processing, wherein the mean values of the luminance components of the unit image areas of the search area of the reference image are compared with the average values of the luminance components of the unit image areas of the search area of another image file by determining the difference of the luminance data, and determining an image file as the desired image file if the result the comparison satisfies a predetermined condition. Image viewer after Claim 1 wherein the predetermined condition is a condition that the difference exceeds a predetermined threshold. Image viewer after Claim 2 wherein the threshold value can be set by the user. Surveillance system with an image viewer according to one of Claims 1 to 3 ,

Images