JP5137863B2 - Object detection device - Google Patents

Description

  The present invention relates to an object detection apparatus that detects a detection target from an image.

  An object detection device that detects whether an image area corresponding to a detection target is included in an image is known.

  There has been proposed an object detection device that inputs an image to a discriminator that has previously learned whether or not the target object exists and detects the target object in the image. For example, Patent Document 1 discloses a technique for tracking a person using a particle filter, and using a cascade type discriminator and using the number of passing stages of the cascade as an indicator of humanity.

  Patent Document 2 discloses a normal determination unit that determines whether or not a monitoring target is normal based on normal learning data and imaging data obtained by learning a normal monitoring target in advance and an abnormal monitoring target in advance. An abnormality determination unit that determines whether or not the monitoring target is abnormal based on the abnormal learning data and the imaging data, and the monitoring target is determined based on the determination result of the normal determination unit and the determination result of the abnormality determination unit An abnormality detection device that determines whether the operation is normal or abnormal is disclosed.

  In Patent Document 3, a cut is obtained from a video composed of time-sequential frames, learning is performed sequentially based on feature vectors extracted from the frames, and based on a statistical distribution of feature vectors. A non-stationary image detection device is disclosed that obtains a conversion parameter for mapping a feature vector in an identification space that is optimal for identification, and detects a non-stationary cut by mapping the feature vector to the identification space based on the conversion parameter.

JP 2008-26974 A JP 2008-234551 A JP 2007-299250 A

  By the way, since the conventional object detection apparatus learns the detection target in advance, the detection target cannot be accurately detected unless learning is performed in consideration of the variation of any installation environment. As the fluctuation of the installation environment, the object detection device is turned on and off when the illumination is turned on and off, and the fluctuation of brightness around the object detection device changes as the sun goes down from day to night. There is a sudden change in time such that the brightness of the surroundings changes. It is necessary to reduce false detection and misreporting due to such temporal variation factors.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide an object detection apparatus that can cope with temporal changes in the installation environment.

  One aspect of the present invention is an object detection device that detects a detection target from an image obtained by imaging a monitoring area, and that is detected from an image acquisition unit that acquires the image and the image that is input from the image acquisition unit. A detection window area image in which a window is set is captured, a similarity indicating the degree of similarity between the detection window area image and a detection target is calculated, and the detection window area image is detected based on the similarity. Using a fixed discriminator that determines whether or not an object exists based on information learned in advance and first dictionary data that is updated with information calculated from the detection window region image, the detection object is A slow learning discriminator that determines whether or not it exists, and a first that is updated with information calculated from the detection window region image so that the influence of past update contents is reduced in a shorter time than the first dictionary data. 2 Whether or not the detection target exists based on the output from the sudden change learning discriminator and the slow learning discriminator that determines whether or not the detection target exists And determining means for determining.

  Here, it is preferable that the slow learning discriminator and the sudden change learning discriminator perform the determination only on the detection window region image determined by the fixed discriminator that the detection target exists. .

  In addition, it is preferable that the determination unit determines that the detection target exists when the slow learning classifier and the sudden change learning classifier determine that the detection target exists.

  In addition, the slow learning discriminator and the sudden change learning discriminator may be configured such that the first dictionary data and the second data are only calculated based on the similarity calculated when the determination unit determines that the detection target does not exist. It is preferable to update the dictionary data.

  According to the present invention, the object to be detected can be detected in response to the temporal variation of the installation environment in the object detection device that detects an object by prior learning.

It is a functional block diagram which shows the structure of the object detection apparatus in embodiment of this invention. It is a figure explaining the detection process of an object. It is a figure which shows the structure of the discriminator in 1st Embodiment. It is a figure which shows the example of the similarity histogram in 1st Embodiment. It is a figure explaining the update method of the similarity histogram in 1st Embodiment. It is a flowchart which shows the process in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the position setting process of the detection window area | region in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the identification process in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the update process in the object detection apparatus in 1st Embodiment. It is a flowchart which shows the determination process in the object detection apparatus in 1st Embodiment. It is a figure which shows the structure of the discriminator in 2nd Embodiment. It is a figure which shows the example of the feature-value in 2nd Embodiment. It is a flowchart which shows the identification process in the object detection apparatus in 2nd Embodiment. It is a flowchart which shows the update process in the object detection apparatus in 2nd Embodiment. It is a flowchart which shows the initial setting process in the object detection apparatus in 2nd Embodiment.

<First Embodiment>
As shown in FIG. 1, the object detection apparatus 1 according to the embodiment of the present invention includes an image acquisition unit 2, a signal processing unit 3, a storage unit 4, and an output unit 5. The object detection device 1 acquires a monitoring image obtained by imaging a predetermined monitoring space, and detects a detection target such as a person or an object captured in the image. The image acquisition unit 2, the signal processing unit 3, the storage unit 4, and the output unit 5 are connected so as to be able to transmit information to each other.

  In this embodiment, an example in which a person shown in an image is a detection target will be described. However, the present invention is not limited to this, and the present invention can also be applied to the case of detecting articles such as commodities placed in distribution, vehicles passing through, and the like.

  The image acquisition unit 2 includes a so-called surveillance camera configured to include an imaging element such as a CCD element or a C-MOS element, an optical system component, an analog / digital converter, and the like. The image acquisition unit 2 may acquire an image via an Internet network. The image acquisition unit 2 transmits the captured image to the signal processing unit 3. The interval at which the images are acquired may not be a fixed time interval.

  The image is, for example, a color image having a width of 320 pixels, a height of 240 pixels, and each pixel expressing R (red), G (green), and B (blue) with 256 gradations.

  The signal processing unit 3 includes an arithmetic circuit such as a CPU, DSP, MCU, or IC. The signal processing unit 3 is connected to the image acquisition unit 2, the storage unit 4, and the output unit 5 so as to be able to transmit information. The signal processing unit 3 is a program that describes the processing in each means such as the detection window region selection means 30, the detection window control means 32, the fixed discriminator 34, the learning discriminator 36, the discriminator update means 37, and the judgment means 38. The computer is caused to function as each means by reading out from the storage unit 4 and executing it.

  As illustrated in FIG. 2, the signal processing unit 3 detects a detection target from the input image 200 from the image acquisition unit 2. For convenience of explanation, the upper left corner of the image 200 is the origin X = 0, Y = 0, the horizontal direction is the X axis, the vertical direction is the Y axis, the X axis is the right direction, and the Y axis is positively increased downward. And In the input image 200, a detection object (person) is shown at the positions of the areas 203 and 204, and rectangles 201 and 202 indicate detection window areas.

  The object detection apparatus 1 scans while gradually shifting the detection window area, and determines whether a person is captured in the detection window area. The arrow indicates the upper left coordinates when the detection window area is shifted, and the detection window area is scanned so as to search the entire image 200 without omission. The rectangles in the areas 203 and 204 indicate detection window areas (person candidate areas) that are determined to be human as a result of the detection process. In the vicinity of an image region where a person is shown, there may be a plurality of detection window regions that are determined to be humans. However, the final detection window region (in FIG. (Thick line: human area). However, the detection window area may be obtained by subjecting the reference background and the input image to difference processing, and labeling each image after the difference as the detection window area, or the entire image as the detection window area. A rectangle in the region 205 indicates a region where a hanger with clothes similar to a person who is a detection target is captured.

  The detection window area selection means 30 determines the width and height of the detection window area. The detection window area is scanned to determine whether or not a person is captured in the image while changing the width and height of the detection window area in order to correspond to people captured in various sizes in the image. Note that one or a plurality of widths and heights of the detection window region are stored in advance in the storage unit 4 in consideration of the size of the detection object on the image. However, even if the width and height are not stored in advance, they may be determined according to a predetermined rule.

  The detection window control means 32 determines an interval (scanning interval) for shifting the detection window region, and shifts the detection window region based on the determined scanning interval.

  The fixed discriminator 34 calculates the degree of similarity of how much the image in the detection window area resembles a person. The fixed discriminator 34 learns in advance using a large number of “person” image data and “non-person” image data.

  The similarity is calculated by the processing procedure shown in FIG. Specific examples of the fixed classifier 34, the learning classifier 36, and the classifier update means 37 shown in FIG. 1 are shown as a cascade type classifier 320, a learning classifier 330, and a classifier update means 332, respectively, in FIG.

  First, an image in the detection window region cut out from the input image is input to the cascade discriminator 320. The cascade type classifier 320 is a classifier in which a plurality of strong classifiers such as the strong classifiers 321, 322, and 323 are arranged in series (here, an example in which N strong classifiers are arranged in series is shown). ). Individual strong classifiers are Histograms of Oriented Gradients (HOG) features (Navneet Dalal and Bill Triggs, Histograms of Oriented Gradients for Human Detection, In Proceedings of IEEE Conference Computer Vision and Pattern Recognition 2005) To learn in advance with AdaBoost. That is, various images of people who are detection objects and images that do not show people are prepared in large quantities, each image is correctly identified as to whether it is an image of the detection object, and these data To learn with Adaboost so that both can be identified. When calculating the similarity, each strong classifier calculates the HOG feature from the input image, and calculates the similarity from the feature amount selected by Adaboost.

  The order in which the strong classifiers 321 to 323 are calculated in advance is determined, and the strong classifier 321 to be calculated first is 1, the next strong classifier 322 to be calculated is 2. Assign the same number as the order of calculation.

  Each of the strong discriminators 321 to 323 receives the image cut out as the detection window region and calculates the similarity. The strong classifiers 322, 323, etc. other than the first strong classifier 321 calculate the similarity only when the similarity calculated in the previous stage is larger than the threshold (arrow T in the figure). When the value is equal to or less than the threshold value (arrow F in the figure), the classifier update unit 332 updates the learning classifier 330. The threshold is set to 0, for example, and if it is greater than 0, it is similar to a person, and if it is 0 or less, it is determined that it is not similar to a person. Hereinafter, the number of the strong classifier for which the similarity is calculated is referred to as the number of determination steps.

  The learning discriminator 330 uses the similarity of each strong discriminator of the cascade discriminator 320 as an input to determine whether or not the person is a person. In the present embodiment, the learning discriminator 330 includes two discriminators, a slow fluctuation discriminator and a sudden fluctuation discriminator, and each discriminator makes a determination using different “non-human” similarity histograms. Process. However, since only the similarity histogram of “non-human” to be used is different and the other processes are the same, the learning discriminator 330 will be described without distinction.

  The learning discriminator 330 performs the determination only when the cascade type discriminator 320 determines that the last strong discriminator 323 is a person (arrow T in the figure). Only when the possibility of “person” is high in the cascade type classifier 320, that is, when the similarity obtained by the strong classifier 323 at the final stage is larger than the threshold (arrow T in the figure), the learning classifier 330 makes the determination. I do. The learning discriminator 330 receives the similarity Sn (n is 1 to M: M is the number of determination stages (1 ≦ M ≦ N)) obtained by the strong classifiers up to the number of determination stages in the cascade classifier 320 as an input. A similarity vector S = (similarity of the first strong classifier, similarity of the second strong classifier,..., A vector representation of the similarities obtained by the strong classifiers of the cascade type classifier 320 arranged side by side. (Similarity of Mth strong classifier) = (S1, S2,..., SM).

  Calculating the probability P (person | S) of “person” when the similarity vector S is obtained, and the probability P (non-person | S) of “non-person” when the similarity S is obtained; If P (person | S)> P (non-person | S), it is determined as “person”, otherwise it is determined as “non-person”.

Calculations of P (person | S) and P (non-person | S) can be obtained by the formulas (1) and (2) from Bayes' theorem.

  Here, P (S | People) obtains the probability (likelihood) of obtaining the similarity S in the case of “People”, and P (S | People) obtains the similarity S in the case of “Non-People”. Probability (likelihood). P (person) is a prior probability that indicates the probability of “person” being included in the detection window area, and similarly P (other than person) is included in the detection window area with a probability of “non-person”. It is a prior probability that represents whether or not. Since the probability that “person” and “non-person” are included in the detection window region in advance is not known, it is assumed that the prior probability P (person) = P (non-person) = 0.5. Since P (non-person | S) = 1−P (person | S), if P (person | S) is obtained, P (non-person | S) is easily obtained. Hereinafter, P (person | S) is referred to as the similarity in the learning discriminator 330.

When a random variable vector s having the similarity of each strong classifier as an element is prepared, the distribution P (s | person) of the similarity vector s of “person” and the distribution P (s) of the similarity vector s of “non-person”. If (other than people) is obtained, P (people | s) can be calculated. Although the similarity vector s is multi-dimensional data, assuming that the similarity sn of each strong classifier is independent, P (s | people) and P (s | others) are expressed by equations (3) and (4). Desired.

  Here, P (sm | person) is the distribution of similarity of “m” of the mth strong classifier, and P (sm | other) is the similarity of “mother” of the mth strong classifier. Distribution.

  In the present embodiment, it is expressed by a histogram such that the sum of similarity distributions of all bins (separators) is 1. Each histogram is stored in the similarity histogram 42 of the storage unit 4. However, the similarity distribution is not limited to the histogram, and may be expressed by a normal distribution, a mixed normal distribution, or the like.

  The “person” similarity histogram 42 is configured by obtaining a similarity distribution of each strong classifier using learning data that is an image including “person” prepared in advance. In the present embodiment, the “person” similarity histogram 42 is not sequentially learned but may be updated. The similarity histogram 42 of “non-person” assumes that no person is included in the first few frames when the surveillance camera is installed, and the similarity distribution of the strong classifiers obtained at that time is shown. Configure by asking. Thereafter, when the detection window region is determined to be “non-human” in the cascade type discriminator 320 or the learning discriminator 330, the similarity histogram 42 of “non-human” is used by using the similarity obtained by the cascade type discriminator 320. Are updated sequentially.

  FIG. 4 shows an example of the similarity histogram 42 when the number of strong classifiers in the cascade classifier 320 is N = 2. The similarity histogram 401 is a distribution of similarity “person” 402: P (s1 | person) and similarity distribution 403: P (s1 | person) of “non-person” in the similarity s1 of the first strong classifier. Except). The similarity histogram 411 shows the distribution of similarity of “person” 412: P (s2 | person) and the distribution of similarity of “non-person” 413: P (s2 | person) in the similarity s2 of the second strong classifier Except). The four similarity histograms 402, 403, 412, and 413 are normalized so that the sum of the similarity distributions of all bins is 1.

  When the determination is made only with the cascade classifier 320, if the degree of similarity obtained by each strong classifier is larger than a predetermined threshold (for example, set to 0), it is determined as “person”. Then, the probability is calculated from the obtained similarity of each strong classifier and the past similarity histogram 42 and can be adaptively determined.

  As an example, when the similarity distribution of “person” and “non-person” is represented by the similarity histograms 401 and 411 shown in FIG. 4, the similarity of the first strong classifier is S1 = 1.0, A case where the similarity of the second-stage strong classifier is S2 = 1.0 will be described. When the threshold values in the first and second stage strong classifiers are 0, the similarity exceeds the threshold value in both the first and second stage strong classifiers. It is determined as “person”. On the other hand, in the learning discriminator 330, the probability of “person” when the similarity vector S = (1.0, 1.0) is obtained: P (person | S = (1.0, 1.0) ) = P (S1 = 1.0 | People) × P (S2 = 1.0 | People) × P (People) / A and the probability of “Non-People”: P (Non-People | S = (1. 0, 1.0)) = 1−P (person | S = (1.0, 1.0)). However, A = P (S | person) × P (person) + P (S | other than person) × P (other than person). Based on the histograms 401 and 411 shown in FIG. 4, P (person | S = (1.0,1.0)) <P (non-person | S = (1.0,1.0)). The discriminator 330 determines “other than human”.

  If a similarity histogram matching the characteristics of the installation environment of the object detection device can be constructed, the learning discriminator 330 can make a more accurate determination based on the similarity calculation result from the cascade discriminator 320.

  After the determination by the learning discriminator 330, the learning discriminator 330 is updated by the discriminator update unit 332. Further, the similarity management unit 334 associates the similarity, the size (width and height) of the detection window region, and the center coordinates, and stores them in the storage unit 4 as the determination information history 40.

  The discriminator update unit 332 updates the similarity histogram of the learning discriminator 330. In the present embodiment, the learning classifier 330 is updated by the classifier updating means 332 when the degree of similarity obtained by each strong classifier is equal to or less than a threshold value (arrow F in FIG. 3). In addition, even if the similarity determined by the strong classifier exceeds the threshold value in the cascade classifier 320 (the arrow T in FIG. 3), the learning classifier 330 determines “other than human” (FIG. 3). In the case of the middle F arrow), the classifier update unit 332 updates the learning classifier 330. That is, only the “non-person” similarity histogram 42 is updated, and the “person” similarity histogram 42 is not updated.

  The discriminator update unit 332 updates the similarity histogram 42 obtained up to one time ago with the data obtained at the current time. For example, as shown in FIG. 5, α blending is used for the update process by the discriminator update unit 332. FIG. 5 shows a process of updating the n-th “non-human” similarity histogram 42 of the learning discriminator 330, based on the similarity histogram 501 before the update up to the previous time and the data obtained at the current time. An updated similarity histogram 502 is shown.

  When it is determined whether or not an image of a person is included in the detection window area, it is determined that the cascade discriminator 320 determines “person” and the learning discriminator 330 determines “other than human”. A method of updating the similarity histogram 501 corresponding to the nth strong classifier when the similarity of the nth strong classifier is 0.8 is shown. First, each bin of the similarity histogram 501 before update is multiplied by (1.0−α). Thereafter, a weight corresponding to α is added to the bin corresponding to the similarity of 0.8 obtained at the current time (the hatched portion of the bin 503 in the figure). α is a predetermined parameter. When α is large, the influence of past information is reduced earlier, and when α is small, the influence of past information is longer. When updating for the first time, that is, when the similarity histogram 501 is empty, the bin value corresponding to the obtained data is set to 1. However, since the influence of the first obtained data is large in this way, α may be set to a relatively large value while the number of updates is small, and may be set to a smaller value as the number of updates increases. Good.

  In the present embodiment, “non-human” similarity histograms 42 are used for the slow fluctuation classifier and the sudden fluctuation classifier, respectively. The update parameter α in the update process of the “non-person” similarity histogram 42 in the slow fluctuation classifier is smaller than the update parameter α in the update process of the “non-person” similarity histogram 42 in the sudden change classifier. Value. However, 0 <(update parameter α in the discriminator for slow fluctuations) <(update parameter α in the discriminator for sudden fluctuations) <1. In the slowly varying discriminator, since the update parameter α is small, the influence of the past update contents remains longer than the suddenly changing classifier in the similarity histogram 42 of “non-human”, and the environment changes slowly over a long time. Can respond to changes in On the other hand, since the update parameter α is large in the sudden change discriminator, the influence of the past update content becomes smaller in the similarity histogram 42 of “non-human” in a shorter time than the slow fluctuation discriminator, and rapidly changes in a short time. Can respond to changes in the environment.

  Here, referring back to FIG. 1, the determination unit 38 determines the size (width, height) and center coordinates of the detection window region stored as the determination information history 40, the similarity of the learning discriminator 330 (for slow fluctuation). Using the similarity obtained by the discriminator and the similarity obtained by the abrupt change discriminator, it is finally determined where the person is shown in the image. Among the data included in the determination information history 40, a detection window region in which the similarity of the learning discriminator 330 (both the slow fluctuation discriminator and the sudden fluctuation discriminator) is equal to or higher than a threshold (for example, set to 0.8). Are extracted as candidate areas. If there is no human candidate area, the process ends because there is no human area in the input image.

  When there is a human candidate area, the human candidate areas that overlap a certain area (for example, more than half of the area of the detection window area) are collected using the size and center coordinates of the detection window area. The detection window region having the highest similarity of the learning discriminator 330 is selected as the human region among the grouped human candidate regions. The detection window area selected for each collected area is set as a human area, and information on the detection window area selected is output to the output unit 5.

  The storage unit 4 is configured by a memory device such as a ROM or a RAM. The storage unit 4 is connected so as to be accessible from the signal processing unit 3. The storage unit 4 can store various programs and various data, and reads and writes these pieces of information in response to requests from the signal processing unit 3. The storage unit 4 functions as the similarity management unit 334, and as the determination information history 40, the size (width, height) of the detection window region, the center coordinates, and the similarity of the learning discriminator 330 (obtained by the slow fluctuation discriminator). Three types of information (similarity obtained by the rapid change discriminator) and stored in association with each other. For each strong classifier included in the cascade classifier 320, a similarity histogram 42 of “person” and “non-person” is stored in association with the number of each strong classifier. For the similarity histogram 42 of “non-human”, two types of similarity histograms 42 are stored for the slow fluctuation discriminator and the sudden fluctuation discriminator. The similarity histogram 42 corresponds to dictionary data, the similarity histogram of the slow fluctuation classifier corresponds to the first dictionary data, and the similarity histogram of the sudden fluctuation classifier corresponds to the second dictionary data.

  The output unit 5 can be configured to include a sound output unit that outputs a notification sound and a display unit that displays an input image. When the determination unit 38 detects a human area, an alarm is sounded by a sound output unit such as a speaker or a buzzer, or an input image is displayed on an external display device such as a display. The output unit 5 may include an interface for connecting the computer to a network or a telephone line. In this case, the output unit 5 sends an input image and human area information to a center device (not shown) via information transmission means such as a telephone line or the Internet. The center device is a host computer installed in a center or the like that monitors a detection target in an image.

  Hereinafter, the processing in the object detection apparatus 1 will be described with reference to the functional block diagram of FIG. 1 and the flowchart of FIG.

  In step S <b> 10, an image is acquired by the image acquisition unit 2, and the image is transmitted to the signal processing unit 3. The image acquisition timing is a fixed time interval.

  In step S20, the size (width and height) of the detection window region is determined. The entire image is scanned with each size of the detection window region while sequentially changing the size of the detection window region to a plurality of preset sizes. In this embodiment, since the detection window region is rectangular, it is only necessary to determine the width and height as the size. However, the detection window region may have an arbitrary shape, and in that case, the shape and size are determined. To decide. This processing is performed by the detection window region selecting means 30.

  The processing from step S30 to step S50 is repeated until the entire image has been scanned with the size of the detection window region set in step S20.

  In step S30, the position of the detection window area is determined. This process is performed by the detection window control means 32. This process is performed according to the flowchart shown in FIG. The upper left coordinate of the detection window area is called a start point, the start point of the detection window area to be determined is (SX, SY), and the previous start point of the detection window area is (BX, BY).

  In step S301, it is determined whether or not the size of the detection window area has been changed. If the size of the detection window area has just been changed, the start point (SX, SY) of the detection window area is set to (0, 0) in step S305, and the process proceeds to step S40. If it is not the first process after the size of the detection window area is changed, the process proceeds to step S302.

  In step S302, the start point (BX, BY) of the previous detection window area is read. In step S303, it is determined whether scanning has been performed from the start point (BX, BY) of the previous detection window area read in step S302 to the right end of the image. When scanning to the right edge of the image is completed in the detection window area, that is, when the right edge BX + W (W is the width of the detection window area) of the detection window area coincides with the right edge of the image, the detection window area in step S306 Is set to (SX, SY) = (0, BY + q), and the process proceeds to step S40. However, q is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, q = 4 pixels is set. If not scanned to the right end, the process proceeds to step S304.

  In step S304, the detection window area is shifted by p pixels in the X direction, and (SX, SY) is determined as the start point of the detection window area so as not to be shifted in the Y direction. That is, the detection window region start point (SX, SY) = (BX + p, BY). However, p is a predetermined constant. For example, when the input image has a width of 320 pixels and a height of 240 pixels, p = 4 pixels is set. Thereafter, the process proceeds to step S40.

  In step S40, a similarity indicating how much the image in the detection window region set in step S30 resembles a person is obtained. This is the discrimination processing of the cascade type discriminator 320 and the learning discriminator 330 shown in FIG. The identification process will be described with reference to the flowchart of FIG.

  In step S401, the HOG feature amount is calculated from the current detection window region. However, this process is faster between steps S10 and S20 by calculating the edge strength and angle of each pixel of the input image and creating an integral image for each edge angle. It becomes possible.

  In step S402, it is determined whether or not all strong classifiers have been investigated. If all the strong classifiers have been investigated, the process proceeds to step S406. If not, the process proceeds to step S403, and the next strong classifier is investigated. In step S403, the similarity for determining whether the image in the detection window area is similar to a person is calculated.

  In step S404, the similarity calculated in step S403 is stored in the temporary storage area together with the number of the strong classifier that has been determined. This is used in similarity calculation in the learning discriminator 330 in step S407 and update processing in step S50.

  In step S405, it is determined whether the similarity calculated in step S403 is greater than a preset threshold value. If it is greater than the threshold value, the process proceeds to step S402. If it is less than the threshold value, the similarity of a learning discriminator 330 described later is stored as a value equal to or less than the threshold value in step S501, and the process proceeds to step S50.

  Steps S406 to S409 are processing of the learning discriminator 330. In step S406, “non-human” in the slowly varying classifier and the suddenly varying classifier whose number of updates is equal to or higher than the reference value in the similarity histogram 42 of the strong classifiers up to the number of determination stages of the cascade classifier 320. The similarity histogram 42 is extracted. This is because if the similarity histogram 42 with a small number of updates is used, the reliability of the similarity calculated by the learning discriminator 330 becomes low. Therefore, the similarity in the learning discriminator 330 is calculated using the similarity histogram 42 updated more than the reference value and the corresponding similarity of the strong discriminator stored in step S404.

  In step S407, the similarity is calculated by the slow fluctuation discriminator. The similarity histogram of “person” and the similarity histogram of “non-person” of the slow fluctuation discriminator are read from the similarity histogram 42, and the similarity is calculated using P (person | S) in the equation (1).

  In step S408, the similarity is calculated by the sudden change discriminator. The similarity histogram of “person” and the similarity histogram of “non-person” of the sudden change discriminator are read out from the similarity histogram 42, and the similarity is calculated using P (person | S) in the equation (1).

  In step S409, the determination information history 40 is obtained by associating the similarity calculated by the slow fluctuation discriminator and the sudden fluctuation discriminator in steps S407 and S408 with the size (width and height) of the detection window region and the center coordinates. Is stored in the storage unit 4 and the process proceeds to step S50.

  In step S50, similarity histogram update processing is performed. This is done by the discriminator update means 37. The update process is performed according to the flowchart of FIG. In the present embodiment, the similarity histogram 42 of “non-person” is updated, and the similarity histogram 42 of “person” is not updated.

  In step S501, when the similarity calculated by the slow change discriminator in step S407 is larger than the threshold, and the calculated similarity calculated by the sudden change discriminator in step S408 satisfies both conditions larger than the threshold. Since there is a high possibility of being a “person”, the similarity histogram 42 is not updated, and the process proceeds to step S60. The threshold value for the slowly varying classifier and the threshold value for the suddenly varying classifier may be the same or different.

  In step S502, the “non-human” similarity histogram 42 corresponding to each strong classifier included in the cascade classifier 320 is updated with the similarity stored in step S404 of FIG. Therefore, only the “non-human” similarity histogram 42 corresponding to the strong classifier used for the current determination is updated. Here, both the “non-human” similarity histogram 42 for the slow fluctuation classifier and the “non-human” similarity histogram 42 for the rapid fluctuation classifier are updated. The update method is as described for the discriminator update means 332. After the update process, the process proceeds to step S60.

  In step S60, it is determined whether or not the detection process has been completed for detection window regions of all sizes. If the inspection is completed for all detection window regions, the process proceeds to step S70, and if not, the process returns to step S20.

  In step S70, a person is finally captured from the size (width and height) of each detection window region stored in the determination information history 40 obtained in step S50, the center coordinates, and the similarity of the learning discriminator 330. Determine the position. This processing is performed by the determination means 38. The determination process is performed according to the flowchart of FIG.

  The processing of steps S701 to S702 is performed for all detection window regions stored as the determination information history 40 obtained in step S50. In step S701, the similarity calculated by the slow fluctuation discriminator is larger than a threshold (for example, 0.8), and the calculated similarity calculated by the sudden fluctuation discriminator is the threshold (for example, 0.8). It is determined whether or not both larger conditions are satisfied. If the conditions are satisfied, the process proceeds to step S702. If not, the next detection window region is determined. In step S702, the detection window area that satisfies the condition in step S701 is added to the human candidate area.

  As shown in FIG. 2, candidate candidate areas obtained by repeating steps S701 to S702 on the information stored in the determination information history 40 are areas 203 and 204 near the area where the person is shown. May be extracted multiple times. In steps S703 to S705, a human region is finally selected from a plurality of human candidate regions.

  In step S703, using the size and center coordinates of the detection window area extracted as the human candidate area, the human candidate areas that overlap a certain area (for example, more than half the area of the detection window area) are grouped together. . In step S704, the detection window region having the highest similarity of the learning discriminator 330 is selected as a human region among the human candidate regions for each group created in step S703. In step S705, the detection window area selected in step S704 is set as a human area, and the process proceeds to step S80.

In step S80, if there is at least one human area based on the human area determined in step S70, the detection window information of the determined human area is output as an abnormal signal together with the image. This processing is performed by the output unit 5.
[Initial settings]
The initial setting process when the surveillance camera is installed will be described below. At the time of initial setting, the similarity histogram 42 of “non-human” corresponding to each strong classifier is registered under the assumption that there is no person in the imaging area of the surveillance camera. It is assumed that the initial setting process is completed after a predetermined number of images are processed after the monitoring camera is installed.

  The initial setting process is a process in which steps S40 and S50 are slightly changed except for steps S70 and S80 in the flowchart of the detection process of FIG. Further, only differences between the processes of steps S40 and S50 will be described.

  In step S40, the processing from step S401 to S405 in FIG. 8 is performed, and if the investigation has been completed for all strong classifiers in step S402 (YES in step S402), the processing in steps S406 to S409 is not performed. The process proceeds to S50. That is, it is different in that the processing by the learning discriminator in steps S406 to S409 is omitted.

  In step S50, since it is assumed that no person is in the monitoring space in the initial setting process, the determination process in step S501 is not performed. In step S502, using the similarity of the strong classifiers up to the number of determination stages obtained in step S40, the similarity histogram of “non-human” of the classifier for slow fluctuation and the classifier for sudden fluctuation of the corresponding strong classifier. 42 is updated, and then the process proceeds to step S60.

  In addition, the similarity histogram 42 of “non-human” is not sequentially updated every time an image is acquired, but the similarity calculated by the cascade discriminator 320 for a predetermined number of images is stored, and similarities are collectively obtained. The degree histogram 42 may be created.

<Second Embodiment>
The object detection apparatus according to the second embodiment performs the determination of the fixed classifier and the learning classifier in parallel, and integrates the respective determination results. In the second embodiment, processing in the fixed discriminator 34, the learning discriminator 36, the discriminator update unit 37, and the determination unit 38 in the first embodiment is different and will be described. In the storage unit 4, the similarity histogram 42 is stored in the first embodiment, but in the second embodiment, it will be described as the learning discriminator feature amount distribution 42.

  The fixed discriminator 34 obtains a similarity indicating how much the image in the detection window area resembles a person. This is the same processing as in the first embodiment. Specific examples of the fixed classifier 34, the learning classifier 36, and the classifier update means 37 shown in FIG. 1 are shown as a cascade type classifier 620, a learning classifier 632, and a classifier update means 636, respectively.

  The cascade type discriminator 620 in FIG. 11 includes a plurality of strong discriminators 621 to 623, and uses an image cut out as a detection window region as an input to calculate the similarity. Strong discriminators 622, 623, etc. other than the strongest discriminator 621 calculate the similarity only when the similarity calculated in the previous stage is larger than the threshold (arrow T in the figure). For example, the threshold is set to 0, and if the similarity obtained by the strong classifiers 621 to 623 is greater than 0, it is similar to a person, and if it is 0 or less, it is determined not to be similar to a person. When the similarity obtained by the strong classifier 323 at the final stage is larger than the threshold (arrow T in the figure), or when the similarity obtained by each strong classifier is less than or equal to the threshold (arrow F in the figure). In addition, the determination stage number and similarity of the cascade discriminator 620 are output to the determination information history unit 630. The determination information history unit 630 associates the output determination step number and similarity with the size and center coordinates of the detection window region and stores the determination information history 40 in the storage unit 4.

  Unlike the first embodiment, the learning discriminator 632 receives an image and determines whether it is a person. The learning discriminator 632 extracts a feature amount related to the edge from the input image of the detection window region. First, an edge image is created from the image of the detection window area. The edge image may be obtained for the entire monitoring image in advance. For example, as shown in FIG. 12, the detection window area is divided into a total of eight rectangular areas divided into two horizontal parts and four vertical parts, and a histogram relating to the edge direction is created for each area. For example, the edge direction is set to nine directions, and a histogram 702 in which the edge strength is weighted for each edge direction in each region is created. Note that since the edges that are inverted by 180 degrees are regarded as the same direction, the directions that the histogram 702 can take are 0 to 180 degrees, and when assigned to 9 directions, the increments are 20 degrees.

  Thus, the histogram 702 is created in each region. The bins of the histogram 702 form a 72-dimensional vector with 9 directions × 8 regions. Finally, normalization is performed so that the norm of the 72-dimensional vector is 1. The vector created in this way is used as the feature amount X of the detection window region.

  The learning discriminator 632 determines whether the detection window area includes “person” or “other than person” based on the feature amount X. In order to determine whether or not a person is a person from the obtained feature quantity X, the probability P (person | X) that is “person” when the feature quantity X is obtained, and the “person” when the feature quantity X is obtained. The probability P (non-human | X) that is “non-human” is calculated, and if P (person | X)> P (non-human | X), it is determined as “human”, otherwise it is determined as “non-human”.

The calculation of P (person | X) and P (non-person | X) can be obtained by the formulas (5) and (6) from Bayes' theorem.

  Here, P (X | person) is the probability (likelihood) that the feature quantity X is obtained for “person”, and P (X | other) is the feature quantity X for “non-person”. Probability (likelihood). P (person) is a prior probability that indicates the probability of “person” being included in the detection window area, and similarly P (other than person) is included in the detection window area with a probability of “non-person”. It is a prior probability that represents whether or not. Since the probability that “person” is included in the detection window region is not known in advance, it is assumed that the prior probability P (person) = P (other than person) = 0.5. Since P (non-person | X) = 1−P (person | X), if P (person | X) is obtained, P (non-person | X) can be easily obtained. Hereinafter, P (person | X) is referred to as the similarity in the learning discriminator 632.

  In the present embodiment, the distribution of P (X | people) and P (X | people) is expressed as a mixed normal distribution. The feature quantity distribution P (X | person) of “person” calculates the feature quantity from prepared learning data of “person”, and uses the EM (Expectation-Maximization) algorithm (AP Dempster, NM) Laird, and DB Rubin. “Maximum likelihood from incomplete data via the EM algorithm”, Journal of the Royal Statistical Society, Series B, 39 (1), 1-38, 1977). The feature quantity distribution P (X | non-person) of “non-human” calculates the feature quantity from a plurality of images taken on the assumption that there is no “person” at the time of installation of the surveillance camera, and the mixed normal distribution by the EM algorithm Determine the parameters.

  After the similarity is obtained by the learning discriminator 632 in FIG. 11, the size (width, height) of the detection window region and the center coordinates and the similarity obtained by the learning discriminator 632 are obtained by the learning similarity management unit 634. The determination information history 40 is associated and stored in the storage unit 4.

  Next, the discriminator update unit 636 updates the learning discriminator 632 based on the similarity of the cascade discriminator 620, the number of determination stages, and the result of the similarity of the learning discriminator 632. In the present embodiment, the discriminator update unit 636 only selects “person” based on the number of determination stages and the similarity of the cascade discriminator 620 and the similarity of the learning discriminator 632. "Non-other" feature quantity distribution P (X | other than people) is updated. However, the feature amount distribution P (X | person) of “person” may be updated only when the possibility of “person” is high.

  The “non-human” feature quantity distribution P (X | non-human) is modeled as a mixed normal distribution, and the feature quantity distribution P (X | human) is obtained each time one piece of data determined as “non-human” is obtained. Radford Neal, RM and GE Hinton (1998) "A new view of the EM algorithm that justifies incremental, sparse and other variants", Learning in Graphical Models, pp.355-368. Kluwer Academic Publishers ) To update.

  In the present embodiment, the learning discriminator 632 includes a slow fluctuation discriminator and a sudden fluctuation discriminator, and a feature amount distribution P (X | non human) is set for each of them. . For the discriminator for slow fluctuation and the discriminator for sudden fluctuation, the feature quantity distribution 42 for learning discriminators other than “person” is used. Here, the update interval of the “non-human” feature quantity distribution P (X | non-person) in the slow fluctuation classifier is the “non-human” feature quantity distribution P (X | non-person) in the sudden change classifier. Set longer than the update interval. The slow fluctuation discriminator has a longer update interval than the sudden fluctuation discriminator, so that the influence of past update contents remains longer than the sudden fluctuation discriminator in the feature distribution P (X | non human) of “non-human”. It is possible to cope with environmental changes that slowly change over a long period of time. On the other hand, the update interval of the sudden change classifier is shorter than that of the slow change discriminator. Therefore, the influence of the past update contents in the feature amount distribution P (X | non person) of “non-human” in a shorter time than the slow change discriminator. It is possible to cope with environmental changes that fluctuate rapidly in a short time.

  The determination unit 38 determines the size (width, height) and center coordinates of the detection window region stored as the determination information history 40, the number of determination stages and the similarity of the cascade type classifier 620, and the similarity of the learning classifier 632. To finally determine where the person is in the image.

  Among the data included in the determination information history 40, the number of determination stages of the cascade type discriminator 620 is the final stage, the similarity is larger than a threshold (for example, set to 0), and the discriminator for slow fluctuations of the learning discriminator 632 All of the similarity calculated in step S is equal to or greater than a threshold (for example, set to 0.8), and the similarity calculated by the rapid change classifier of the learning classifier 632 is equal to or greater than the threshold (for example, set to 0.8). The detection window area satisfying the above condition is extracted as a human candidate area. If there is no human candidate area, the process ends because there is no human area in the input image.

  When there is a human candidate area, the human candidate areas that overlap a certain area (for example, more than half of the area of the detection window area) are collected using the size and center coordinates of the detection window area. A detection window region having the highest similarity calculated by the cascade discriminator 620 is selected as a human region among the grouped human candidate regions. The detection window area selected for each collected area is set as a human area, and information on the detection window area selected is output to the output unit 5. The selection as a human region may use the similarity of the learning discriminator 632 without using the similarity of the cascade discriminator 620.

  The storage unit 4 is configured by a memory device such as a ROM or a RAM. The storage unit 4 is connected so as to be accessible from the signal processing unit 3. The storage unit 4 can store various programs and various data, and reads and writes these pieces of information in response to requests from the signal processing unit 3. The storage unit 4 functions as the learning similarity management unit 634 and stores the determination information history 40 and the learning discriminator feature amount distribution 42. The determination information history 40 includes the size (width, height) and center coordinates of the detection window region, the number of determination stages and the similarity of the cascade type discriminator 620, and the similarity of the learning discriminator 632 (calculated by the discriminator for slow fluctuation). Information of similarity and the similarity calculated by the sudden change classifier) are stored in association with each other. The learning identifier feature quantity distribution 42 is a human feature quantity distribution P (X | person) and a non-person feature quantity distribution P (X | other person). Specifically, parameters relating to the mixed normal distribution with “people”, that is, the mixing ratio, average value, and covariance matrix of each normal distribution are stored. In addition, as parameters related to the mixed normal distribution of “non-human”, the mixing ratio, average value, covariance matrix of the classifier for slow fluctuation, and the mixing ratio, average value of the mixture normal distribution of the classifier for sudden fluctuation A covariance matrix is stored. The learning discriminator feature quantity distribution 42 corresponds to the dictionary data, the slow discriminating classifier feature quantity distribution is the first dictionary data, and the sudden fluctuation discriminator feature quantity distribution is the second. It corresponds to each dictionary data.

  Hereinafter, the process in this Embodiment is demonstrated. Since the processing in the present embodiment is almost the same as the processing shown in the flowchart of FIG. 6, only different steps S40, S50, and S70 will be described.

  In step S40, a similarity indicating how much the image in the detection window area set in step S30 resembles a person is obtained. This is processing in the cascade discriminator 620 and learning discriminator 632 shown in FIG. 11, and will be described with reference to the flowchart of FIG. Steps S451 to S455 are processing in the cascade discriminator 620, and steps S456 to S459 are processing in the learning discriminator 632.

  In step S451, the HOG feature amount is extracted from the current detection window region. However, this process is faster between steps S10 and S20 by calculating the edge strength and angle of each pixel of the input image and creating an integral image for each edge angle. It becomes possible.

  In step S452, it is determined whether or not all strong classifiers have been investigated. If all strong classifiers have been investigated, the process proceeds to step S455, and if not complete, the process proceeds to step S453, and the next strong classifier is investigated. In step S453, the similarity for determining whether the image in the detection window area is similar to a person is calculated.

  In step S454, it is determined whether the similarity calculated in step S453 is greater than a preset threshold (for example, set to 0). If it is greater than the threshold value, the process proceeds to step S452, and if it is less than the threshold value, the process proceeds to step S455.

  In step S455, the similarity calculated last in step S453 and the number of determination steps thereof are stored in the storage unit 4 as the determination information history 40 in association with the size (width, height) of the detection window region and the center coordinates. Here, the similarity calculated last becomes the similarity of the fixed classifier. In step S459, the similarity of the learning discriminator 632 is also stored.

  In step S456, the feature quantity X used in the learning discriminator 632 is obtained. This is as described above.

  In step S457, the similarity of the slowly varying discriminator of the learning discriminator 632 is calculated based on the feature amount X obtained in step S456. The degree of similarity P (person | X) can be obtained from Equation (5), and the parameter of the mixed normal distribution of the classifier for slow fluctuation is used as the feature amount distribution P (X | other than person) other than people.

  In step S458, the similarity of the discriminator for sudden change of the learning discriminator 632 is calculated based on the feature amount X obtained in step S456. The degree of similarity P (person | X) can be obtained from Equation (5), and the parameter of the mixed normal distribution of the classifier for sudden change is used as the feature quantity distribution P (X | other than person) other than people.

  In step S459, the similarity between the slow change discriminator and the sudden change discriminator calculated in steps S457 and S458 is associated with the similarity of the fixed discriminator stored in step S455 as the determination information history 40 in the storage unit 4. Remember. Thereafter, the process proceeds to step S50.

  In step S50, the learning discriminator feature quantity distribution is updated. This is performed by the discriminator update means 636. The update process is performed according to the flowchart shown in FIG. In the present embodiment, the feature quantity distribution of “non-person” is updated, and the feature quantity distribution of “person” is not updated.

  In step S551, it is determined whether or not it is “other than a person”. “Non-human” is a step when both the conditions that the number of determination stages of the cascade classifier 620 is equal to or smaller than the threshold T1 (1 ≦ T1 ≦ N) and the similarity of the cascade classifier 620 is smaller than the threshold T2 are satisfied. The process proceeds to S552, and if not, the process proceeds to step S60.

  In step S552, when the similarity of the classifier for slow fluctuation of the learning classifier 632 is smaller than the threshold value T3, or the condition that the similarity of the classifier for sudden fluctuation of the learning classifier 632 is smaller than the threshold value T4 is satisfied Shifts to step S553, otherwise shifts to step S60.

  The determinations in steps S551 and S552 are for updating the learning discriminator feature quantity distribution 42 in the case of “other than human”.

  In step S553, the feature amount distribution 42 of “non-human” is updated. Using the feature amount X obtained from the detection window region at the current time, the mixed normal distribution P (X | other than people) corresponding to the classifier for slow fluctuation and the person corresponding to the classifier for sudden fluctuation corresponding to the classifier for slow fluctuation. The mixed normal distribution P (except for X | people) is updated using the sequential EM algorithm.

  However, the mixed normal distribution P (X |) other than the “non-human” corresponding to the sudden change classifier is more than the mixed normal distribution P (X | other) than the “non-human” corresponding to the slow fluctuation classifier. Also update in a short cycle. For example, the mixed normal distribution P (X |) other than the person corresponding to the sudden change classifier is updated every time, and the mixed normal distribution P (X |) other than the person corresponding to the slow fluctuation classifier is updated. The person other than the person does not update if a certain period of time has not passed since the last update. Thereafter, the process proceeds to step S60.

  In step S70, an area in which a person is finally captured is determined based on the determination information history 40 obtained in step S50. This processing is performed by the determination means 38. The process in step S70 differs from the process in the first embodiment along the flowchart shown in FIG. 10 only in the condition in step S701. In step S701, the number of determination stages of the cascade discriminator 620 is the final stage, the similarity of the cascade discriminator 620 is greater than the threshold, the similarity of the slow fluctuation discriminator is greater than the threshold, Only when all the conditions that the degree of similarity is greater than the threshold value are satisfied, it is determined as a human candidate region in step S702.

[Initial settings]
The initial setting process when the surveillance camera is installed in the present embodiment will be described below. At the time of initial setting, on the assumption that there is no person in the imaging area of the surveillance camera, the feature distribution for learning discriminator 42: P (X | other than person) of “non-person” is created. The feature quantity distribution P (X | person) for the learning discriminator “person” is created by obtaining the mixture ratio, average, and covariance matrix of each normal distribution of the mixed normal distribution from previously collected learning data.

  In the initial setting process, a feature quantity X is obtained from a predetermined number of images after the monitoring camera is installed, and the feature quantity distribution P for learning discriminator P (X | non-person) is determined from these feature quantities X. And finish. It should be noted that the feature distribution P for learning discriminators other than “human” (X | other than human) created in the initial setting is used as the feature distribution P for slow fluctuation discriminators P (X | other than human) and the discriminator for sudden fluctuations. It is assumed that the feature quantity distribution P (X | other than people).

  The initial setting process is performed according to the flowchart of FIG. Steps S101 to S103 are the same as steps S10 to S30. In step S <b> 104, a feature quantity for learning discriminator is created and stored in the temporary storage area of the storage unit 4. Step S105 is the same as step S60.

  In step S106, it is determined whether or not a predetermined number of images have been processed. If a predetermined number of images have been processed, the process proceeds to step S107. Otherwise, the process returns to step S102.

  In step S107, the feature amount X accumulated in step S104 is used to estimate the feature amount distribution 42: P (X | non-person) of “non-person”. The feature amount distribution P (X | other than people) is assumed to be represented by a mixed normal distribution composed of a sum of a predetermined number of normal distributions, and the mixture ratio, average, and variance of each normal distribution are obtained by an EM algorithm.

  Moreover, although the said embodiment demonstrated the aspect which implement | achieves the function of each part of the object detection apparatus 1 with one computer, it is not limited to this. The functions of each part of the object detection device 1 can be realized by controlling a general computer by a program, and the functions of these devices may be appropriately combined and processed by a single computer. Distributed processing may be performed by a plurality of computers connected via a network or the like.

  DESCRIPTION OF SYMBOLS 1 Object detection apparatus, 2 Image acquisition part, 3 Signal processing part, 4 Memory | storage part, 5 Output part, 30 Detection window area | region selection means, 32 Detection window control means, 34 Fixed discriminator, 36 Learning discriminator, 37 Discriminator update Means, 38 judgment means, 40 judgment information history, 42 learning feature distribution (similarity histogram), 200 input image, 201, 202 rectangle, 203, 204 area, 310 detection window cutting means, 320 cascade type discriminator , 321, 322, 323 strong classifier, 330 learning classifier, 332 classifier update unit, 334 similarity management unit, 401,411 similarity histogram, 402,403 distribution, 411 similarity histogram, 412, 413 distribution, 501 , 502, 503 Histogram, 620 Fixed classifier, 621, 622, 623 Strong classifier, 6 30 judgment information history means, 632 learning discriminator, 634 learning similarity management means, 636 discriminator update means, 702 histogram.

Claims (4)

An object detection device for detecting a detection target from an image obtained by imaging a monitoring area,
An image acquisition unit for acquiring the image;
Taking a detection window area image in which a detection window is set from the image input from the image acquisition unit, calculating a similarity indicating a degree of similarity between the detection window area image and a detection object, A fixed classifier that determines whether or not a detection target exists in the detection window region image based on similarity based on information learned in advance;
A slow learning discriminator that determines whether or not a detection target exists using the first dictionary data updated with information calculated from the detection window region image;
Using the second dictionary data updated by the information calculated from the detection window area image so that the influence of past update contents is reduced in a shorter time than the first dictionary data, the detection target exists. A sudden change learning classifier that determines whether or not
Determining means for determining whether or not a detection target exists based on outputs from the slow learning classifier and the sudden change learning classifier;
An object detection apparatus comprising:   2. The slow learning discriminator and the sudden change learning discriminator make a determination only on a detection window region image determined by the fixed discriminator that a detection target exists. The object detection apparatus described in 1.   3. The object according to claim 1, wherein the determination unit determines that a detection target exists when it is determined by the slow learning classifier and the sudden change learning classifier that the detection target exists. Detection device.   The slow learning discriminator and the sudden change learning discriminator update the first dictionary data and the second dictionary data only with the similarity calculated when it is determined that the detection target does not exist. The object detection device according to claim 1, wherein the object detection device is an object detection device.

Images