JP2017097815A - Information processing device, information processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To detect a specific subject appearing in a moving image at high speed on the basis of a compressed range image.SOLUTION: Provided is an information processing device comprising: setting means for setting a warning space for detecting trespassing of a subject in a monitoring space; acquisition means for acquiring compressed data that is obtained by encoding a range image in an encoding format in which encoding per pixel and encoding per multiple pixels are mixed; estimation means for estimating whether a codeword in the compressed data that is to be decoded has been encoded once for each pixel or encoded once for multiple pixels; and determination means which, when encoding per pixel is estimated by the estimation means, decodes the codeword to be decoded, and for one pixel having distance information, determines whether there is a change to the distance information in the warning space, and when encoding per multiple pixels is estimated by the estimation means, decodes the codeword to be decoded, and for multiple pixels having distance information, determines whether there is a change to the distance information in the warning space.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a method for detecting a specific subject in a moving image based on distance information to the subject.

  2. Description of the Related Art Conventionally, there is known a system that uses an imaging device called a network camera to determine whether a suspicious person has entered a shooting target space from a captured moving image. For detecting such a specific subject, there is a method using a distance image obtained by measuring the distance from the imaging device to the subject. Patent Document 1 discloses a method for determining whether or not a moving object is within a predetermined range from the imaging device based on the distance to the detected moving object and the distance of the moving object to be detected when the moving object is monitored by the imaging device. doing. On the other hand, there has also been proposed an environment recognition device that accurately recognizes significant information in an environment to be captured using images obtained from a plurality of cameras and distance images obtained by configuring those cameras as stereo cameras. (Patent Document 2).

  In addition, the moving image obtained by imaging is compressed as data in order to reduce the amount of data when transmitting data to other information processing devices via the network and to reduce the amount of data stored in the memory. Often handled. From the same point of view, it is desirable to handle the distance image obtained by measuring the distance to the subject with the imaging device as the viewpoint as compressed data. 3D Video Coding (hereinafter referred to as 3DV) has been developed as a standard technique for encoding distance images. In 3DV, a distance image for performing free viewpoint video composition with high image quality is generated. However, H. As with the RGB image encoding technology such as H.264, frequency conversion is performed, so that large degradation is likely to occur near the edge of the distance image. A compression method in which a large amount of deterioration occurs in some pixels is a serious problem depending on how the range image is used. For example, in the method disclosed in Patent Document 2, if distance information compressed using a compression method in which some of the pixels are greatly deteriorated is used, there is deterioration due to the compression, so that an image is captured in the environment. It becomes difficult to accurately recognize significant information. Therefore, the distance image is encoded such that a maximum distortion (error) of each pixel value due to encoding can be suppressed to an arbitrary value, as in a known compression method called JPEG-LS (Non-Patent Document 1). It is desirable to use a method.

JP 11-196320 A JP 2010-224918 A

The LOCO-I Lossless Image Compression Algorithm: Principles and Standardization into JPEG-LS, (IEEE TRANSACTION ON IMAGE PROSINGING, VOL.9, NO.

  In order to detect a specific subject by referring to the compressed data obtained by encoding the distance image, it is necessary to first decode the compressed data. After that, it takes time to detect the intrusion of the subject based on the distance image obtained by decoding. Particularly when frame images constituting a moving image are sequentially input and intrusion of a subject is detected, particularly high-speed processing is required.

  Accordingly, an object of the present invention is to detect a specific subject appearing in a moving image at high speed and more appropriately based on a compressed distance image.

In order to solve the above-described problem, an information processing apparatus according to the present invention provides a distance image in an initial state of the monitoring space and a subject in a monitoring space in which distance information from the imaging device to the subject can be obtained by imaging with the imaging device. Obtained by encoding a distance image consisting of distance information for each pixel by a setting means for setting a warning space for detecting an intrusion of the image and an encoding format in which encoding for each pixel and encoding for each of a plurality of pixels are mixed. An acquisition means for acquiring compressed data, and whether a codeword to be decoded for the compressed data is a codeword encoded for each pixel or a codeword encoded for each of a plurality of pixels Estimating means for estimating;
Decoding the codeword to be decoded to obtain distance information, and according to the result estimated by the estimation means, for the pixel position of the distance information, the distance information changes in the distance information in the alert space. Determining means for determining whether or not there is a distance information by decoding the decoding target codeword when the estimating means estimates that the codeword is encoded for each pixel; And determining whether or not there is a change in the distance information in the alert space based on the distance information and the initial state of the monitoring space, and the estimating means encodes each pixel If it is estimated that the code is a decoded code, the distance in the alert space is determined based on the distance information and the initial state of the monitoring space for a plurality of pixels having distance information by decoding the code to be decoded. And judging whether there is a change of information.
It is characterized by that.

  According to the present invention, a specific subject appearing in a moving image can be detected at high speed and appropriately based on a compressed distance image.

The figure which shows an example of monitoring space, a monitoring target, and an imaging device. The figure which shows the example of the alert space in the monitoring space of FIG. 3 is a block diagram showing a functional configuration of the information processing apparatus 3. FIG. The figure explaining a moving image and a distance image. The figure which shows the structural example of a computer system. The flowchart of the process in the case of implement | achieving the function of information processing apparatus on a computer system. The flowchart which shows the detail of a detection process. The figure which shows the structural example of the apparatus which produces | generates the data of the moving image and distance image used as the input to information processing apparatus. The figure which shows the structural example of the distance image encoding part 880 in FIG. The figure explaining Golomb-Rice encoding. The figure which shows the positional relationship of a focused pixel and its surrounding pixel. The figure which shows the relationship between the parameter k of Golomb-Rice encoding, the integer value MV, and a code word.

  Preferred embodiments of the present invention will be described below with reference to the accompanying drawings. The embodiment described below is merely an example when the present invention is specifically implemented, and the present invention is not limited to the illustrated configuration.

<First embodiment>
In the first embodiment, an information processing apparatus that determines whether or not there is a subject that approaches a monitoring target in a moving image obtained by imaging an imaging device in a space in which the monitoring target is arranged will be described. A monitoring space is imaged continuously by an imaging device to obtain a moving image. Here, the moving image is data composed of a plurality of time-series frame images. The imaging device performs predetermined image processing and encoding on the imaging result, and transfers the compressed data of the moving image and the compressed data of the corresponding distance image sequence to the information processing device. The information processing device is a monitoring device that exists in a location away from the monitoring space and the imaging device, and analyzes the transferred compressed data to detect an intruder into the alert space.

  In FIG. 1A, a monitoring object 12 is arranged in a monitoring space 11. In such a monitoring space, a moving image is picked up by two image pickup devices of the main camera 100 and the sub camera 101. When the main camera 100 and the sub camera 101 image the monitoring space 11, two image sequences are obtained as shown in FIG. One is a moving image composed of a frame image sequence, and the frame image is color image data represented by RGB. Data obtained from the main camera 100 is used for the moving image. The other image sequence is a distance image sequence, and is composed of distance images corresponding to each frame image of the moving image. Each distance image is an image that holds the distance information from the camera position to the subject in the corresponding frame image for each pixel. Each distance image is calculated by a known method using the corresponding frames of the main camera 100 and the sub camera 101 as two stereo images having different parallaxes. As described above, the moving image and the distance image sequence are data corresponding to each frame image and the distance image, and each pixel position in the frame image of the moving image is associated with each pixel position in the distance image.

  As shown in FIG. 1B, the camera 102 and the range sensor 103 may be combined to generate a distance image. In this case, a moving image obtained from the camera 102 is a moving image, and distance measurement data obtained from the range sensor 103 is a distance image sequence.

  A moving image and a distance image sequence obtained by imaging with each camera are encoded in a predetermined compression format and converted into compressed data in a predetermined format. Here, FIG. 8 is a block diagram illustrating a logical configuration of an image processing apparatus in which compressed data of a predetermined format is generated based on a moving image obtained from the imaging apparatus. In the figure, a main camera 100 and a sub camera 101 are the cameras shown in FIG. Each camera outputs a moving image obtained by imaging the monitoring space 11 including the monitoring target 12. The image acquisition unit 830 acquires a moving image composed of a plurality of frame images output from the main camera 100 and the sub camera 101 and outputs the moving image to the image processing unit 840 for each frame image. The image processing unit 840 generates a distance image corresponding to the frame image by using the frame image captured by the main camera 100 and the frame image captured by the sub camera 101 as a stereo image. Since the distance image is generated for each frame, the distance image is also output as a distance image sequence corresponding to the moving image. The frame image sequence generated by the image processing unit 840 is temporarily stored in the moving image memory 850, and the distance image sequence is temporarily stored in the distance image memory 860.

  The moving image encoding unit 870 encodes the frame image sequence held in the moving image memory 850, converts the frame image sequence into compressed data, and outputs the compressed data to the output unit 890. The distance image encoding unit 880 encodes each distance image sequence held in the distance image memory 860, converts it into compressed data, and outputs the compressed data to the output unit 890. The output unit 890 outputs the encoded frame image sequence obtained from the moving image encoding unit 870 to an information processing apparatus that detects an intruder in the present embodiment through a communication line (not shown). Similarly, the output unit 890 outputs the encoded distance image sequence obtained from the distance image encoding unit 880 to the information processing apparatus through a communication line (not shown).

  Next, a configuration example of the above-described distance image encoding unit 880 will be described with reference to FIG. As described above, when a pixel value whose distance information is greatly degraded by compression is used, the accuracy of determination using the distance image may be deteriorated. Therefore, it is desirable to use an encoding method that can suppress the maximum distortion of each pixel value due to compression to an arbitrary value. Therefore, in the present embodiment, the distance image encoding unit 880 encodes the distance image using a known encoding method called JPEG-LS for the distance image. JPEG-LS is a method that encodes pixel values of each pixel to be encoded in a pixel unit by a technique called predictive encoding, and a run-length encoding of a pixel sequence composed of a plurality of continuous pixels having the same pixel value. This is a coding format in which a method of coding by a method called “a” is mixed.

  FIG. 9 is a block diagram showing a more detailed logical configuration of the range image encoding 880. The distance image input unit 910 sequentially inputs distance images from the distance image sequence held in the distance image memory 860. The distance image input unit 91 inputs the pixel value of each pixel constituting the distance image to the mode selection unit 920 in line units in raster scan order. The mode selection unit 920 sequentially analyzes the line-by-line pixel value sequence acquired from the distance image input unit 910 from the top pixel value of the line, and determines whether or not run-length encoding is to be performed. Here, regarding the pixel determined to be run-length encoded, the pixel value of the pixel is output to the run-length encoding unit 940. Otherwise, the pixel value of the pixel is output to Golomb encoding section 930. The Golomb encoding unit 930 generates a code word obtained by Golomb-Rice encoding the pixel in units of pixels under the control of the mode selection unit 920. In addition, the run-length encoding unit 940 generates a code word obtained by encoding the pixel as a group of a plurality of pixels on the same line including adjacent pixels. The code generation unit 950 combines the codeword generated by the Golomb encoding unit 930 and the codeword generated by the run-length encoding unit 940 to generate a codeword sequence for a pixel value sequence for one line. The data is output to the code output unit 960. The code output unit 960 sequentially forms code data of a distance image composed of a plurality of lines arranged in the raster scan order from the code word string for one line output from the code generation unit 950, and outputs the code data to the output unit 890.

  An example in which the distance image encoding unit 880 described above is realized by software processing on a computer system will be described with reference to the hardware configuration diagram of the computer system in FIG. 5 and the flowchart in FIG. 10. The CPU 501 functions as a central processing unit, and controls the operation of the entire image forming system using input data and computer programs stored in a RAM 502 and a ROM 503 described later. The RAM 502 has a storage area for temporarily storing computer programs and data read from the external storage device 507 and data received from the outside via the I / F unit 509. It has a storage area for temporarily storing computer programs and data read from the external storage device 507 and data received from the outside via the I / F unit 509. A ROM 503 is a memory that can be read only by random access, and stores setting parameters, a boot program, and the like for setting each unit in the image forming system. A keyboard 504 and a mouse 505 are pointing devices and accept input of instructions from the user. A display device 506 typified by a liquid crystal display displays a digital image. An external storage device 507 typified by a hard disk or the like is a large-capacity external storage device. The storage medium drive 508 drives a removable storage medium. An I / F unit 509 is an interface for exchanging data with an external device. Each of the above units is connected to the bus 510 and exchanges data via the bus 510.

  FIG. 10 is a flowchart for executing encoding of a distance image. The distance information encoding unit 880 can be realized by storing a program describing the processing flow shown in FIG. 10 in the RAM 502 or the ROM 503 in advance, and the CPU 501 executing the program. In FIG. 10, when the process is started, in step S101, the distance image input unit 910 identifies the first distance image in the time series as the first frame in the distance image sequence, and sets the top line in the distance image of the first frame as the first frame. Set as line. Next, in step S102, the distance image input 910 determines whether or not the processing of the last line of the distance image to be processed has been completed. If it has been completed, the process proceeds to step S103. If not, the process proceeds to step S105.

  In step S103, it is determined whether or not the processing of the last frame in the time series in the distance image sequence has been completed. If it has been completed, a series of distance information encoding is completed, and if not, Advances to step S104. In step S104, the distance image input unit 910 sets the processing target as the first line of the distance image corresponding to the next frame, and returns to step S102. In step S105, the distance image input unit 910 inputs the pixel value sequence of the pixel sequence on the processing target line, and proceeds to step S106. Pixel values in the distance image are input in units of lines and sequentially encoded in raster scan order.

  In step S106, the mode selection unit 920 sets the pixel located at the head of the pixel value sequence input in step S105 as the target pixel, and proceeds to step S107. In step S107, the mode selection unit 920 selects an encoding mode for the pixel of interest. As encoding modes, there are a pixel encoding mode and a run-length encoding mode. A method of selecting the encoding mode in step S107 will be described with reference to FIG. FIG. 11 shows the positional relationship between the pixel of interest and the peripheral pixels. Here, the pixel x is a pixel of interest and is a pixel to be encoded. Further, since the pixels are encoded in the raster scan order, the left pixel a, the upper left pixel c, the upper pixel b, and the upper right pixel d are already encoded pixels. In the initial setting, the run length RL = 0 is set. When the run length RL is other than 0, or when the surrounding pixels satisfy the run-length encoding condition, the mode selection unit 920 selects the run-length mode. The run-length encoding condition is that the pixel values of the left pixel a, the upper left pixel c, the upper pixel b, and the upper right pixel d are a = c, c = b, and b = d. Therefore, when the pixel values of the peripheral pixels satisfy a = c, c = b, and b = d, the mode selection unit 920 selects the run-length encoding condition. In other cases, the mode selection unit 920 selects a pixel encoding mode. In step S108, the mode selection unit 920 determines whether or not the encoding mode is set to the run-length mode in step S107. If the encoding mode is set to the run-length mode, the process proceeds to step S109. Otherwise, the process proceeds to step S110.

  In step S109, the run-length encoding 940 performs run-length encoding. When the pixel of interest x shown in FIG. 11 has the same pixel value as that of the left pixel a, the run length RL is increased by 1, and the process proceeds to step S112. When the target pixel x has a pixel value different from that of the left pixel a, the run length RL so far is Huffman-coded, the run length RL is initialized to 0, and the process proceeds to step S112. Note that run-length encoding is a known technique and, for example, is the same processing as JPEG-LS, and thus detailed description thereof is omitted.

  In step S110, Golomb coding 930 predictively converts the pixel value of the pixel. The prediction conversion uses MED (Media Edge Detection) prediction. The prediction formula using the peripheral pixels in FIG. 11 is as shown in Formula (1).

ここで着目画素の予測誤差は式（２）である。
予測誤差（Ｄｉｆｆ）＝着目画素ｘの画素値−予測値ｐ （２）
予測変換を終えるとステップＳ１１１へ進む。

Here, the prediction error of the pixel of interest is Equation (2).
Prediction error (Diff) = pixel value of target pixel x−predicted value p (2)
When the prediction conversion is completed, the process proceeds to step S111.

  In step S111, the Golomb coding 930 performs Golomb-Rice coding on the prediction error of the pixel of interest calculated in Step S110. First, the prediction error (Diff) is converted into a non-negative integer value (MV). The conversion formula is as shown in Formula (3).

次に、パラメータｋを用いて非負の整数値（ＭＶ）をゴロム・ライス符号化する。ゴロム・ライス符号化の手順は以下の通りである。
（１）ＭＶを２進数表現して、ＭＶをｋビット右シフトした値の０を並べ、その後に１を付加する。
（２）（１）の後ろに、ＭＶの下位ｋビットを取りだして付け加える。

Next, Golomb-Rice coding is performed on the non-negative integer value (MV) using the parameter k. The procedure for Golomb-Rice coding is as follows.
(1) MV is expressed as a binary number, 0's of values obtained by right-shifting MV by k bits are arranged, and 1 is added after that.
(2) The lower k bits of MV are extracted and added after (1).

  FIG. 12 shows the relationship between the Golomb-Rice coding parameter k, the non-negative integer value (MV), and the code word. The configuration of Golomb-Rice coding is not limited to this. For example, codes may be configured by reversing 0 and 1, and the order of (1) and (2) described in the above procedure may be changed. The codes may be configured by replacing them. Here, the method for determining the encoding parameter k is not particularly specified, but it is sufficient that the same parameter can be used on the encoding side and the decoding side. When the process of step S111 is completed, the process proceeds to step S112.

  In step S112, it is determined whether or not the encoding process has been completed up to the last pixel of the line being processed. If completed, the process proceeds to step S114. If not, the process proceeds to step S113. In step S113, the pixel to be processed is set to the next pixel on the same line, and the process returns to step S107. In step S114, the codeword obtained from the Golomb coding unit 930 and the codeword obtained from the run-length coding unit 940 are combined to generate a codeword string for the processing target line. Next, in step S115, the distance information input unit 910 sets the pixel to be processed as the first pixel of the next line, and returns to step S102. The example in which the distance information encoding unit 880 is realized by software processing on the computer system has been described above.

  The configuration of the distance information encoding unit 880 described above can also be applied to the moving image encoding unit 870. In the present embodiment, the moving image encoding unit 870 in FIG. 8 also generates a frame image sequence composed of color images expressed by a plurality of components in a known color space such as RGB and YCbCr held in the moving image memory 850. Each color component is encoded as a monochrome image. It is assumed that the encoded compressed data is output to the output unit 890 in this way.

  From here, the information processing apparatus according to the present embodiment will be described. FIG. 3 is a block diagram illustrating a logical configuration of the information processing apparatus 3. The information processing apparatus 3 includes a moving image acquisition unit 301, a distance image acquisition unit 302, an initial space calculation unit 303, a warning space setting unit 304, a mode estimation unit 305, a coordinate calculation unit 306, a determination unit 307, and a warning control unit 308. The moving image acquisition unit 301 sequentially acquires compressed data obtained by encoding a moving image (frame image sequence), and the distance image acquisition unit 302 sequentially acquires compressed data obtained by encoding a distance image. Both the frame image of the moving image and the corresponding distance image are always acquired. The initial space calculation unit 303 calculates a normal distance image in the monitoring target space. Here, among the acquired compressed data of the distance image, the first frame is decoded as the initial state (no intruder), and the distance information of each pixel is calculated as the initial state.

  The alert space setting unit 305 designates and accepts an area where it is desired to monitor the intrusion of an object in the monitoring space captured by the moving image captured by the user. First, an initial frame image (here, the first frame image) of the moving images held in the moving image memory 850 is displayed on the display device 506. Next, the user uses a pointing device 505 such as a mouse to designate an area to be alerted on the display image. FIG. 4A shows the display image 11, and designated points 41 to 44 indicate four points that define the monitoring area input by the user. The warning space setting unit 305 sets the warning space in the three-dimensional space based on the area specified by the user. FIG. 4B shows an example in which a space represented by a quadrangular prism with the area 40 designated by the user as the bottom is set as a warning space 45. The warning space setting unit 305 expresses the three-dimensional position (three-dimensional coordinates) of the warning space, which is generated from the position of the point in the distance image corresponding to the frame image and the pixel value (distance information) of the position. Output as spatial information. Note that obtaining three-dimensional data from a distance image can be realized by calculating by a method disclosed in Japanese Patent No. 3823559, for example. The two-dimensional pixel position (horizontal position x and vertical position y on the image) of each pixel on the distance image represents the direction within the angle of view as viewed from the distance measuring point (observation point). The pixel value of each pixel is a depth value (z) representing the distance from the distance measuring point (observation point) to the subject in the direction of the pixel position. The above known method is described as a program executable by a computer, and is executed on a computer system as shown in FIG. As a result, the alert space setting unit 305 can generate, as the alert space information, each vertex of the bottom surface that defines the border of the alert space (quadratic column) in the three-dimensional space.

  The mode estimation unit 303 determines whether the codeword of the acquired compressed data of the distance image is encoded by a run-length mode that encodes for each of a plurality of pixels or a pixel encoding mode that encodes for each pixel. Is estimated. The coordinate calculation unit 304 generates three-dimensional coordinates in real space for the pixels in the distance image. The determination unit 306 refers to the distance image and determines whether the distance information of the monitoring space has changed beyond a predetermined condition with respect to the initial state, according to the encoding mode in which the distance image is compressed. If the determination unit 306 determines that there is a change in the distance information, the determination unit 306 considers that there is a subject that has entered the alert space. In this way, it is determined at any time whether there is an intruding subject in the alert space designated by the alert space setting unit 305.

  An example in which the functions of the information processing apparatus shown in FIG. 3 are realized by software processing on a computer system having the configuration shown in FIG. 5 will be described with reference to the flowchart of FIG. Here, as with the above-described distance image encoding unit 880, the function of the information processing apparatus shown in FIG. 3 is the same as that of the computer system described above with reference to FIG. Realize with.

  In FIG. 6, when the process is started, in step S601, the moving image acquisition unit 301 acquires image data obtained by capturing an initial state of the monitoring space in order to calculate an initial setting of the monitoring space including the monitoring target. Here, it is assumed that the compressed data of the first frame in the frame image sequence constituting the moving image is input. The image data input at this time is an image of a state in which a subject to be alerted does not enter the monitoring space.

  In step S602, the distance image acquisition unit 301 acquires compressed data of the distance image corresponding to the frame image input by the moving image acquisition unit 301 in step 601. In step S603, the initial space calculation unit 303 decodes the compressed data acquired in step S601 and displays the decoded data on the display device 506. At this time, the user can confirm the initial state of the monitoring space. The initial space calculation unit 303 also decodes the compressed data of the distance image acquired in step S602, and calculates distance information of each pixel in the initial state.

  In step S604, the warning space setting unit 304 sets a warning space in the monitoring space in accordance with a user instruction. The alert space means a space in which an intrusion of a subject that does not exist in the initial state should be detected. As described above, the alert space setting unit 304 according to the present embodiment generates, as three-dimensional position (three-dimensional coordinate value) information, the vertex of a quadrangular prism having the designated points 41 to 44 shown in FIG. . Referring to the example of FIG. 4, first, from each of the points indicated by the designated points 41 to 44 in FIG. 4A, the points 41 and 42, the points 42 and 43, the points 43 and 44, and the points 44 A quadrilateral is formed by four line segments that circulate as shown in FIG. At this time, in general, each of the four points 41 to 44 may have a coordinate value that does not necessarily form a plane as it is due to a measurement error or a calculation error. Here, it is assumed that the designated points 41 to 44 should be on the same plane, and the description will be continued assuming that the designated points 41 to 44 are corrected if necessary and are set as the coordinate values of the points 41 to 44 again. As a correction method, a floor surface is estimated from a three-dimensional point group obtained from a distance image by a publicly known method disclosed in Japanese Unexamined Patent Application Publication No. 2007-271408, and each of designated points 41 to 44 is on this floor surface. It is assumed that each coordinate value is corrected as a point. In addition, for example, the coordinate value of 44 may be corrected so that 44 is also placed on a plane defined by three points from 41 to 43. In addition, the above-mentioned Japanese Patent Application Laid-Open No. 2007-271408 also discloses that the floor surface estimated from the three-dimensional point group is converted into a coordinate system having the XY axis and the direction orthogonal thereto as the Z axis. . In the present embodiment, an expression obtained by converting the identified floor surface into the coordinate system with the XY axis as the identified floor surface and the Z axis as the direction orthogonal thereto by a known method such as estimation from a three-dimensional point group obtained from the distance image. It may be used. As described above, information that defines a column body composed of a bottom surface that provides a boundary that defines a space to be alerted and a plane group orthogonal thereto is generated as alert space information. The representation of the column body information itself includes, for example, each of the three-dimensional coordinate values of the vertexes of the column body, information regarding the planes that form the column bodies, and the connections between the vertices that configure these planes. The generated information is held in an area not shown in the RAM 502 and different from other information, and the process of step S604 is completed.

  In step S605, the moving image acquisition unit 301 sequentially inputs a moving image captured while monitoring the monitoring space for each frame as a LIVE moving image. The moving image acquisition unit 301 decodes the encoded data to generate a frame image, and holds it in an area not shown in the RAM 502 and different from other information. In step S606, one distance image in the distance image sequence corresponding to the frame image input in step S605 is input and held in a memory area not shown in the RAM 502 and different from other information. A compression method in which data obtained by compressing a distance image includes encoding in units of pixels as described above and encoding in units of runs including a plurality of continuous pixels on the same line having the same pixel value. It is generated by.

  In step S607, the monitoring space at the timing when the input frame image is captured is analyzed, and the presence or absence of an intruder into the alert space is determined. FIG. 7 shows a detailed flowchart of the determination process for the presence or absence of an intruder in step S607.

  First, in step S701, the mode estimation unit 305 encodes the current decoding target data (codeword) in either the pixel encoding mode or the run-length encoding mode from the pixel value of each pixel in the already decoded region. It is estimated whether the data is converted into data. This estimation uses FIG. 11 which is the same as that used when explaining how to select the encoding mode. Regarding the pixel of interest x, the left pixel a, the upper left pixel c, the upper pixel b, and the upper right pixel d, which are surrounding pixels, are already decoded pixels. Therefore, using the pixel value of the decoded pixel, the same determination as in step S107 of FIG. 10 described in the description at the time of encoding is performed. Thereby, it can be estimated in which processing mode the position of interest x was encoded.

  If the estimation result is the pixel encoding mode in step S702, the process proceeds to step S703, and if not, the process proceeds to step S704. In step S704, the code data under attention is decoded as encoded in the run-length encoding mode, and the line length of the run is identified. In step S705, the coordinate three calculation unit 306 obtains the pixel position in the distance image of both end points of the run from the line length of the run obtained in step S704 and the position of the target pixel. Further, by combining the pixel values of the pixels located at these end points, the three-dimensional coordinate values of the end points of the run are obtained by the known method described above.

  On the other hand, in step S703, the coordinate calculation unit 306 decodes the pixel position of interest as having been encoded in the pixel encoding mode. From the position of the target pixel in the distance image and the pixel value obtained by decoding, the three-dimensional coordinate value of the target pixel position is obtained by the above-described known method.

  Next, in step S721, when it is estimated that the pixel encoding mode is set, the determination unit 307 performs an intrusion detection process for a pixel obtained by decoding data according to the pixel encoding mode. From the three-dimensional coordinates of the pixel of interest obtained in step S703 and the alert space information generated in step S604, it is determined whether there is an intruder in the pixel of interest. When the pixel value (distance information) in the target pixel is located in the space defined by the alert space information in the three-dimensional space, the process proceeds to step S707. If the pixel value (distance information) in the target pixel is not located in the space defined by the alert space information in the three-dimensional space, the process proceeds to step S716. In step S <b> 707, the determination unit 307 holds information indicating that the pixel of interest is in the alert space in an area not shown in the RAM 502 and different from other information.

  In step S708 and subsequent steps, processing for determining the presence or absence of an intruder is performed on the data estimated in step S701 as data encoded in the run-length encoding mode. First, in step S708, based on the three-dimensional coordinates of the left end point of the run and the warning space information, whether or not the left end point of the run is in the warning space in the three-dimensional space. Determine. If it is determined that the left end point of the run is in the alert space, the process proceeds to step S710. On the other hand, if the left end point of the run is not in the alert space, the process proceeds to step S709. In step S709, the determination unit 307 determines whether or not the right end point of the run is in the alert space. If it is determined in step S709 that the right end point of the run is not in the alert space, neither the left end point nor the right end point of the run is in the alert space. Since it is not in the alert space, the process proceeds to step S716.

  In step S709, if the determination unit 307 determines that the left end point of the run is not in the alert space but the right end point of the run is in the alert space, the midway from the right end point of the run to the left end point is partially If it is in the alert space, the process proceeds to step S714. In step S714, the determination unit 307 calculates the intersection between the run and the plane that defines the alert space, and the process proceeds to step S715. In step S715, it is determined that each pixel on the run from the intersection of the run obtained in step S714 and the plane defining the alert space to the right end point of the run is in the alert space. Information indicating that a series of pixels in the section on the run are in the alert space is held in a region not shown in the RAM 502 and different from other information. In step S710, when the left end point of the run is in the alert space, the determination unit 307 determines whether the right end point of the run is also in the alert space. If the right end point of the run is also in the alert space, it is determined that all pixels on the run are in the alert space, and the process proceeds to step S713. On the other hand, if it is determined that the right end point of the run is not in the alert space, the process proceeds to step S711. In step S713, information that all the pixels on the run from the left end point to the right end point of the run are in the alert space is held in an area not shown in the RAM 502 and different from other information.

  In step S711, the intersection of the run and the plane that defines the alert space is calculated, and the process proceeds to step S712. In step S712, it is determined that each pixel on the run from the left end point of the run to the intersection of the run obtained in step S711 and the plane defining the alert space is in the alert space. Information that a series of pixels in the section on the run is in the alert space is held in an area not shown in the RAM 502 and different from other information, and the process of step S712 is finished.

  In step S716, the determination unit 307 determines the determination result of the pixel determined not to be in the alert space, and determines that there is no entry into the alert space.

  In step S717, the determination unit 307 compares the pixel value (distance information) of each pixel in the distance image with the pixel value at the same position in the distance image generated in step S603 for the pixel determined to be in the alert space. The initial space information generated in step S613 is distance information when there is no intruder in the alert space. Therefore, in step S717, it is determined whether or not the distance information of the same pixel is greatly different between the initial space information and the distance image being monitored. Here, the distance information may have an error within a predetermined error due to encoding. Therefore, the determination unit 307 determines whether or not there is a difference greater than a predetermined error. The predetermined error is preferably set based on an allowable error set when encoding the distance image. When encoding parameters can be acquired when encoding a distance image, an allowable error that can be generated by encoding is set as a predetermined error with reference to the encoding parameters. However, when an encoding parameter cannot be acquired, an error amount that may occur when encoding a distance image may be determined in advance.

  In the initial space information and the distance image being monitored, if there is a difference greater than a predetermined error in the distance information of the same pixel in the alert space, the pixel position determined to be in the alert space is different from the initial state. It is determined that the subject has entered. Otherwise, it is determined that the situation that originally existed in the alert space in the initial state is detected as it is. The information on the pixel position determined to be in the alert space held in the unillustrated area on the RAM 502 is updated to the state of only the pixel determined to have entered. As a result, when the pixel determined to be in the alert space remains, it is determined that the intrusion into the monitoring target space has occurred, and when it does not remain, it is determined that the intrusion has not occurred. When the process of step S717 is completed, the intrusion detection process corresponding to the encoding process mode is terminated.

  There is a known method for determining whether a point in a three-dimensional space is inside or outside a closed space surrounded by a plane in the same space. -100777. In addition, in the above-described step S717, the determination that the point determined to be in the alert space is in a state different from the initial state has been described using only the distance image, but in the color image at the same pixel position. The determination may also be made using the difference in pixel values between these pixels. The step S607 has been described in detail with reference to FIG.

  Thereafter, returning to the flowchart of FIG. 6, the warning control unit 308 proceeds to step S609 when it is determined in step S608 that there is an intrusion into the warning space in the moving image being monitored, and otherwise, Proceed to step S610. In step S609, the alert control unit 308 executes the control set when there is an entry into the alert space. Here, first, in the frame image obtained by imaging the monitoring space being monitored, the pixel value at the same position as the pixel position determined to have entered in step S717 is used as the corresponding pixel position in the superimposition frame image area. Copy to. The superimposition frame image area is not described, but is assumed to be secured in an area not shown in the RAM 502 and different from other information. In addition, it is assumed that the process has been initialized to include no superimposed pixel value until the process of step S609 is started. The image data of the frame image area for superimposition is data superposed by a known method on the environmental space in the initial state generated in step S603 (intrusion into the alert space has not yet been detected). The alert control unit 308 causes the display device 506 to display the image data of the superimposition frame image area.

  In step S610, the frame image and distance image for one frame of the monitored space acquired in step S631 are processed in raster scan order in step S640, and the process is completed up to the final processing unit of the final scanning line of the frame. Determine whether or not. If completed, the process proceeds to step S620. If not completed, the process returns to step S607, and the process of the next processing unit in the raster scan order is started. In step S620, it is determined whether a series of processing has been instructed from the outside of the apparatus via the I / F unit 509 or by an operator using a pointing device 505 such as a mouse or a keyboard 504. judge. If there is no end instruction, the process returns to step S605 to start the process for the next frame. If there is an end instruction, the series of monitoring processes ends.

  As described above, according to the present embodiment, by encoding the distance image, the amount of data to be transferred from the imaging device to the information processing device that is the monitoring device and the amount of memory required for saving in the information processing device are reduced. Yes. On the other hand, since the intruder is detected based on the distance image, the intruder can be efficiently determined by determining the intruder according to the encoding mode in which the data to be decoded is encoded. Particularly in the case of compressed data encoded in the run length mode, a plurality of pixels on the run can be determined collectively.

<Other embodiments>
In the first embodiment, as the warning space information, four points of information that define the warning area are input by the operator. However, the area to be specified does not necessarily have to be four points. For example, the bottom surface may be a rectangle, and two points that are two vertices on the diagonal may be input. Further, the bottom surface does not necessarily have to be a rectangle, and a plurality of vertices constituting a polygon may be input. In addition, among the subjects existing in the monitoring space in the initial state, the operator may designate a subject that should be alerted to the approach of the subject, and the alerting space may be set so as to include the subject to be alerted.

  Further, as a response when an intrusion occurs, the alert control unit 308 has been described as superimposing and displaying the partial image determined to have intruded on the monitoring space initial image (background image). However, the present invention is not limited to this. Configured to perform predetermined actions such as sounding an alarm, displaying a warning, or reporting warning information when a pixel that is determined to have truly entered is generated You may do it.

  Further, in the above-described embodiment, the description has been given on the basis of the compressed data of the distance image encoded by the encoding method in which the encoding in units of pixels and the encoding in units of runs are mixed. However, it is not limited to this. That is, for example, in a method called block coding in which an image is divided into a plurality of blocks and encoded for each block, the compression is encoded by a coding method in a format in which a plurality of blocks of different sizes are mixed. Data may be used. A region of pixels having the same pixel value in the distance image is classified into a plurality of sizes (for example, 64 × 64, 32 × 32,..., 4 × 4, 2 × 2, 1 × 1) may be based on a method of encoding with the largest possible size. In this case, for example, as determined for each run in the run length mode in the above-described embodiment, it may be configured to determine whether or not each block is included in the alert space. It is possible to determine whether or not the entire block is outside the alert space based on the pixel positions and the pixel values of the four corner pixels of each block. If the entire block is outside the warning space or cannot be determined to be inside the warning space, and partially relates to the inside or outside of the warning space, the entire block may be decoded and processed even if each pixel in the block is decoded. In the case where the determination can be made with, the processing speed can be increased. Even when it partially covers the inside and outside of the alert space, the inside / outside determination may be re-executed in units of sub-blocks of half the vertical and horizontal sizes. In this case, it is possible to realize processing at a higher speed than the case where the entire block is decoded and the inside / outside determination is performed for each pixel.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in the computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (9)

In a monitoring space in which distance information from the imaging device to the subject can be obtained by imaging by the imaging device, a setting unit that sets a distance image in the initial state of the monitoring space, and a warning space for detecting intrusion of the subject,
An acquisition means for acquiring compressed data obtained by encoding a distance image composed of distance information for each pixel in an encoding format in which encoding for each pixel and encoding for each of a plurality of pixels are mixed.
Estimating means for estimating whether a codeword to be decoded in the compressed data is a codeword encoded for each pixel or a codeword encoded for each of a plurality of pixels;
Whether or not there is a change in the distance information in the alert space for the pixel position of the distance information according to the result estimated by the estimation means, by decoding the codeword to be decoded and obtaining the distance information A determination means for determining
The determination means includes
When the estimation unit estimates that the codeword is encoded for each pixel, the distance information and the initial state of the monitoring space are obtained for one pixel having distance information by decoding the codeword to be decoded. To determine whether there is a change in the distance information in the alert space,
When the estimation unit estimates that the code word is encoded for each of a plurality of pixels, the distance information and the initial value of the monitoring space are obtained for a plurality of pixels having distance information by decoding the code word to be decoded. An information processing apparatus that determines whether there is a change in the distance information in the alert space based on a state.   Furthermore, if the determination means determines that there is a change in the distance information in the alert space, it is regarded as indicating that the subject has entered the alert space, and control means for warning that the subject has entered the alert space. The information processing apparatus according to claim 1, further comprising: The encoding for each of the plurality of pixels is run-length encoding.
If the estimation unit determines that the estimation unit is a codeword encoded for each of a plurality of pixels, the determination unit specifies the length of the run, and whether a part or all of the run is in the alert space. And if part or all of the run is in the alert space, the distance information of the pixels in the alert space is the same as the pixels in the alert space. The information processing apparatus according to claim 1, wherein the information processing apparatus determines whether there is a difference greater than a predetermined error and distance information in an initial state.   4. The information processing according to claim 3, wherein the determination unit calculates coordinates in a three-dimensional space of both end points of the run, and determines whether or not the end points of the run are in the monitoring space. apparatus. Further, the acquisition unit acquires a moving image captured by the imaging device in the monitoring space for each frame,
The information processing apparatus according to claim 1, wherein the distance image is a distance image corresponding to each frame of the moving image.   The determination means compares the distance information of the pixels located in the alert space with the distance information in the initial state of the monitoring space, and determines whether there is a difference greater than or equal to a predetermined error. The information processing apparatus according to any one of claims 1 to 5, wherein the information processing apparatus is detected. Furthermore, it has a parameter acquisition means for acquiring a parameter for encoding corresponding to the compressed data,
The information processing apparatus according to claim 1, wherein the predetermined error is set according to the parameter.   A computer program that causes a computer to function as the information processing apparatus according to claim 1 by being read and executed by a computer. In the monitoring space where the distance information from the imaging device to the subject can be obtained by imaging with the imaging device, the distance information of the initial state of the monitoring space and the warning space for detecting the intrusion of the subject are set,
In a coding format in which coding for each pixel and coding for each of a plurality of pixels are mixed, compressed data obtained by coding a distance image composed of distance information for each pixel is obtained,
Estimating whether a codeword to be decoded in the compressed data is a codeword encoded for each pixel or a codeword encoded for each of a plurality of pixels;
Decoding the codeword to be decoded to obtain distance information;
When it is estimated that the code word is encoded for each pixel, one pixel having distance information by decoding the code word to be decoded is based on the distance information and the initial state of the monitoring space. Determine whether there is a change in the distance information in the alert space,
When it is estimated that the code is encoded for each of a plurality of pixels, the plurality of pixels having distance information by decoding the code to be decoded are based on the distance information and the initial state of the monitoring space. And determining whether there is a change in the distance information in the alert space.

Images