JP2015226260A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor and an image processing method capable of preventing increase of the load on the processor and the consumption of electric power by efficiently performing coding of swinging image.SOLUTION: The image processor extracts and encodes one entire image based on a difference S1-S2 between an output S1 from a swing amount detection section 3 which has a gyro sensor 31 and an output S2 from a lowpass filter 4 passing through the output S1, on the other hand, after picking out a part of the entire image and encodes the same after performing a swing correction. With this, the swinging image can be encoded efficiently; and thus the load on the processor 10 and the consumption of electric power are prevented from increasing.

Description

  The present disclosure relates to a video processing apparatus and a video processing method suitable for use in a surveillance camera system having a wearable camera that is worn on a human body.

  In a surveillance camera system, a camera device equipped with a wide-angle lens or a fish-eye lens capable of imaging all directions may be used. As a technique that discloses this type of camera device, for example, one described in Patent Document 1 is known.

  Camera devices used in surveillance camera systems include wearable camera devices that are used by being worn on the clothes of guards or police officers. Some of the wearable camera devices have a function of outputting an entire image (high-resolution image), cutting out a part of the entire image, performing shake correction, and outputting the image.

JP 2004-015766 A

  However, the coding of a shaking image increases the amount of data of the image, so that the coding efficiency is not good (that is, because there is a lot of movement, the compression rate does not increase). Further, when shake correction is performed on the entire image, the amount of data increases, which causes a problem that the load and power consumption of the processor increase.

  The present disclosure has been made in view of such circumstances, and a video processing apparatus and a video processing method capable of suppressing an increase in processor load and power consumption by efficiently encoding a shaking image. The purpose is to provide.

  The video processing device according to the present disclosure is a video processing device mounted on a moving object, and the video processing device includes an imaging unit, a direction detection unit, and a processor, and the direction detection unit includes the moving object. The orientation of the imaging unit in the video processing device mounted on the processor is detected, and the processor is smaller than the first frame image with respect to the first frame image acquired by the imaging unit in a first period A second frame image having a data amount is generated in a second period, the orientation of the imaging unit is received from the orientation detection unit, and when the orientation of the imaging unit is within a predetermined range, the second frame A third frame image having a data amount larger than that of the image is extracted from the first frame image and generated.

  According to the present disclosure, it is possible to efficiently encode a shaking image, and thus it is possible to suppress an increase in processor load and power consumption.

1 is a block diagram illustrating a schematic configuration of a video processing apparatus according to an embodiment of the present disclosure. Waveform diagram showing an example of the output of the shake amount detection unit and the output of the low-pass filter of the video processing apparatus of FIG. The figure which shows an example of the process which designates the image to encode in the video processing apparatus of FIG. The block diagram which shows an example of the structure which implement | achieved the video processing apparatus of FIG. 1 like software The flowchart for demonstrating operation | movement in the video processing apparatus of FIG.

  Hereinafter, preferred embodiments for carrying out the present disclosure will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram illustrating a schematic configuration of a video processing apparatus according to an embodiment of the present disclosure. In the figure, the video processing apparatus 1 according to the present embodiment detects a shaking amount of the imaging unit 2 attached to a moving body by a shaking amount detection unit 3 having a gyro sensor 31, and outputs the output S 1 to the low-pass filter 4. Pass through. The low-pass filter 4 is used to detect the front of a moving body whose direction changes with movement. The imaging unit 2 includes a wide-angle lens and an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and outputs an image captured by the imaging element. The comparison unit 5 takes a difference S1−S2 between the output S2 of the low-pass filter 4 and the output S1 of the fluctuation amount detection unit 3. The difference S1-S2 is a relative direction of the imaging unit 2 with respect to the front of the moving object, that is, a shaking component. The image cutout unit 6 cuts out, for example, a portion where a moving object is shown from the image (first frame image) picked up by the image pickup unit 2. The image cutout unit 6 performs shake correction by changing the cutout position of the image using the difference S1-S2. The encoding unit 7 encodes the image (second frame image) cut out by the image cutting unit 6.

  On the other hand, the absolute value determination unit 8 takes the absolute value of the difference S1-S2, compares the result with a predetermined threshold value, and if it is equal to or less than the threshold value, encodes the entire image (third frame image) at that time. The encoding unit 9 is instructed to do so. Here, the whole image may be the first frame image itself captured by the imaging unit 2, or an image that occupies a part of the first frame image and is larger than the second frame image. It may be. The encoding unit 9 encodes the image instructed from the absolute value determination unit 8. The image when the absolute value of the difference S1-S2 is equal to or less than the threshold is an image captured by the imaging unit 2 when the imaging unit 2 faces the front (the front of the moving object). As described above, the threshold value to be compared with the absolute value of the difference S1-S2 is determined to be a value that can extract an image when the imaging unit 2 faces the front.

  FIG. 2 is a waveform diagram showing an example of the output S1 of the fluctuation amount detection unit 3 and the output S2 of the low-pass filter 4. In the figure, the line that swings up and down is the output S1 of the swing amount detection unit 3, and the line that gently swings up and down is the output S2 of the low-pass filter 4. The point T at which the output S1 of the shake amount detection unit 3 and the output S2 of the low-pass filter 4 are the same is the timing for encoding the entire image, but in reality, the absolute value of the difference between the outputs S1 and S2 is within a predetermined range. The time when entering is the timing.

  FIG. 3 is a diagram illustrating an example of processing for designating an image to be encoded. In the outputs S1 and S2 shown in FIG. 2, the absolute value determination unit 8 takes the absolute value of the difference S1-S2 and compares the result with a predetermined threshold value. When the absolute value determination unit 8 determines that the absolute value of the difference S1-S2 is equal to or less than a predetermined threshold value, the absolute value determination unit 8 instructs the encoding unit 9 to encode the entire image at that time. Among images I1, I2, I3, I4,..., I8, I9 shown in FIG. It is an image instructed to be encoded at the following timing. These images I1, I4, and I8 form an image sequence with little shaking, and the difference between successive images is small, so that high coding efficiency is obtained.

  In FIGS. 1 to 3, as an embodiment of the present disclosure, an example in which the present disclosure is implemented with a hardware configuration is shown. The present disclosure can be configured not only by hardware but also by software using a general-purpose CPU. FIG. 4 and the following are used to illustrate another embodiment of the present disclosure.

  FIG. 4 is a block diagram showing an example of a configuration in which the video processing apparatus 1 according to the present embodiment is realized as software. As shown in the figure, elements other than the imaging unit 2, the shake amount detection unit 3, and the packet transmission unit 11 are configured by one processor 10. As described above, the imaging unit 2 includes a wide-angle lens and an imaging element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and outputs an image captured by the imaging element. As described above, the shake amount detection unit 3 detects the shake amount of the imaging unit 2 attached to the moving body.

  The processor 10 includes a CPU, a ROM, and a RAM, and includes a camera direction detection unit 101, a front direction detection unit 102, a shake correction amount calculation unit 103, an encoding unit 104, a packet assembly unit 105, an image buffer 106, and an image buffer 106. It has functions of a cutout unit 107, an encoding unit 108, a packet assembly unit 109, and a packet buffer 110.

  The camera orientation detection unit 101 has a function equivalent to the function of the low-pass filter 4 described above. The front direction detection unit 102 has functions equivalent to the functions of the comparison unit 5 and the absolute value determination unit 8 described above, and the output (corresponding to the output of the low pass filter 4 described above) S2 and the amount of shaking. The difference of the output S1 of the detection unit 3 is taken, the absolute value of the difference S1-S2 is compared with a predetermined threshold value, and when the absolute value of the difference S1-S2 is less than or equal to the threshold value, the image is sent to the encoding unit 104. Encoding is performed. Here, when the absolute value of the difference S1-S2 between the output S2 of the camera direction detection unit 101 and the output S1 of the shake detection unit 3 is equal to or smaller than a predetermined threshold, the image obtained from the imaging unit 2 is captured as described above. It is an image when the part 2 faces the front of a moving body. As described above, the threshold value to be compared with the difference S1-S2 is determined to be a value with which an image can be obtained when the imaging unit 2 faces the front.

  The encoding unit 104 has a function equivalent to the function of the encoding unit 9 described above, and encodes and outputs an image (third frame image) when the imaging unit 2 faces the front. The packet assembling unit 105 assembles the image packet output from the encoding unit 104. The shake correction amount calculation unit 103 has a function equivalent to that of the comparison unit 5 described above, and calculates and outputs a correction amount of the shake amount of the imaging unit 2 detected by the shake amount detection unit 3. This shake correction process is one of the first processes in the processor 10.

  The image buffer 106 accumulates images (first frame images) output from the imaging unit 2 for each frame in time series. The image cutout unit 107 has a function equivalent to the function of the image cutout unit 6 described above, and cuts out, for example, a portion where a moving object is shown from each image stored in the image buffer 106 in time series. Then, shake correction is performed on the clipped image (second frame image) using the difference S1-S2. The encoding unit 108 has a function equivalent to the function of the encoding unit 7 described above, and encodes the image that has been extracted and shake-corrected by the image extraction unit 107. This encoding process is one of the first processes in the processor 10. The packet assembling unit 109 assembles the image packet output from the encoding unit 108. The packet buffer 110 accumulates the packets assembled by the packet assembling units 105 and 109 and outputs them to the packet transmitting unit 11 in chronological order. The packet transmission unit 11 transmits a packet output from the packet buffer 110 to a network such as the Internet.

  Here, the amount of data of each of the image before cutting out the image, that is, the first frame image, the cut out image, that is, the second frame image, and the image when the imaging unit 2 faces the front, that is, the third frame image, is small. The relationship is as follows: first frame image ≧ third frame image> second frame image.

  Further, the processor 10 generates the second frame image in the second cycle with respect to the first frame image acquired by the imaging unit 2 in the first cycle. However, the first cycle and the second cycle are generated. May be equal to or different from each other.

  In addition, since the imaging unit 2 captures an image with a wide-angle lens, the first frame image is a wide-angle image.

FIG. 5 is a flowchart for explaining the operation in the video processing apparatus 1 according to the present embodiment. In the same figure, step S1-step S11 are repeatedly performed with a fixed period. This period is a normal frame rate (1/30 sec, 1/60 sec, etc.). First, the shaking amount detection unit 3 takes in the output (coordinate data (x, y, z)) of the gyro sensor 31 that it has in the current cycle (step S1). Next, the camera direction detection unit 101 performs a smoothing process on the coordinate data (x, y, z) captured by the shake amount detection unit 3 (step S2). That is, the front direction is calculated. For example, the coordinates (x ₀ , y ₀ , z ₀ ) ← ρ (x ₀ , y ₀ , z ₀ ) + (1−ρ) (x, y, z) after smoothing are calculated. Where ρ is a low-pass filter coefficient. Here, ρ satisfies the relationship (1>ρ> 0).

After the camera direction detection unit 101 performs the smoothing process, the front direction detection unit 102 calculates a difference amount D from the front (step S3). That is, the absolute value of the difference between the sensor raw information and the front direction is calculated. For example, D ← sqrt ((x−x ₀ ) ² + (y−y ₀ ) ² + (z−z ₀ ) ² ) is calculated (sqrt is a square root operation).

  After calculating the difference amount D from the front, the front direction detection unit 102 determines whether the calculated difference amount D is equal to or less than a predetermined threshold Th (step S4), and the difference amount D from the front has a predetermined threshold Th. If it is determined that it exceeds (when it is determined not facing the front), no processing is performed and the process proceeds to step S7. On the other hand, when it is determined that the difference amount D from the front is equal to or less than the predetermined threshold Th (when it is determined that the front is substantially facing the front), the encoding unit 104 encodes the current image (step) S5). That is, an image is encoded only when it is determined that the front side. After encoding the image determined to be in front, the packet assembling unit 105 packetizes the image encoded by the encoding unit 104 (step S6). Then, the packet transmission unit 11 transmits the image packetized by the packet assembly unit 105 (step S7).

On the other hand, after the smoothing process is performed in step S2, the shake correction amount calculation unit 103 calculates a shake correction amount (difference from the front) (step S8). That is, the shake correction amount (X, Y, Z) = ((x−x ₀ ), (y−y ₀ ), (z−z ₀ )) is calculated. After the shake correction amount calculation unit 103 calculates the shake correction amount, the image cutout unit 107 cuts out an image from the current image (step S9). That is, a rectangular area shifted from the standard position by the correction amount is cut out from the image. The image to be cut out is, for example, a portion where a moving object such as a suspicious person is shown. After the image is cut out, the encoding unit 108 encodes the cut out image (step S10). After encoding the clipped image, the packet assembling unit 109 packetizes the encoded image (step S11). Then, the packet transmission unit 11 transmits the packetized image (step S7).

  As described above, according to the video processing device 1 in the present embodiment, one is based on the difference S1-S2 between the output S1 of the shaking amount detection unit 3 having the gyro sensor 31 and the output S2 of the camera direction detection unit 101. An image is extracted from the entire image and encoded, and the other is extracted by extracting a part of the entire image and performing shake correction. Therefore, for an image with a large amount of information extracted from the entire image, an image captured within a predetermined range, that is, an image having a similar configuration is continuously encoded. Therefore, the efficiency of the motion compensation process etc. at the time of encoding increases. As described above, it is possible to efficiently encode a shaking image, and to suppress an increase in the load and power consumption of the processor 10.

  As mentioned above, although one embodiment was described referring to drawings, it cannot be overemphasized that this indication is not limited to this example. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present disclosure. Understood.

(Overview of one aspect of the present disclosure)
The video processing device according to the present disclosure is a video processing device mounted on a moving object, and the video processing device includes an imaging unit, a direction detection unit, and a processor, and the direction detection unit includes the moving object. The orientation of the imaging unit in the video processing device mounted on the processor is detected, and the processor is smaller than the first frame image with respect to the first frame image acquired by the imaging unit in a first period A second frame image having a data amount is generated in a second period, the orientation of the imaging unit is received from the orientation detection unit, and when the orientation of the imaging unit is within a predetermined range, the second frame A third frame image having a data amount larger than that of the image is extracted from the first frame image and generated.

  According to the above configuration, the first frame image is acquired in the first cycle, the second frame image having a data amount smaller than the first frame image is generated in the second cycle, and the orientation of the imaging unit is predetermined. Since the third frame image having a data amount larger than that of the second frame image is extracted from the first frame image and generated, it is possible to efficiently encode the shaking image. Therefore, even if shake correction is performed on the entire image, increases in processor load and power consumption can be minimized.

  In the configuration described above, the processor performs at least a first process including an encoding process on the second frame image when generating the second frame image.

  According to the above configuration, it is possible to always monitor and record the image of the imaging unit attached to the moving body.

  In the above configuration, the first process includes a shake correction process in addition to the encoding process for the second frame image.

  According to the above configuration, the output of the direction detection unit (camera direction detection unit (low-pass filter)) used in the shake correction process can be used for other processes.

  In the above configuration, when the orientation of the imaging unit is within a predetermined range, the case where the orientation of the imaging unit substantially coincides with the front of the moving body is included.

  According to the above configuration, the efficiency of motion compensation processing can be improved.

  In the above configuration, the orientation detection unit detects the orientation of the imaging unit based on the output of a gyro sensor that detects the amount of shaking of the imaging unit attached to the moving body.

  According to the above configuration, the amount of shaking can be accurately detected.

  In the above configuration, the orientation detection unit includes a low-pass filter, and detects the orientation of the imaging unit based on an output of the gyro sensor and an output obtained from the output of the gyro sensor via the low-pass filter. To do.

  According to the above configuration, the front of the moving body can be detected even when the direction of the moving body changes.

  In the above configuration, the imaging unit captures an image by a wide-angle lens, the first frame image is a wide-angle image, and the second frame image is an image of a region including a center point in the wide-angle image.

  According to the above configuration, even when a wide-angle lens having a large coding load on the entire image is used, it is possible to perform processing efficiently.

  The video processing method of the present disclosure is a video processing method for processing an image captured by an imaging device attached to a moving object, and detecting an orientation of an imaging unit in the imaging device attached to the moving object; Generating a second frame image having a data amount smaller than that of the first frame image in a second cycle with respect to the first frame image acquired by the imaging unit in a first cycle; And generating a third frame image having a data amount larger than that of the second frame image by extracting from the first frame image.

  According to the above method, the first frame image is acquired in the first cycle, the second frame image having a data amount smaller than the first frame image is generated in the second cycle, and the orientation of the imaging unit is predetermined. Since the third frame image having a data amount larger than that of the second frame image is extracted from the first frame image and generated, it is possible to efficiently encode the shaking image. Therefore, even if shake correction is performed on the entire image, increases in processor load and power consumption can be minimized.

  In the method, the step of generating the second frame image performs a first process including an encoding process for at least the second frame image when generating the second frame image.

  According to the above method, it is possible to always monitor and record the image of the imaging unit attached to the moving body.

  In the above method, the first process includes a shake correction process in addition to the encoding process for the second frame image.

  According to the above method, the output of the direction detection unit (camera direction detection unit (low-pass filter)) used in the shake correction process can be used for other processes.

  In the above method, when the orientation of the imaging unit is within a predetermined range, the orientation of the imaging unit substantially coincides with the front of the moving body.

  According to the above method, the efficiency of the motion compensation process can be improved.

  In the above method, the step of detecting the orientation of the imaging unit detects the orientation of the imaging unit based on the output of a gyro sensor that detects the amount of shaking of the imaging unit attached to the moving body.

  According to the above method, the amount of shaking can be accurately detected.

  In the above method, the step of detecting the orientation of the imaging unit detects the orientation of the imaging unit based on an output of the gyro sensor and an output obtained from the output of the gyro sensor via a low-pass filter.

  According to the above method, the front of the moving object can be detected even when the direction of the moving object changes.

  In the above method, the imaging unit captures an image by a wide-angle lens, the first frame image is a wide-angle image, and the second frame image is an image of a region including a center point in the wide-angle image.

  According to the above method, even when a wide-angle lens having a large encoding load on the entire image is used, it is possible to perform processing efficiently.

  The present disclosure has an effect that encoding of a swaying image can be efficiently performed and an increase in processor load and power consumption can be suppressed. For example, a wearable camera that is worn on a human body is used. Application to a surveillance camera system is possible.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Image pick-up part 3 Shaking amount detection part 4 Low pass filter 5 Comparison part 6 Image extraction part 7, 9 Coding part 8 Absolute value determination part 10 Processor 11 Packet transmission part 31 Gyro sensor 101 Camera direction detection part 102 Front Direction detection unit 103 Shake correction amount calculation unit 104, 108 Coding unit 105, 109 Packet assembly unit 106 Image buffer 107 Image extraction unit 110 Packet buffer

Claims (14)

A video processing device mounted on a moving object,
The video processing device includes:
An imaging unit;
An orientation detector;
A processor;
Have
The direction detector
Detecting the orientation of the imaging unit in the video processing device mounted on the moving body;
The processor is
For the first frame image acquired by the imaging unit in the first cycle, a second frame image having a data amount smaller than the first frame image is generated in the second cycle,
When the orientation detection unit receives the orientation of the imaging unit and the orientation of the imaging unit is within a predetermined range, a third frame image having a data amount larger than that of the second frame image is transmitted to the first frame. Extract from image and generate,
Video processing device. The processor performs a first process including an encoding process on at least the second frame image when generating the second frame image.
The video processing apparatus according to claim 1. The first process includes a shake correction process in addition to the encoding process for the second frame image.
The video processing apparatus according to claim 2. When the orientation of the imaging unit is within a predetermined range, including the case where the orientation of the imaging unit substantially coincides with the front of the moving object,
The video processing apparatus according to claim 1. The orientation detection unit detects the orientation of the imaging unit based on an output of a gyro sensor that detects a shake amount of the imaging unit attached to the moving body.
The video processing apparatus according to claim 1. The orientation detection unit includes a low-pass filter, and detects the orientation of the imaging unit based on an output of the gyro sensor and an output obtained by outputting the output of the gyro sensor via the low-pass filter.
The video processing apparatus according to claim 5. The imaging unit captures an image with a wide-angle lens,
The first frame image is a wide-angle image, and the second frame image is an image of a region including a center point in the wide-angle image.
The video processing apparatus according to claim 1. An image processing method for processing an image captured by an imaging device attached to a moving object,
Detecting an orientation of an imaging unit in the imaging device attached to the moving body;
Generating a second frame image having a data amount smaller than that of the first frame image in a second cycle with respect to the first frame image acquired by the imaging unit in a first cycle;
When the orientation of the imaging unit is within a predetermined range, extracting and generating a third frame image having a data amount larger than the second frame image from the first frame image;
Video processing method. The step of generating the second frame image performs a first process including an encoding process for at least the second frame image when generating the second frame image.
The video processing method according to claim 8. The first process includes a shake correction process in addition to the encoding process for the second frame image.
The video processing method according to claim 9. When the orientation of the imaging unit is within a predetermined range, including the case where the orientation of the imaging unit substantially coincides with the front of the moving object,
The video processing method according to claim 8. The step of detecting the orientation of the imaging unit detects the orientation of the imaging unit based on the output of a gyro sensor that detects the amount of shaking of the imaging unit attached to the moving body.
The video processing method according to claim 8. The step of detecting the orientation of the imaging unit detects the orientation of the imaging unit based on the output of the gyro sensor and the output obtained by passing the output of the gyro sensor through a low-pass filter.
The video processing method according to claim 12. The imaging unit captures an image with a wide-angle lens,
The first frame image is a wide-angle image, and the second frame image is an image of a region including a center point in the wide-angle image.
The video processing method according to claim 8.

Images