JP2005286596A - Image processing apparatus and image data transfer method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus capable of reducing load imposed on processing of an image processing unit side, and further reducing an apparatus configuration in size and cost. <P>SOLUTION: The apparatus is provided with an electronic camera 101 for acquiring image data obtained by picking up an image; a region extractor 113 for detecting the feature of the image data acquired by the electronic camera 101 for a video image processing apparatus provided with a video processor 3 for creating display data capable of processing the acquired image data, and displaying an image in a predetermined format; a label number encoder 115 for specifying label information indicating the detected feature; a UV signal label number multiplexing section 109 for transferring the label number specified by the encoder 115 to the section 3 together with the image data; and a UV signal port 119b. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus and an image data transfer method.

  In image processing where it is desired to process a captured image in real time, such as a security system, it is currently possible to detect the characteristics of a specific object (subject) that is determined in advance from the captured image. It has been broken. The features to be detected include, for example, a person, a person's face, a vehicle, and the like. The detection is performed by measuring a feature amount corresponding to the state of the subject, and the feature amount refers to, for example, the color or luminance level of the subject. The measurement is performed by an analysis process for analyzing the shape of the subject, measurement by the reflectance of the luminance level, or the like.

  Further, there is a conventional technique for detecting a moving subject by separating the moving subject from a stationary background image (for example, Patent Document 1). Specifically, the individual frame images constituting the moving image are arranged in time series, the difference in the position of the subject is detected between the previous frame image and the subsequent frame image, and the generated difference is output. Such a method of detecting a moving subject is relatively simple, but is effective for detecting a specific subject from a moving image taken by a fixed camera such as a surveillance camera.

  FIG. 15 is a diagram illustrating a general image processing apparatus that executes a process of detecting a subject from an image. The illustrated configuration includes a video signal source 1501 that is a signal source of an image of a digital camera or the like (here, a video is considered because it is a moving image), and a video processing unit 1502. The video signal source 1501 generates and outputs image data by capturing an image. The image data is YUV or R, G, B data (YUV in the figure). In the example shown in FIG. 15, the U signal and the V signal are multiplexed by the UV multiplexing unit 1503 inside the video signal source 1501 and transferred to the video processing unit 1502 together with the Y signal.

The video processing unit 1502 includes a UV separation unit 1505 that separates the multiplexed U signal and V signal from the transferred image data, and interpolates the separated U signal and V signal (thinned by multiplexing). Interpolation unit 1506 for interpolating pixels, a video memory 1507 for storing image data generated by the interpolated U and V data and the input Y data, and image data (display) stored in the video memory 1507 An area detecting unit 1508 for detecting an area having a feature from (data) is provided.
JP 2003-58892 A

  However, the video processing unit 1502 is mainly composed of a CPU and a video memory for reasons such as increasing the degree of freedom of image processing or changing relatively frequently, and the above processing is executed by software. The process of extracting image features from display data using software generally has a heavy load on the CPU. In addition, the CPU of the video processing unit 1502 is required to perform processing such as image interpolation, compression, and display.

Further, since the transfer from the image signal source 1501 to the video processing unit 1502 is performed in real time at a speed of about 30 fps (frame / sec), the video processing unit 1502 processes the image data transferred and stored in real time. Therefore, higher processing capacity is required. This makes the entire image processing apparatus large and expensive.
The present invention has been made in view of the above points, and provides an image processing apparatus and an image data transfer method that can reduce the load on processing on the image processing unit side, further reduce the size and cost of the apparatus configuration. The purpose is to provide.

  In order to solve the above problems, an image processing apparatus according to the present invention processes image data acquired by an image data acquisition unit that acquires image data obtained by capturing an image, and processes the image data acquired by the image data acquisition unit. An image processing apparatus comprising image data processing means for creating display data capable of displaying an image in a format, wherein the feature detection means detects the characteristics of the image data acquired by the image data acquisition means, and the feature detection Label information specifying means for specifying label information indicating the characteristics detected by the means, and image data transfer means for transferring the label information specified by the label information specifying means together with the image data to the image data processing means. It is characterized by.

  According to such an invention, the feature of the acquired image data can be detected, and the feature can be transferred to the image data processing means together with the image data as label information. For this reason, the process for detecting the feature of the image data in the image data processing means becomes unnecessary, and the load on the image data processing means can be reduced. Further, it becomes possible to adopt a configuration such as a CPU having a lower processing capacity for the image data processing means, thereby further reducing the size and cost of the image data processing means, and further reducing the size and cost of the entire image processing apparatus. Can be

In the image processing apparatus of the present invention, the image data transfer means multiplexes the label information into a UV signal of a video signal composed of a luminance signal Y and a color difference signal UV.
In the image processing apparatus of the present invention, when the image data transfer means multiplexes the label information on the UV signal, the UV signal is multiplexed on either the odd line or the even line of the video signal, and the UV The label information is output at the output timing of the other line on which the signal is multiplexed.

  According to such an invention, the label information can be transferred without greatly changing the configuration of the image data processing means in the conventional video system such as a surveillance camera. Further, since the label information and the image data can be transferred in synchronization, the load on the data synchronization processing on the image processing apparatus side can be suppressed. In addition, the data transfer system can be effectively used to increase the data transfer efficiency.

Moreover, the image processing apparatus of the present invention is characterized in that the feature detection means detects at least one feature of the color, shape, and reflectance of the imaging target indicated by the image data.
According to such an invention, at least one of a specific color, shape, and reflectance can be detected from the image data as a feature. For this reason, the presence of a specific object can be automatically detected from the image data by setting the color, shape, and reflectance according to the object to be specified in advance.

Moreover, the image processing apparatus of the present invention is characterized in that the feature detection means detects the movement of the imaging object indicated by the image data as a feature.
According to such an invention, since the movement of the imaging target indicated by the image data can be detected as a feature, it is possible to automatically detect that there is something moving within the imaging target range.

The image processing apparatus of the present invention is characterized in that the image data transfer means transfers only the label information relating to the representative value among the features detected by the feature detection means together with the image data.
According to such an invention, the amount of label information transferred is reduced. Therefore, other label information can be transferred at the transfer timing, and more detailed information can be added to the image data.

The image data transfer method of the present invention includes a feature detection step for detecting a feature of image data, a label information specification step for specifying label information indicating the feature detected in the feature detection step, and the label information And an image data transfer step of transferring to display data storage means capable of displaying an image together with the image data.
According to such an invention, the feature of the acquired image data can be detected, and the feature can be transferred to the image data processing means together with the image data as label information. For this reason, the process for detecting the feature of the image data in the image data processing means becomes unnecessary, and the load on the image data processing means can be reduced. Further, it becomes possible to adopt a configuration such as a CPU having a lower processing capacity for the image data processing means, thereby further reducing the size and cost of the image data processing means, and further reducing the size and cost of the entire image processing apparatus. Can be

  Embodiments of an image processing apparatus according to the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram for explaining the configuration of an image processing apparatus according to an embodiment of the present invention. The illustrated configuration includes an electronic camera 101, a label information multiplexing unit 2, and a video processing unit 3. The electronic camera 101 is an image data acquisition unit that acquires image data obtained by capturing an image. The video processing unit 3 is an image data processing unit that processes image data acquired by the electronic camera 101 and generates display data that can display an image. In the present embodiment, since the electronic camera 101 is configured to capture a moving image, a signal indicating image data is particularly referred to as a video signal.

  The label information multiplexing unit 2 includes a labeling unit 111. The labeling unit 111 includes a region extracting unit 113 that is a feature detecting unit that detects features of image data, and a label number encoder 115 that is a label information specifying unit that specifies label information indicating the features detected by the region extracting unit 113. I have. The label information multiplexing unit 2 includes a UV signal / label number multiplexing unit 109 and a UV signal port 119b that transfer the label information specified by the label number encoder 115 to the video processing unit 3 together with the video signal. Since the label information of this embodiment is represented by a number, in this embodiment, the label information is hereinafter referred to as a label number.

  The region extraction unit 113 detects at least one of the color, shape, and reflectance of the imaging target indicated by the video signal as a feature. In the present embodiment, such a feature is hereinafter referred to as a feature amount. The label number encoder 115 is a configuration for determining a feature value to be assigned as a label number from the extracted feature values. Specific configurations of the area extraction unit 113 and the label number encoder 115 will be described later.

  By detecting the color, shape, and reflectance of the imaging object, the image processing apparatus according to the present embodiment can determine whether the imaging object is a human, a human face, a car, or the like. . For example, it is possible to detect an area of a color that is considered to be a skin color in the image and to determine that a skin portion such as a human face is shown in this area. Further, by detecting a region of an image that matches a preset shape or size condition, it can be determined that the object to be photographed is an automobile or the like.

  When the image processing apparatus of the present embodiment is applied to a security system, for example, when a video signal with labeling information indicating that there is a flesh-colored image is transferred to the video processing unit 3, the video processing unit 3 It is also possible to notify a notification device (not shown) that “a person is in the shooting range of the electronic camera 101”. This notification can be used as a trigger for switching the security mode such as starting more detailed monitoring in the security system, for example.

  Further, the labeling unit 111 according to the present embodiment can detect the movement of the imaging target indicated by the input video signal as a feature amount. By detecting the movement of the imaging object as a feature amount, the image processing apparatus according to the present embodiment can determine that there is a possibility that someone has entered the imaging range. In such a determination, for example, when a video signal with labeling information indicating that there is a moving subject to be moved beyond a predetermined area is transferred to the video processing unit 3, the video processing unit 3 notifies the notification (not shown). It is also possible to notify the apparatus that “the moving object is in the shooting range of the electronic camera 101”.

  In the image processing apparatus of the present embodiment, the electronic camera 101 generates a video signal as a Y signal (Y-signal), a U signal (U-signal), and a V signal (V-signal), and the label information multiplexing unit 2 To enter. The Y, U, and V signals represent a signal related to an image with luminance information (Y) and two color information (U and V). The ITU-R BT. 601 is standardized as a YUV system.

  The label information multiplexing unit 2 converts each Y, U, and V signal input from the electronic camera 101 into a YUV411 format signal, and the Y signal and the UV signal (a signal obtained by multiplexing the U and V signals). YUV conversion unit 105 that outputs the Y signal, the Y signal converted by the YUV conversion unit 105, the delay amount adjustment unit 107 that synchronizes the UV signal and a label number described later, and the delay amount adjustment unit 105 that adjusts the timing and outputs A UV signal / label number multiplexing unit 109 that multiplexes the label number output from the labeling unit 111 to the UV signal.

  Further, a synchronization signal is input from the electronic camera 101 to the label information multiplexing unit 2. The label information multiplexing unit 2 includes a pixel counter control unit 117, and the pixel counter control unit 117 receives a control signal from the region extraction unit 113 and the label number encoder 115 of the labeling unit 111 according to the input synchronization signal. The region extraction unit 113 and the label number encoder 115 operate according to the control signal, so that the output label number can have a correspondence relationship with the coordinates in the image (the control signal gives it).

  Also, the pixel counter control unit 117 inputs a multiplexing control signal based on the synchronization signal (giving a correspondence relationship with the coordinates of the image) to the UV signal / label signal multiplexing unit 109. The UV signal / label signal multiplexing unit 109 multiplexes the label number with the UV signal based on the multiplexing control signal, and outputs the multiplexed signal from the UV signal port 119b to the video processing unit 3 as a UV label number multiplexed signal. The Y signal is output from the Y signal port 119a to the video processing unit 3. At this time, since the synchronization signal input from the electronic camera 101 is output from the pixel counter control unit 117 to the video processing unit 3 from the synchronization signal port 119c, the video processing unit 3 inputs the Y signal and the UV label number multiplexed signal. The timing shift can be adjusted by the synchronization signal.

The configuration described above operates as follows.
The electronic camera 101 transfers a video signal (YUV signal) obtained by imaging to the label information multiplexing unit 2. The transferred video signal is input to the labeling unit 111 together with the YUV 411 conversion unit 105.
The YUV411 conversion unit 105 converts the transferred YUV signal into the YUV411 format. The YUV411 format is one of the YUV methods described above, and expresses color information with a data amount of 12 bits per pixel. At this time, a method is adopted in which 4 samples of luminance information (Y) are selected from 4 pixels of 2 × 2, one sample each of the difference between luminance and red component (U), and the difference between luminance and blue component (V). Say.

  FIG. 2 is a diagram for explaining the transfer of the Y signal of the video signal converted in the YUV411 format. FIG. 3 is a diagram for explaining an example of transfer of the U signal of the video signal converted in the YUV411 format. As shown in FIG. 2, the Y signal is output from the YUV 411 conversion unit 105 in order from the pixel Y1 at the left end of the first line in the drawing. Then, after the pixel Y640 at the right end in the drawing of the line is output, the pixel Y1 at the left end of the second line is output. As described above, all Y signal pixels are sequentially output from the YUV 411 conversion unit 105 to the delay amount adjustment unit 107 in order from the pixels input first.

  Further, as shown in FIG. 3, the YUV411 conversion unit 105 obtains one representative value (for example, U1) from four pixels (for example, u11, u12, u21, and u22) adjacent to each other in the video signal, as shown in FIG. calculate. Then, only the obtained representative value is output to the delay amount adjustment unit 107. Similarly, only representative values calculated from four pixels are output for the V signal. Therefore, the U and V signals output from the YUV411 conversion unit 105 to the delay amount adjustment unit 107 are ¼ of the total data amount of the Y signal.

The YUV411 format described above can reduce the amount of data to be transferred for each color compared to the RGB method for transferring all pixels, and even when the amount of data is reduced by using the low spatial resolution of the human eye, the image quality can be reduced. It can be said that deterioration of the resin can be suppressed.
FIG. 4 is a diagram for explaining the transmission format of U signal representative values U1, U2, U3... And V signal representative values V1, V2, V3. The representative values U1, U2, U3... And the representative values V1, V2, V3... Are alternately transmitted (that is, multiplexed) on one line as shown. Such multiplexing is realized by the YUV 411 conversion unit 105 alternately switching and outputting the representative value of the U signal and the representative value of the V signal.

  As described above, the data amount of the U and V signals is ¼ that before the conversion to the YUV411 format. As a result, even after multiplexing the U and V signals, the representative value data is only present on the odd lines. Exists and there is no data to be transmitted on even lines. In the general YUV411 format, dummy data is transmitted at the transmission timing of even lines. As the dummy data, for example, odd-numbered line data that has already been transferred is used. In this embodiment, as will be described later, it is assumed that the label number is transmitted at the dummy data transfer timing and the label number is multiplexed with the UV signal.

  The Y signal and the UV signal converted into the YUV411 format are sent to the delay amount adjustment unit 107, respectively. The delay amount adjustment unit 107 is configured to synchronize the input Y signal and UV signal with the label number output from the labeling unit 111. Specifically, the output timing of the YUV signal is delayed in accordance with the delay of the label number generated in the line memory or delay element of the labeling unit 111. The delay amount adjustment unit 107 sends the Y signal to the Y signal port 109a and the UV signal to the UV signal / label number multiplexing unit 109 at the delayed timing.

  On the other hand, the labeling unit 111 detects a feature amount of a predetermined feature from the YUV signal input from the electronic camera 101 in the region extraction unit 113. The labeling unit 111 detects feature amounts for all pixels and outputs the result to the UV signal / label number multiplexing unit 109. The UV signal label number multiplexing unit 109 according to the first embodiment sets one feature value as a representative value for every four adjacent pixels (decimation processing), and outputs only the representative value (described later).

  FIG. 5 is a diagram for explaining a more detailed configuration of the region extraction unit 113 in the labeling unit 111. The region extraction unit 113 includes a plurality (n in this embodiment) of region detectors 501a, 501b,. The area detectors 501a to 501n have specific area detectors 502a to 502n and noise removal filters 503a to 503n, respectively. The YUV video signal is input to each of the area detectors 501a to 501n. The specific area detectors 502a to 502n of each area detector detect the feature amount of the YUV signal bit by bit.

  FIG. 6 is a diagram illustrating a configuration of specific area detection units 502a to 502n that detect the color of a video signal as a feature amount. The specific area detecting unit 502a shown in the figure is composed of three sets of comparators 601a, 601b, 601c and a plurality of AND circuits 602a to 602e. The Y signal indicating the luminance is compared with two threshold values (YHigh, YLow) whose values are preset by the two comparators 601a. The AND circuit 602a outputs a signal “1” only when the luminance indicated by the Y signal is lower than YHigh and higher than YLow, that is, within a predetermined range.

  The U signal indicating the color is compared with two threshold values (UHigh and ULow) whose values are set in advance by the two comparators 601b. The AND circuit 602b outputs a signal “1” only when the value of the U signal is between UHigh and ULow. Further, the V signal is compared with two threshold values (VHigh, VLow) by two comparators 601c. The AND circuit 602c outputs a signal “1” only when the value of the V signal is between VHigh and VLow.

  The signals output from the AND circuits 602b and 602c are input to the AND circuit 602d. The AND circuit 602d outputs “1” only when the Y signal is in a predetermined value range and the V signal is in a predetermined value range. Further, the signal output from the AND circuit 602d is input to the AND circuit 602e together with the signal output from the AND circuit 602a. The AND circuit 602e outputs a signal of “1” only when both of the input signals are “1”. Output.

  That is, the specific area detection unit 502a outputs the signal “1” only when all the values of the YUV signal are within a predetermined threshold range, and outputs the signal “0” otherwise. Therefore, by setting the threshold value of each value of YUV in accordance with the range of values when the YUV signal represents the color of human skin, the video signal can be converted into a human face or limb using the specific area detection unit 502a. Etc. can be detected. It is also possible to detect a vehicle or the like by setting a threshold value for each value of YUV in accordance with a color range of the vehicle or the like for which a YUV signal is predicted in advance. The signal output from the specific area detection unit 502a is hereinafter referred to as an area detection flag. In addition, a pixel having a detection area flag “1” is referred to as a valid pixel, and a pixel having “0” is referred to as an invalid pixel.

  FIG. 7 is a diagram for illustrating the configuration of the noise removal filter 503a. The noise removal filter 503a includes three binary pixel basic filters 701a, 701b, and 701c, receives an area detection flag, and interpolates missing effective pixels or removes dust. In the present embodiment, three binary pixel basic filters are combined. However, the performance of the noise removal filter improves as the number of binary pixel basic filters increases.

  FIGS. 8A, 8 </ b> B, and 8 </ b> C are diagrams for explaining a 3 × 3 binary pixel basic filter. FIG. 8A shows a configuration of a binary pixel basic filter (for example, 701a), and FIGS. 8B and 8C are diagrams for explaining processing performed by the binary pixel basic filter. The area detection flag of “1” or “0” is sequentially stored in the two 1-bit line memories 801a and 801b and the delay element 802 for delay processing. The binary pixel basic filter 701a performs, for example, 3 × 3 filter processing shown in (b) using this delay data. The characteristics of the filter are determined by the configuration of the binary image filter logic 803. That is, the output is uniquely determined by the logical expression.

  In the illustrated filter, the binary image filter logic 803 processes the center pixel D22 as the target pixel. The pixel interpolation process will be described with reference to FIG. In the binary image filter logic 803, for example, pixels (D11, D12, D13, D21, D31) shown in black in FIG. 6C are effective pixels, and pixels (D23, D32, D33) shown in white are shown. If it is an invalid pixel, the target pixel D22 is determined as a valid pixel. According to such an example, a smooth line can be drawn except for the jaggy of the diagonal line of the image. The interpolated validity detection flag is output bit by bit to the next stage filter.

The binary pixel basic filter can remove both effective pixels and dust by changing the configuration and processing conditions of the binary image filter logic. Further, by combining a plurality of binary image basic filters having different characteristics, this embodiment can constitute a noise removal filter capable of interpolating pixels of a video signal and removing dust.
The region detectors 501a to 501n of the present embodiment described above detect different feature amounts that are set in advance. Then, it is determined whether each pixel constituting the video signal is an effective pixel or an invalid pixel having a characteristic amount, and effective pixel omission and dust removal processing is performed and output to the label number encoder 115. Accordingly, n area detection flags corresponding to the area detectors 501a to 501n are input to the label number encoder 115.

The label number encoder 115 converts the input region detection flag into a label number and outputs it to the UV signal / label number multiplexing unit 109.
For example, when there are 16 area detectors, conversion of the area detection flag into the label number can be realized by using 16 signals output from the area detector as 4-bit label number signals. In the following description of the present embodiment, the label number is assumed to be 8-bit information (256 types of areas can be identified).

  Further, in the present embodiment described above, the feature amount is detected by the luminance or color difference of the YUV signal. However, the feature amount of the present embodiment is not limited to the luminance and color of the YUV signal, and may be the shape and movement of the subject indicated by the video signal. The shape of the subject can be detected as a feature amount by, for example, pattern matching between a preset shape and an image. In addition, a configuration for detecting the movement of the subject as a feature amount is possible by detecting a change with time of the pixels constituting the video signal. Since such a detection method is well known, further detailed explanation is omitted.

  The image processing apparatus according to the present embodiment represents the label number with 8 bits as described above. When the feature amount is detected based on the color and luminance of the pixel and the feature amount based on the movement of the subject is detected from the video signal, for example, 4 bits out of 8 bits are labeled numbers indicating the feature amounts. Can also be assigned. According to such a configuration, not only information that there is a human image or a vehicle image in the video signal, but also information that a human has entered or that the vehicle has passed through the shooting range can be obtained. Can do.

  In the present embodiment described above, the feature amount is immediately detected from each pixel of the image data input by the electronic camera 101. When image data obtained by imaging is converted into display data that can display an image, generally, the number of pixels is increased for high resolution processing, or gradation processing is performed to match the characteristics of the display device. It is known. For this reason, in the present embodiment, the number of pixels to be processed becomes smaller than that in which the feature amount is detected after the image data is converted into the display data, and the processing load can be reduced. In addition, since the output from the image sensor can be directly processed and the output exhibits linearity, it is possible to extract a feature amount with higher accuracy.

  Next, the label information multiplexing unit 2 multiplexes the input UV signal and the label number in the UV signal / label number multiplexing unit 109. As described above, the YUV 411 format transfers the UV signal together with dummy data that does not need to be transferred, and the UV signal / label number multiplexing unit 109 sends the label number to the video processing device 3 at the timing of dummy data transfer. By transferring, the label number is multiplexed with the image data in place of the dummy data and transferred.

  That is, in the multiplexing in the UV signal / label number multiplexing unit 109, as shown in FIG. 9A, the UV signal / label number multiplexing unit 109 has a multiplexer 901, and the multiplexer 901 outputs the UV signal and the label number. Realized by switching and outputting. 10 shows multiplexed UV signals and label numbers. Label numbers L1, L2,... L320 are output at the dummy data output timing (even lines) shown in FIG. Multiplexed into signal.

According to the present embodiment in which the label number is transferred instead of the dummy data, the label information can be transferred without largely changing the configuration of the image data processing means in the video system such as the conventional surveillance camera.
Further, since the label information and the image data can be transferred in synchronization, the load on the data synchronization processing on the image processing apparatus side can be suppressed. In addition, the data transfer system can be effectively used to increase the data transfer efficiency.

  In this embodiment, as shown in FIG. 10A, the label number output from the UV signal / label number multiplexing unit 109 is only for the representative value determined for every four adjacent pixels. Therefore, there is a vacancy in the transfer timing of the data (referred to as reserve data) indicated as R1, R2,. At the transfer timing of the reserve data, dummy data may be transferred, or other feature quantities such as the movement of the photographing object may be further multiplexed on the UV signal.

  FIG. 9B is a diagram for explaining the configuration of the multiplexer when the reserve data is multiplexed with the label number into the UV signal. When the reserve data is multiplexed together with the label number, the label number L is sent to the multiplexer 902 together with the reserve data R, and the output timings of both are switched for each pixel and multiplexed. Further, the multiplexed label number L and reserve data R are sent to the multiplexer 903 together with the UV signal, and the multiplexer 903 switches the label number L, the reserve data R and the output timing of the UV signal for each line and outputs them. In this case, the multiplexed signal shown in FIG. 10B is realized. In this embodiment, the label number is transferred at the timing of transferring the redundant portion of the transfer data such as dummy data in the YUV411 format, so that the bandwidth (data transfer capability) of the data transfer system is effectively utilized to the maximum extent. A new function called label number transfer can be added while maintaining the conventional physical configuration, transfer rate, and other standards.

  The signal generated as described above is transferred from the UV signal port 119b to the video processing device 3 as a label number multiplexed signal. The Y signal is transferred from the Y signal port 119a to the video processing unit 3. The label number multiplexed signal and the Y signal are transferred at a frame rate of 30 fps in this embodiment. The transfer is performed based on the synchronization signal (synchronization signal IN) input to the label information multiplexing unit 2 by the electronic camera 101 and the synchronization signal (synchronization signal OUT) generated by delaying the synchronization signal. The synchronization signal is a vertical synchronization signal indicating the start of a frame of video data to be transferred, a horizontal synchronization signal indicating the start of a line, three signals of pixel synchronization signals, or a composite signal of these three signals.

  FIG. 11 is a diagram for explaining the configuration of the video processing device 3. The video processing device 3 includes a UV signal separation / synchronization unit 1101 and a UV signal storage unit 1102. The UV signal separation / synchronization unit 1101 includes a delay amount adjustment unit 1103, a UV signal / label number separation unit 1104, and a pixel counter control unit 1105. The UV signal storage unit 1102 includes a UV signal separation unit 1106, a label number interpolation unit 1107, and a VRAM 1108.

  Of the signals output from the label information multiplexing unit 2, the Y signal is input to the delay amount adjusting unit 1103. The UV signal / label number multiplexed signal is input to the UV signal / label number separation unit 1104, and the synchronization signal OUT is input to the pixel counter control unit 1105. FIG. 12 is a diagram for explaining the configuration of the UV signal / label number separation unit 1104. The UV signal / label number separation unit 1104 includes a demultiplexer 1201. The demultiplexer 1201 separates the input UV signal / label number multiplexed signal into a UV signal and a label number by separately outputting based on a control signal that is switched for each line number.

A control signal that is switched for each line number is output by the pixel counter control unit 1105 based on the synchronization signal OUT. The delay amount adjustment unit 1103 is configured to delay the Y signal transferred together with the UV signal by the processing time in the UV signal / label number separation unit 1104 and synchronize the output timing of the Y signal with the UV signal.
The Y signal, UV signal, label number, and synchronization signal are all input to the UV signal storage unit 1102. The UV signal storage unit 1102 includes a UV signal separation unit 1106 and a label number interpolation unit 1107. The Y signal is input to the VRAM 1108 as it is and stored. Further, the UV signal is input to the UV signal separation unit 1106, separated, interpolated, and stored in the VRAM 1108. The label number is input to the label number interpolation unit 1107, interpolated, and stored in the VRAM 1108. The synchronization signal is input to the VRAM 1108, and the VRAM 1108 operates based on the synchronization signal.

  FIG. 13 is a diagram for explaining the configuration of the UV signal separation unit 1106. The UV signal separation unit 1106 includes a line memory 1301 that accumulates an input UV signal for one line, a multiplexer 1302, a delay element 1303, and a multiplexer 1304. The UV signal is input to the multiplexer 1302 via the line memory 1301 or directly. When an odd-numbered line of the UV signal is transferred, the UV signal separation unit 1106 outputs this signal and stores it in the line memory 1301. On the other hand, when the even-numbered line is transferred, the odd-numbered signal stored in the memory line 1301 is output. As a result, the multiplexer 1302 always outputs an odd-numbered line UV signal.

  In addition, when an odd-numbered (pixel unit) pixel is input to the UV signal separation unit 1106, this pixel is output to the VRAM 1108 and delayed by the delay element 1303. When an even-numbered pixel (pixel unit) is input, the pixel delayed by the delay element 1303 is output to the VRAM 1108. As a result, the U signal is separated from the V signal and stored in the VRAM 1108 as shown in FIG.

  In FIG. 14, the pixels that have been once sent to the line memory 1301 and the delay element 1301 are delayed, and the sent pixels are shown in gray as interpolation pixels. Although not shown, the V signal is also separated from the U signal and stored in the VRAM 1108 by the operation described above. The area of the VRAM 1108 in which the U signal is stored is called a U plane, and the area of the VRAM 1108 in which the V signal is stored is called a V plane.

  Furthermore, in this embodiment, the label number separated from the UV signal by the UV signal / label signal separation unit 1104 is also input to the UV signal storage unit 1102. Then, the label number interpolation unit 1107 performs interpolation. In this interpolation, the label number interpolation unit 1107 has a configuration substantially similar to the configuration shown in FIG. 13 and outputs, for example, the transferred data as it is only when the signal of the even-numbered line is transferred. Accumulate in line memory. When the odd-numbered line is transferred, the signal stored in the line memory is output, and if the output is switched for each pixel in one line, the label numbers L1, L2,... Shown in FIG. The feature amounts transferred at the timings R1, R2,... Can be transferred to the VRAM 1108 and stored separately.

  After the processing described above, the video processing unit 3 processes the YUV signal stored in the VRAM 1108 as necessary to create display data that can display an image in a predetermined format. The process of converting a YUV signal into data that can display an image refers to a process of converting a YUV signal into an RGB signal, for example. In addition, the video processing unit 3 performs a process of specifying a shooting target (human or car) indicated by the video signal from the label number as needed, and notifying the specified shooting target to the outside.

According to this embodiment described above, it is possible to attach a label to a video signal obtained by a video signal source such as an electronic camera and transfer it to a video processing unit. For this reason, the process for detecting features from the image data in the video processing unit 3 is not required, and the load on the CPU of the video processing unit 3 can be reduced. For this reason, it is possible to adopt a small and low cost CPU by the CPU of the video processing unit 3, and as a result, it is possible to reduce the size and cost of the entire image processing apparatus. Further, when the CPU of the video processing unit 3 has an existing processing capability, this capability can be used for processing other than the feature amount detection of the image data.
Further, according to the image data transfer method of the present invention, the label information can be transferred without greatly changing the configuration of the image data processing means in the conventional video system such as a surveillance camera. And the device can be configured.
In addition, according to the image data transfer method of the present invention, the label information and the image data can be transferred in synchronization, so that the specific image can be obtained in real time simply by ANDing the transferred label information and the image data. (For example, a person, a vehicle, etc.) can be extracted. In addition, since a plurality of feature amounts can be transferred by the same method, more accurate region extraction can be performed.

  In the present embodiment, the label information multiplexing unit 2 is substantially configured by hardware such as a logic circuit. For this reason, it is possible to simplify the process of detecting the feature amount of the image in the image processing rather than using the CPU or the like, and to reduce the load on the image processing of the entire image processing apparatus.

It is a block diagram for demonstrating the structure of the image processing apparatus of one Embodiment of this invention. It is a figure for demonstrating transfer of the Y signal of the video signal converted by the YUV411 format. It is a figure for demonstrating the process which calculates one representative value from a some pixel in a YUV411 conversion part. FIG. 4 is a diagram for explaining a transmission format of representative values calculated in the configuration shown in FIG. 3. It is a figure for demonstrating the more detailed structure of the labeling part shown in FIG. It is a figure which illustrates the structure which detects the color of a video signal as a feature-value. It is a figure for demonstrating the structure of a general noise removal filter. It is a figure for demonstrating a general binary pixel basic filter. It is a figure for demonstrating multiplexing of UV signal and label number of one Embodiment of this invention. It is a figure for demonstrating the process which calculates one representative value from a some label number. It is a figure for demonstrating the structure of the video processing apparatus of one Embodiment of this invention. It is a figure for demonstrating the structure of the UV signal and label number separation part shown in FIG. It is a figure for demonstrating the structure of the UV signal separation part shown in FIG. It is a figure for demonstrating the state of the video signal stored in VRAM shown in FIG. It is a figure for demonstrating the structure of the image process which is the prior art example of this invention.

Explanation of symbols

2 Label information multiplexing unit, 3 video processing unit, 101 electronic camera, 105 YUV411 conversion unit, 108 VRAM, 109 UV signal / label number multiplexing unit, 111 labeling unit, 113 area extraction unit, 115 label number encoder, 119b UV signal port 1101 UV signal separation synchronization unit 1102 UV signal storage unit

Claims (7)

Image data acquisition means for acquiring image data obtained by capturing an image, and image data processing means for processing the image data acquired by the image data acquisition means to generate display data capable of displaying an image in a predetermined format An image processing apparatus comprising:
Feature detection means for detecting features of the image data acquired by the image data acquisition means;
Label information specifying means for specifying label information indicating the feature detected by the feature detecting means;
An image processing apparatus comprising: image data transfer means for transferring the label information specified by the label information specifying means together with the image data to the image data processing means.   The image processing apparatus according to claim 1, wherein the image data transfer unit multiplexes the label information into a UV signal of a video signal including a luminance signal Y and a color difference signal UV.   The image data transfer means, when multiplexing the label information on the UV signal, multiplexes the UV signal on either the odd line or the even line of the video signal and the other one of the lines on which the UV signal is multiplexed. The image processing apparatus according to claim 2, wherein the label information is output at an output timing of the line.   The image processing apparatus according to claim 1, wherein the feature detection unit detects at least one feature among a color, a shape, and a reflectance of an imaging target indicated by the image data.   The image processing apparatus according to claim 1, wherein the feature detection unit detects a movement of an imaging target indicated by the image data as a feature.   6. The image processing according to claim 1, wherein the image data transfer unit transfers only label information relating to a representative value among the features detected by the feature detection unit together with the image data. apparatus. A feature detection step for detecting a feature of the image data; a label information specification step for specifying label information indicating the feature detected in the feature detection step;
An image data transfer step of transferring the label information to display data storage means capable of displaying an image together with the image data;
An image data transfer method comprising:

Images