JP2000184360A - Moving image monitoring system and its application system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce the computational complexity of a mobile object and to shorten the computing time by detecting a mobile object of every block from the decoding result of every block of the encoded moving image data, synthesizing the area of the mobile object with a reproduction image signal to display them and deciding the mobile object to display or output this deciding result. SOLUTION: The image data acquired by an image input means 101 are preserved by an input image recording means 103 in an encoding block 125. An image control means 104 sets both image size and image compressibility. Then the image data encoded by a moving image encoding means 105 are inputted to a decoding block 126. A moving image is recorded by a recording means 120 when the encoded data need the monitoring image data which are inputted to a moving image decoding means. When a mobile object detecting means 114 is used, the encoded data inputted to the moving image decoding means inputs the decoded data to a mobile object detecting block 128 and then to the means 114 and detects a mobile object included in an image under the condition of a mobile object control means 113.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture monitoring apparatus using image processing and an application system thereof.

[0002]

2. Description of the Related Art Conventionally, in a moving image monitoring apparatus, usually, in order to continuously monitor an image for a long time and store data, it is necessary to reduce the number of stored moving images or to lower the image quality. . For this reason, when the monitoring image is reproduced, it is difficult to conduct a detailed investigation from the reproduced video because the image quality is low. In addition, it is necessary to issue a warning or take an action after a person has watched and determined the monitoring contents.

Recently, moving objects can be detected by image processing, but a large amount of calculation is required, and there is a problem in terms of calculation time and cost.

In order to detect the color of an object in an image, it is necessary to check the color for each pixel. However, this requires a large amount of calculation, and there is a problem in terms of calculation time and cost. Furthermore, a still image as a monitoring result can be photographed and transmitted with high definition, but there is a problem that a large transmission capacity is required.

[0005]

SUMMARY OF THE INVENTION The present invention solves the above-mentioned problems of the prior art, and reduces the amount of calculation in detecting a moving object using image processing and detecting the color of an object in an image. A video that can reduce the time and cost and reproduce high-quality video even when monitoring is continued for a long time, automatically determine the monitoring content and display or warn as output, or can be used as an actuator input It is an object to provide an image monitoring device and an application system thereof.

[0006]

According to a first aspect of the present invention, there is provided a moving image monitoring system, comprising: an image input unit for photographing an object to be monitored and inputting an image signal; and a moving image for encoding the image signal as a moving image. Encoding means, moving picture decoding means for decoding encoded moving picture data, and decoding of each block decoded by the moving picture decoding means from a result of decoding so that an image in a block is in a static state or a moving state. Moving object detecting means for judging whether the moving object is determined from the result of the static motion determination for each block, and means for combining and displaying the determined area of the moving object with the reproduced image signal created by the moving image decoding means And at least a moving object is determined and displayed or output.

According to the first aspect of the present invention, in the moving picture monitoring system, it is determined whether the image in the block is still or moving from the decoding result for each decoded block, and the moving image is moved from the still / moving determination result for each block. Since the object is determined, the amount of calculation, the calculation time, and the cost can be reduced in the detection of the moving object, so that a moving image monitoring system can be configured quickly and without waste, and a wide range of applications can be obtained.

According to a second aspect of the present invention, there is provided a moving image monitoring system, comprising: an image input unit for photographing a monitoring target and inputting an image signal; a moving image encoding unit for encoding the image signal as a moving image; Moving image decoding means for decoding the decoded moving image data; and determining, for each block decoded by the moving image decoding means, whether the image in the block is still or moving from the decoding result. Moving object detecting means for determining a moving object from the result of the static motion determination, specific color detecting means for detecting an object of a specific color, display of a region where the moving object and the specific color are detected, and creation by the moving image decoding means Image synthesizing means for synthesizing the reproduced image,
Means for combining and displaying the determined area of the moving object with the reproduced image signal created by the moving image decoding means, and determining or displaying or outputting the moving object and the specific color object. Is what you do.

According to the second aspect of the present invention, in the moving image monitoring system, it is determined whether the image in the block is still or moving from the decoding result for each decoded block, and the moving image is moved from the still / moving determination result for each block. Since the object is determined and the specific color is detected, the amount of calculation and the calculation time and cost can be reduced in the detection of a moving object and the detection of a specific color object. A monitoring system can be configured for a wide range of applications.

According to a third aspect of the present invention, there is provided a moving image monitoring system, comprising: an image input unit for photographing a monitoring target and inputting an image signal; a moving image encoding unit for encoding the image signal as a moving image; A still image encoding unit and a still image data storage unit for encoding as an image, a moving image decoding unit for decoding the encoded moving image data, and a static image decoding unit for decoding the encoded still image data Image decoding means;
A moving object detecting unit that determines whether the image in the block is still or moving from the decoding result for each block decoded by the moving image decoding unit and determines a moving object from the still / moving determination result for each block; A specific color detecting means for detecting a color object; an image synthesizing means for synthesizing a display of a moving object and a region where the specific color is detected and a reproduced image created by the moving image decoding means; Means for synthesizing and displaying the area with the reproduced image signal created by the moving image decoding means, and a still image display means for displaying a still image, and further processes and outputs the determination result of the moving object. In addition to the output means, the still image display means includes means for calling, transmitting, decoding, and displaying the still image data stored upon request. According to the third aspect of the present invention, in addition to the moving object detecting means and the specific color object detecting means, there is provided a means for retrieving, transmitting, decoding, and displaying the still image data stored in response to the request. It is possible to retrieve detailed information by calling a still image with good image quality.

According to a fourth aspect of the present invention, there is provided a moving image monitoring system, comprising: an image input unit for photographing a monitoring target and inputting an image; an input image recording unit for recording the input image; Time adding means for managing, image adjusting means for adjusting an image, moving image encoding means for encoding an image signal from the image adjusting means as a moving image, and encoding control means for controlling each encoding A still image encoding synchronizing unit for selecting a specific image and instructing the encoding control unit to encode a still image, and inputting a specific still image from an input image recording unit in response to an instruction from the encoding control unit to perform encoding. And a still image decoding unit for decoding the encoded still image data; a still image display unit for displaying the decoded still image; Decrypt Moving image decoding means, recording means for storing encoded moving image data using a recording means, and decoding of each block decoded by the moving image A moving object detecting unit that determines whether the object is in a state and determines a moving object from a result of the static motion determination for each block; a specific color detecting unit that detects an object of a specific color from the result detected by the moving object detecting unit; An image synthesizing unit for synthesizing a display of an area where a specific color is detected and a reproduced image created by the moving image decoding unit; a moving image display unit for displaying the synthesized image; and a decoded still image And a display means including a still image display means for displaying a moving image.

According to the invention of claim 4, the present invention can be effectively implemented.

According to a fifth aspect of the present invention, in the moving image monitoring system, the moving image monitoring system includes a multiplexing unit that multiplexes the encoded moving image data and the still image data, and a multiplexing unit that multiplexes the multiplexed data. It has a transmitting / receiving stage for transmitting and receiving, and demultiplexing means for separating multiplexed data.

According to the fifth aspect of the present invention, by transmitting image data and the like using a transmission path, it is possible to monitor a moving image from a distance by the system of the present invention.

According to a sixth aspect of the present invention, there is provided a fire monitoring system, wherein a moving object in a moving image is subjected to a flame by performing the moving image monitoring for an image input of a target object using the moving image monitoring system. Alternatively, fire detection and / or fire situation determination is performed by detecting smoke, and the fire notification and alarm are automatically performed.

According to the sixth aspect of the present invention, it is possible to accurately detect the occurrence of a fire and determine the state of a fire by detecting a moving object and a specific color.

According to a seventh aspect of the present invention, there is provided a costume changing system according to the present invention, wherein an image of a target person is taken using the moving image monitoring system, an image is input, and a moving image is monitored, so that the size of the person can be adjusted. Means for enlarging and reducing the size of the costume and displaying it on a person so as to match the costume and the like.

According to the invention described in claim 7, it is possible to quickly grasp the target person and adjust the costume. It is also easy to match costumes in remote locations.

The wheelchair guidance system according to claim 8 of the present invention uses the moving image monitoring system to perform moving image monitoring on an image input of an object taken from an image input device mounted on a wheelchair. The present invention is characterized in that a wheelchair is automatically driven by following the moving object or a specific color and can be moved to a required place.

According to the eighth aspect of the invention, for example, using a wheelchair in a hospital or the like, the passage of the hospital is color-coded, so that the procedure from issuing a consultation ticket at a reception desk to a required medical department is automatically performed by a wheelchair. Provides guidance, performs specific color detection on moving image data input as a moving image attached to a wheelchair, and automatically drives the wheelchair according to the target color detected by the specific color detection to the color specified in advance in the consultation ticket. Or go. In addition, a moving object or an obstacle that crosses ahead while moving is detected by the moving object detection, and a warning and an avoidance can be performed.

According to a ninth aspect of the present invention, there is provided an intersection monitoring system, wherein an image of an intersection is taken by using the moving image monitoring system, an image is input, and a moving image is monitored. By monitoring the status of
The moving image monitoring system according to any one of claims 1 to 5, wherein the automatic acquisition of a warning and a still image is performed unattended.

According to the sixth aspect of the present invention, a moving object of moving image data is detected using a moving image input device installed at an intersection, and a moving object of moving image data is detected at an intersection. Detects the color of a moving object that has entered it and detects specific colors, monitors the movement of moving vehicles and people, monitors the color of signals and monitors the color of vehicles by detecting specific colors, and ignores signals, etc. The system can automatically issue voice alerts and, if necessary, automatically acquire still images to monitor and warn about accidents and the movement of people and vehicles. According to a tenth aspect of the present invention, there is provided a road surveillance system having an invisible line of sight, which captures an image of a road with an unclear line of sight by using the moving image monitoring system, inputs an image, and performs a moving image monitoring. An unsightly road surveillance system that detects a moving object on an unfamiliar road and gives a necessary warning.

According to the seventh aspect of the present invention, a road monitoring system having poor visibility is constructed using the moving image monitoring system according to the first or second aspect. Using a moving image input device installed at a corner with poor visibility, moving object detection of moving image data is performed, and the movement of cars and people moving on an unseen road can be monitored, and signals such as signals And automatically give voice alerts to prevent accidents. It is possible to provide a road monitoring system with poor visibility that has means for monitoring and warning of accidents and the movements of people and vehicles by performing automatic income of a still image by controlling and the like.

A vehicle speed monitoring system according to an eleventh aspect of the present invention is characterized in that a vehicle running by using the moving image monitoring system is photographed, an image is input, and the moving image monitoring is performed. And a warning is issued to a vehicle exceeding the speed limit, so that a still image can be automatically acquired.

According to the eleventh aspect of the present invention, a moving object can be detected from moving image data using a moving image input device installed on a road or a downhill where a speed can easily be increased. That is, the speed can be calculated from the change in the size of the moving object that has entered the moving image, the speed of the vehicle can be monitored and a warning can be issued, and the control can be performed. It is possible to provide a speed monitoring system having a means for performing the above.

According to a twelfth aspect of the present invention, there is provided the sports form analysis system, wherein the moving image monitoring system is used to photograph a batter having a mark of a specific color attached to each part of a body or a tool, input an image, and input the image. By performing image monitoring,
It is characterized by performing comparative analysis of sports forms.

According to the twelfth aspect, a sports monitoring system can be constructed. A system that checks a form using moving object detection and specific color detection. For example, when checking the batting form of baseball, attach colored marks to the tip of the bat, elbows, shoulders, hips, head, and knees, monitor the video, and monitor the movement of the body, the trajectory of the bat, and the ball. By detecting, it is possible to check the backing form and compare it with the correct form.

According to a thirteenth aspect of the present invention, the automatic elevator control system according to the thirteenth aspect uses the moving image monitoring system to capture an image of the vicinity of a door on each floor of an elevator, input an image, and perform the moving image monitoring. It is characterized in that a stop of a moving object near a door is detected and input as an elevator call signal.

The automatic elevator control system according to a fourteenth aspect of the present invention is the automatic elevator control system, wherein the number of moving objects stopped near doors on each floor of the elevator is detected, and the elevator is controlled in accordance with the number of moving objects. It is characterized by performing priority control.

According to the twelfth and thirteenth aspects of the present invention, it is possible to automatically measure and input a person who comes near a door on each floor of an elevator and input the same, thereby stopping the elevator or controlling the priority of the elevator.

According to a fifteenth aspect of the present invention, there is provided the passer management system, wherein the moving image monitoring system is used to capture an image of a person passing through a specific area, input an image, and monitor the moving image, thereby monitoring the passing of the specific area. It is characterized by automatically identifying persons and performing passage management. According to the fifteenth aspect, it is possible to detect and manage a participant in a sport having a large number of participants, such as a marathon sport, by detecting a moving object and detecting a specific color. In marathon competitions, the number of marathon runners in each color is automatically recorded using specific color detection at each point using colored bib numbers instead of bib numbers, so that even a very large number of people can grasp the situation in real time can do.

[0032] The entrance / exit monitoring system according to claim 16 of the present invention monitors the input of an image of the entrance / exit of the area specified by using the moving image monitoring system to monitor the input / output of the area. It is characterized by performing area management.

According to the fifteenth aspect of the present invention, for example, when entering and exiting a building, detection is performed by detecting a moving object and a specific color to perform management. By using cards and batches with a specific color in the entrance and exit areas and automatically recording the time when entering that color and the ratio of the color of clothes, etc., it is possible to grasp a very large number of visitors. it can.

A shelf monitoring system according to a seventeenth aspect of the present invention manages a shelf by performing a moving image monitoring on an image input of an image of the article on the shelf and the periphery of the shelf using the moving image monitoring system. It is characterized by performing.

According to the seventeenth aspect, for example, the position of a shelf such as a department store is designated as a range, and a moving object entering the range and a color mark attached to food are monitored.
Recording of a moving image is performed only when there is a change in the moving image within the range, and monitoring for a long time, abnormality monitoring on a shelf or the like, and inventory management can be performed.

[0036] The water pollution degree monitoring system according to claim 18 of the present invention manages the water pollution degree by monitoring the moving image with respect to an image input photographing the water surface using the moving image monitoring system. In addition, an alarm is issued when the degree of contamination exceeds a predetermined threshold.

According to the eighteenth aspect of the invention, for example, a wide-range moving image of a lake surface or a sea surface is input, and the presence or absence and range of the detected color are recorded. A water pollution warning can be issued when it exceeds. For the detection of a specific color, a color corresponding to water pollution, such as blue-green algae or red tide, can be used as a designated color to perform the detection operation without waste.

The astronomical moving image monitoring system according to the nineteenth aspect of the present invention monitors a moving image in response to an input of an astronomical moving image, specifies a color corresponding to a supernova, performs a color detection operation, and outputs a color. It detects a star based on the presence or absence of a star, compares it with a database that records known stars, and issues a notification output if it does not exist in the database.

According to the nineteenth aspect of the present invention, a moving image is monitored for an input of a celestial moving image, a detection operation is performed using a color corresponding to a supernova as a designated color, and a database in which known star positions are recorded is provided. If the detected star does not exist in the database, the detected location and time can be recorded and a notification output can be issued.

According to a twentieth aspect of the present invention, in the home surveillance system according to the first aspect of the present invention, when a partial moving image or a whole moving image of a platform at a station is input using the moving image monitoring system, the color of a white line on the platform is designated as a designated color. The image monitoring is performed, and the monitoring result is notified to the train by the communication unit.

According to the twentieth aspect of the present invention, a partial moving image or a whole moving image of the platform at the station is input, and the color of the white line on the platform is set as the designated color, so that the change of the white line on the platform causes a person to enter the track. Can be detected as approaching,
The train can drive safely by receiving this result.

A runway passage detection system for an airplane according to claim 21 of the present invention is adapted to detect an image of an airport runway and a taxiway using the video monitoring system.
By monitoring the moving image, it is detected that an airline company having a predetermined color passes through the runway.

According to the twenty-first aspect of the invention, the color of the airplane of each airline is set as the designated color in advance, and the output which does not correspond is removed according to the moving speed, the moving direction, or the designated color condition, and only the selected necessary information is obtained. Can be output.

A high-temperature iron monitoring system according to a twenty-second aspect of the present invention uses the moving image monitoring system to input a moving image of a high-temperature iron on a production line of an ironworks, and The moving image monitoring is performed by using a color as a designated color, and the monitoring result is notified to a manufacturing control device using a communication unit.

According to the twenty-second aspect of the present invention, the color of iron having a normal temperature is designated as a specific color, a color detection operation is performed, and the color is notified to the manufacturing control device. Temperature control can be performed so that the color range of temperature is widened.

In the hazardous area monitoring system according to claim 23 of the present invention, a moving image of an area at risk of landslide using the moving image monitoring system is input, and the moving image is set using a previously designated color of the earth and sand as a designated color. Image monitoring is performed, and an alarm is output when the monitoring result value exceeds a specified threshold value.

According to the twenty-third aspect of the present invention, a moving image is input by photographing from a position where a landslide danger area or a bank of a river can be seen. Dangerous area monitoring can be performed by outputting an alarm when the value of the color range change or the value of the color shift exceeds the specified threshold value.

According to a twenty-fourth aspect of the present invention, there is provided a road guidance monitoring system using the moving image monitoring system, wherein a color of a traffic light is designated as a designated color in response to a moving image input from an image input device installed in a vehicle. By performing the monitoring, the signal state of the traffic light is monitored and the traffic guidance is performed.

The road guidance monitoring system according to a twenty-fifth aspect of the present invention is the road guidance monitoring system,
It is characterized in that the position accuracy and the guidance accuracy are improved by interlocking with the car navigation using the PS.

According to the inventions described in claims 24 and 25, the signal state of the traffic signal is monitored and the traffic is guided, so that the vehicle can travel safely.

[0051]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be specifically described in the order of the claims.

Embodiment 1 (Embodiments of Claims 1 to 4) FIG. 1 is a block diagram showing a configuration of a system according to an embodiment of the present invention. In FIG. 1, an encoding block 125 from the image input means to each encoding, a decoding block 126 from the decoding to each decoding, each detecting means and a display means.
In the encoding block 125, the image data obtained by the image input means 101 is stored in the input image recording means 103. At this time, the acquisition time of the input image is added from the time adding means 102 and stored. The stored moving image data is adjusted by the image adjusting means 104 in accordance with encoding such as image size, image compression ratio, and color correction.

The image adjusting means 104 also serves as condition setting means such as image size, image compression ratio, color correction and the like. It is input to the moving picture coding means 105 and coded. Further, a still image encoding unit is provided. The still image has a structure in which the still image is branched from the input image recording means 103 and direct output data is input to the still image encoding means 115 for encoding to obtain an image close to the acquired original image. Each encoding unit performs encoding according to an instruction from the decoding control means 126. The encoding control means 116 re-acquires the specified time image data from the decoding block 126
To order. Upon receiving the instruction, the still image encoding unit 115 acquires the image data of the designated time from the input image recording unit 103, encodes the still image, and transmits it to the decoding block.

The coded image data is a decoded block
Entered into 126. The encoded still image data is input to the still image decoding means 118 and decoded. The still image encoding / synchronizing means 117 confirms that the data of the still image decoding means 118 is the requested data from the time information added to the image data in the encoded data. The decoded still image data is input to the still image display means 119 and displayed on the screen.

As for the moving image, the encoded data is transferred to the moving image decoding means 10.
When the monitoring image data input to 8 is required, the encoded data is recorded in the recording means 120 as it is. Recording means
The reason for recording the encoded data in 120 is that the moving object detection means 114 and the specific color detection means 112 necessary for automatic monitoring have stored various detections in order to perform detection based on the data obtained by the moving picture decoding means 108. Save the data before decryption to perform from the data.

When the moving object detecting means is used, the coded data inputted to the moving picture decoding means 108 enters the data in the decoding process into the moving object detecting block 128 and the moving object detecting means 128.
The moving object in the image is detected under the conditions of the moving object adjusting means 113. The moving object adjusting means 113 detects the minimum macro block of the moving object, the maximum macro block of the detection,
It is provided with settings such as detection position and detection sensitivity.

The output of the moving object detecting means 114 is input to the specific color detecting means 112 in the specific color detecting block 129 as required, and the specific color is detected under the conditions of the specific color adjusting means 111. The specific color adjustment unit 111 sets a specific color, determines whether or not to perform a specific color detection on the macroblock detected by the moving object detection unit 114, searches for colors in the macroblock detected by the moving object detection, and specifies a specific color. Calculation of the specific color ratio in the macroblock detected by the detection can be set.

The moving object detecting means 114 and the specific color adjusting means 11
The macroblock information detected in step 1 is the detection result output means.
The data is output from the 121 and can be registered in the database and calculated as needed. At the same time, the image synthesizing unit 109 synthesizes the macroblock display of the moving object detection result and the specific color detection result on the decoded image in order to reflect the image on the output image of the moving image decoding unit 108 and display it. The system can be displayed at 110.

Here, a description will be given of the still image acquisition as an explanation of the characteristic functional unit. FIG. 4 is a block diagram showing an example of the configuration of the encoding unit 125. The input video signal sent from the image input means 101 is first converted by an A / D unit 201 from an analog signal to a digital signal. The digitized video signal passes through a switch 202 for determining whether or not to encode the frame, and is then input to a format conversion unit 203. The format converter 203 converts the RGB signals into YCbCr
The signal is converted into a signal and input to the image size conversion unit 204 and the input image buffer 206. The image size conversion unit 204 applies a low-pass filter to the input video signal, down-samples the image signal to reduce the image size, and inputs the reduced image size to the video encoding unit 205. The moving image encoding unit 205 encodes the input video signal as a moving image, and inputs the encoded data to the multiplexing unit 210. On the other hand, the encoding control unit 209 turns on the switch 207 when receiving a retransmission request signal from the feedback path 106,
A frame that matches the time code of the retransmission request signal is selected from the input image buffer, encoded by the still image encoding unit 208, and sent to the decoding unit. The coding control unit 209 determines coding parameters using the coding results of the moving picture coding unit 203 and the still picture coding unit 208, and controls the code amount.

FIG. 5 is a block diagram showing an example of the configuration of the moving picture coding unit 105. In FIG. 5, an input video signal is divided into macroblocks by a blocking unit 301. The input video signal divided into macroblocks is input to a subtractor 302, and the difference from the predicted image signal is calculated to generate a prediction residual signal. This prediction residual signal,
One of the input video signals from the blocking unit 301 is selected by the mode selection switch 303, and is subjected to discrete cosine transform by the DCT (discrete cosine transform) unit 304. The DCT coefficient data obtained by the DCT section 304 is
Quantized by 5. The signal quantized by the quantizer 305 is
The signal is branched into two, and one of them is subjected to variable-length coding by a variable-length coding unit 313. On the other hand, the other of the two signals that have been quantized by the quantization unit 305 and subjected to inverse processing by the inverse quantization unit 306 and the IDCT (inverse discrete cosine transform) unit 307 are the same as those of the quantization unit 305 and the DCT unit 304. After receiving in order, switch by adder 308
A local decoded signal is generated by adding the predicted image signal input via the input terminal 311 to the predicted image signal. This local decoded signal is stored in the frame memory 309 and input to the motion compensation unit 310. The motion compensation unit 310 generates a predicted image signal and sends necessary information to the mode selection unit. The mode selection unit 312 includes a motion compensation unit for each macroblock.
Based on the prediction information P from 310, a macroblock for performing interframe coding and a macroblock for performing intraframe coding are selected. When performing intra-frame coding (intra-coding), the mode selection switch information
Let M be A and switch information S be A. When inter-frame coding (inter coding) is performed, the mode selection switch information M is set to B, and the switch information S is set to B.
In the mode selection switch 303, the mode selection switch information P
, And the switch changes the switch in the switch 116 based on the switch information S. Here, the modes include an intra mode (INTRA), an inter mode (INTER), and a non-coding mode (NOT_CODED), which are associated with each macroblock, and the INTRA macroblock is an image area to be intra-coded. , INTE
The R macroblock is an image area to be coded between frames, and the NOT_CODED macroblock is an image area that does not need to be coded.

FIG. 6 is a block diagram showing an example of the configuration of the still image encoding unit 115. In FIG. 4, an input video signal is divided into macroblocks by a blocking unit 401. The input video signal divided into macro blocks is DC
A discrete cosine transform is performed by a T (discrete cosine transform) unit 402. The DCT coefficient data obtained in the DCT section 402 is quantized in the quantization section 403. The quantized signal is variable-length coded by a variable-length coding unit 404 and sent to a decoding unit 126. FIG. 7 is a block diagram showing an example of the configuration of the decoding unit 126. The encoded data sent from the encoding unit 125 is first encoded video data and encoded still image data,
Each is composed of a moving image decoding unit 503 and a still image decoding unit 504.
Send to The moving picture decoding section 121 decodes the coded moving picture data, creates a reproduced picture signal, and sends it to the display section 110.
On the other hand, the still image decoding unit 118 decodes the still image data, creates a reproduced image signal, and sends it to the display unit 119.

FIG. 8 is a block diagram showing an example of the configuration of the moving picture decoding unit 121. In the variable length decoding unit 601, if the mode of the macroblock is INTRA, the mode changeover switch 607 is turned off, and the variable length decoding unit 601
The quantized DCT coefficient information decoded in
The IDCT unit 603 performs inverse discrete cosine transform processing to generate a reproduced image signal. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the display unit 110. In the variable length decoding unit 601, the mode of the macroblock is set to INTER.
And NOT_CODED, the mode changeover switch 607 is turned off, and the quantization decoded by the variable length decoding unit 601 is performed.
The DCT coefficient information is inversely quantized by the inverse quantization unit 602, and the
The inverse discrete cosine transform process is performed by the unit 603, and the motion compensation unit 606 performs motion compensation on the reference image based on the motion vector information decoded by the variable length decoding unit 601. An image signal is generated.
The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the display unit 110.

FIG. 9 is a block diagram showing an example of the configuration of the still image decoding unit 118. In the variable length decoding unit 701,
The quantized DCT coefficient information decoded by the variable length decoding unit 701 is inversely quantized by the inverse quantization unit 702, and the IDCT unit 703 performs inverse discrete cosine transform processing to generate a reproduced image signal. This reproduced image signal is input to the display unit 110.

Next, detection of a moving object will be described.
In FIG. 10, encoded data is output to the variable length decoding unit 601 in the video decoding unit 121. Variable length decoding unit 60
In step 1, the variable length code of the information of each syntax is decoded.

In the variable length decoding unit 601, if the mode of the macroblock is INTRA, the mode changeover switch 607
Is turned off, and the quantized DCT coefficient information decoded by the variable-length decoding unit 601 is inversely quantized by the inverse quantization unit 602, and the IDCT unit 603 performs inverse discrete cosine transform processing. A reproduced image signal is generated. The reproduced image signal is stored in the frame memory 605 as a reference image, while the moving object synthesis display unit 814 in the moving object detection unit 128
Is input to

In the variable length decoding unit 601, if the mode of the macroblock is INTER and NOT_CODED, the mode changeover switch 607 is turned off, and the variable length decoding unit
The quantized DCT coefficient information decoded in
The IDCT unit 603 performs inverse discrete cosine transform processing, and based on the motion vector information decoded by the variable length decoding unit 601, a motion compensation unit 606 performs motion compensation on the reference image, and At 604, a reproduced image signal is generated. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the moving object synthesis display unit 814 in the moving object detection unit 128.

On the other hand, in the moving object detection unit 128,
As shown in the flowchart of 11, the macroblock static / movement determination unit 811 performs three processes of macroblock static / movement determination (S101), moving object determination (S102), and moving object composite display (S103) for each frame. The object determination unit 813 and the moving object synthesis display unit 814 perform the processing.

The macroblock still / movement determining section 811 includes the mode information, motion vector information, DCT coefficient information decoded by the variable length decoding section 601 and the INTRA calculated by the subtractor 812.
From the sum of absolute differences (SAD) between the reproduced image signal at the time of the macroblock and the frame memory of one frame before, a still / moving determination for each macroblock in the frame is performed.

FIG. 12 is a flowchart for determining whether or not a macroblock is moving (S101).
3 is a specific flowchart of FIG. Here, i and j represent the addresses of the vertical and horizontal macroblocks in the frame, respectively, and V_NMB and H_NMB represent the numbers of the vertical and horizontal macroblocks in the frame. The two-dimensional array M [i] [j] is an array for storing information as to whether each macroblock is a moving macroblock. TRUE indicates a moving macroblock, and FALSE indicates a still macroblock.

First, a variable length decoding unit is provided for each macroblock.
The mode information MODE from 601 is determined (S203). If MODE is INTRA, the sum of absolute differences (SAD) between the reproduced image signal of the macroblock and the frame memory one frame before is calculated and compared with the threshold value T0 (S204). If the macroblock is larger than the threshold T0, the macroblock is determined to be a moving macroblock, and TRUE is substituted for M [i] [j] (S206). If the value is equal to or smaller than the threshold value T0, the macro block is determined to be a still macro block, and M [i] [j] is set to FALSE.
Is substituted (S207). If MODE is INTER, from the motion vector information and DCT coefficient information from the variable length decoding unit 103, the sum of absolute values of motion vectors of the macroblock Σ | MV | and the sum of absolute values of DCT coefficients Σ | COF Is calculated and compared with the respective thresholds T1 and T2 (S205). If the sum of the absolute value of the motion vector and the sum of the absolute values of the DCT coefficients are smaller than the threshold, the macroblock is determined to be a still macroblock, and FALSE is substituted for M [i] [j] (S207). . Otherwise, the macro block is determined to be a moving macro block, and TRUE is substituted for M [i] [j] (S206). If NOT_CODED, the macroblock is determined to be a still macroblock, and M [i]
FALSE is substituted for [j] (S207). Moving object determination unit 8
In 13, a moving object is determined from the static / movement determination information for each macroblock by the macroblock static / movement determination unit 811.

FIG. 13 is a specific flowchart. Figure 4
As shown by, the processing content is composed of two parts, a noise removal processing (S301) and a moving object inclusion processing (S302). In the noise removal processing (S301), in order to prevent false detection of a moving macroblock due to fluctuations of small objects in the background image or noise at the time of image capturing, a moving macroblock in which all surrounding 8 macroblocks are still is used. Removed.
The moving object inclusion process (S302) performs a process of detecting a minimum rectangle that includes an adjacent region where a moving macroblock exists, from the still / moving determination result after removing noise. Through this process, a rectangle that includes the moving object can be found.

FIG. 14 shows the specific processing contents of the noise removal processing. As in FIG. 12, i and j represent the addresses of the vertical and horizontal macroblocks in the frame, respectively, and V_NMB and H_NMB represent the numbers of the vertical and horizontal macroblocks in the frame. Two-dimensional array M [i]
[j] is an array for storing information as to whether or not each macroblock is a moving macroblock.
FALSE indicates a still macroblock.

First, the result of the still / moving judgment for each macroblock is examined. (S403) If the value of the two-dimensional array M [i] [j] is FALS
E, that is, if it is a still macroblock, do nothing in that macroblock and go to the next macroblock. If the value of the two-dimensional array M [i] [j] is TRUE, that is, if it is a moving macroblock, the static / moving determination results of eight macroblocks around the macroblock are checked (S405).
FALSE, that is, if the macroblock is a still macroblock, the macroblock is determined to be noise and rewritten to a still macroblock (S406). If at least one of the eight surrounding macroblocks is TRUE, it is not determined to be noise, and the process moves to the next macroblock. The outside of the screen is assumed to be a still macroblock.

FIG. 15 shows the specific processing contents of the moving object inclusion processing. Here, n is a counter indicating the number of moving objects, S1 to S4 are parameters indicating a search range of a rectangle including the moving object, and S1 and S2 are start and end points of a vertical address. Yes, S3 and S4 are the start and end points of the horizontal address.

In the flowchart of FIG. 15, initialization is performed so that the entire frame is designated as a search range (S501). Next, a function Rectangular for searching for the smallest rectangle including the moving object in the specified search range is called (S502).

The flowcharts of FIGS. 16 and 17 show the function Rectangular.
Shows the contents of the processing. The function Rectangular receives a search range S1 to S4, n indicating the number of moving objects, and a two-dimensional array M in which the result of static / dynamic determination of each macroblock is stored, and stores a rectangular address of the search result. The dimensional arrays B1 to B4 and n indicating the number of moving objects are output. Here, the one-dimensional array HV is a working array for creating a histogram of the number of moving macroblocks in the vertical direction, and the one-dimensional array HH is used for creating a histogram of the number of moving macroblocks in the horizontal direction. It is a working array. Also, the variable VF
LAG is a flag that changes to TRUE when the value of the horizontal histogram is not 0, and changes to FALSE when the value of the histogram is 0.The variable HFLAG is a flag that indicates that the value of the vertical histogram is not 0. This flag is changed to TRUE when the value is 0 and FALSE when the value is 0.

First, the range of S1 to S2, which is the search range of the work array HV for creating a histogram of the number of moving macroblocks in the vertical direction, is initialized to a value of 0. (S601) Next LO
In the double loop of OP1 and LOOP2, a histogram of the number of moving macroblocks in the vertical direction in the search range is created. That is, the static / dynamic determination result M [i] for each macroblock
The value of [j] is compared (S604), and if the value is TRUE, that is, if it is a moving macroblock, HV [i] is incremented by 1 (S605).
If FALSE, the action is to do nothing.
Next, a non-zero continuous part is searched from the vertical histogram. First, set the flag VFLAG to FALSE
Set to. (S608) Next, it is checked whether the histogram HV is not 0 and VFLAG is FALSE in the order of the search range S1 to S2. (S610) This condition corresponds to the start point of a continuous portion where the histogram is not 0. Therefore, since it is a candidate for the starting point in the vertical direction of the rectangle being searched, the address i is stored in the one-dimensional array B1 [n], and VFLAG is set to TRUE. (S611) Next, when the histogram HV is 0 or at the end point of the search range, VFLAG is
Check if it is TRUE. (S612) The condition corresponding to this condition is the end point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the vertical end point of the rectangle being searched, if the histogram HV is 0
In the case of (1), the address i-1 is stored in the one-dimensional array B2 [n] (S614). Otherwise, the address i is stored in the one-dimensional array B2 [n] (S615). And VFLAG again FALSE
(S611). FIG. 17 is a continuation of the flowchart of FIG. This time, the range of S3 to S4, which is the search range of the work array HV for creating a histogram of the number of moving macroblocks in the horizontal direction, is initialized to a value of 0 (S617). Next LOOP
In the double loop of 4 and LOOP5, a histogram of the number of horizontal moving macroblocks in the search range is created.
That is, the value of the static / movement determination result M [i] [j] for each macroblock is compared (S604), and if the value is TRUE, that is, if it is a moving macroblock, HH [i] is incremented by 1 (S605). In the case of FALSE, the action that does nothing is taken. Next, a non-zero continuous portion is searched from the horizontal histogram. First, the flag HFLAG is set to FALSE. (S624) Next, it is checked whether the histogram HH is not 0 and the HFLAG is FALSE in the order of the search ranges S3 to S4 (S626). This condition is applied to the starting point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the starting point in the horizontal direction of the rectangle being searched, the address j is stored in the one-dimensional array B3 [n], and the HFLA
Set G to TRUE. (S627) Next, the histogram HH
Is 0 or at the end of the search range, check if HFLAG is TRUE. (S628) The condition that satisfies this condition is the end point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the horizontal end point of the rectangle being searched, if the histogram HH is 0, the address j-1 is stored in the one-dimensional array B4 [n] (S63).
0), otherwise, store address j in one-dimensional array B4 [n]. (S631) Then, HFLAG is set to FALSE again (S632).

Here, the search using one vertical and horizontal histogram has been completed.
It is checked whether or not n] to B4 [n] matches the search range S1 to S4 (S633). If they match, there is no more search range, so it can be determined that the minimum rectangle has been obtained. (S634). Then, n representing the number of moving objects is incremented by one (S635), and the process proceeds to the search for the next moving object. If the search results B1 [n] to B4 [n] do not match the search ranges S1 to S4, a plurality of moving objects still exist in the search result range, and the search results B1 [n] to B4 [n ] S1 ~ S4
And the function Rectangular is called again (S637).

FIG. 18 shows an example of the judgment result of the moving object judgment unit 113. Here, two moving objects were determined. As shown in the figure, the result of each determination is that, in this example, the origin of the address of the macro book is at the upper left of the frame, and B1
A moving object is determined by an arrangement such as [0] to B4 [0] and B1 [1] to B4 [1].

The moving object synthesis display section 814 creates an image in which the moving object area determined by the moving object determination section 813 and the reproduced moving image signal are synthesized. FIG. 19 is a specific flowchart. Here, n indicates the number of moving objects in the frame obtained by the moving object determination unit 813. The processing flow is as follows. For each moving object, B1 [i] to B4 [i] represent the four corner macroblocks of the moving object, so draw a rectangular white line surrounding it, Combine (S602).

FIG. 20 is an example of an image created by the moving object synthesis display unit.

Next, detection of a moving object will be described.

In FIG. 21, in the moving picture decoding section 121, the encoded data is output to the variable length decoding section 601.
The variable length decoding unit 601 decodes a variable length code of information of each syntax. In the variable length decoding unit 601,
If the mode of the macroblock is INTRA, the mode changeover switch 607 is turned off, and the quantized DCT coefficient information decoded by the variable length decoding unit 601 is inversely quantized by the inverse quantization unit 602, and the IDCT A reproduction image signal is generated by performing an inverse discrete cosine transform process in the unit 603. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the specific color object synthesis display unit 817 in the specific color detection unit 128.

In the variable length decoding section 601, if the mode of the macroblock is INTER and NOT_CODED, the mode changeover switch 607 is selected to be off, and the variable length decoding section
The quantized DCT coefficient information decoded in
The IDCT unit 603 performs inverse discrete cosine transform processing, and based on the motion vector information decoded by the variable length decoding unit 601, a motion compensation unit 606 performs motion compensation on the reference image, and At 604, a reproduced image signal is generated. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the specific color object synthesis display unit 817 in the specific color object detection unit 129.

On the other hand, in the specific color object detecting section 129,
As shown in the flowchart of FIG. 22, the macroblock static / moving determination unit performs three processes of macroblock static / movement determination (S701), specific color object determination (S702), and specific color object composite display (S703) for each frame. 811, the specific color object determination unit 818 and the specific color object synthesis display unit 817 perform the processing.

The macroblock still / movement determining unit 811 includes the mode information, motion vector information, DCT coefficient information decoded by the variable length decoding unit 601 and the INTRA calculated by the subtractor 812.
From the sum of absolute differences (SAD) between the reproduced image signal at the time of the macroblock and the frame memory of one frame before, a still / moving determination for each macroblock in the frame is performed.

FIG. 23 is a specific flowchart of the macroblock still / movement determination (S801). Here, i and j represent the addresses of the vertical and horizontal macroblocks in the frame, respectively, and V_NMB and H_NMB represent the numbers of the vertical and horizontal macroblocks in the frame. A two-dimensional array
M [i] [j] is an array for storing information as to whether each macroblock is a moving macroblock. TRUE indicates a moving macroblock, and FALSE indicates a still macroblock.

First, a variable length decoding unit is provided for each macroblock.
The mode information MODE from step 103 is determined (S803).

If MODE is INTRA, the sum of absolute differences (SAD) between the reproduced image signal of the macroblock and the frame memory one frame before is calculated and compared with the threshold value T0 (S804). If it is larger than the threshold value T0, the macroblock is determined to be a moving macroblock, and M [i] [j]
Is set to TRUE (S806). If MODE is INTER, the motion vector information from variable length decoding section 601 and DC
From the T coefficient information, the sum of the absolute value of the motion vector of the macroblock Σ | MV | and the sum of the absolute values of the DCT coefficients Σ | COF | are calculated and compared with the respective thresholds T1 and T2 (S805).
If the sum of the absolute value of the motion vector and the sum of the absolute values of the DCT coefficients are equal to or larger than the threshold, the macroblock is determined to be a moving macroblock, and TRUE is substituted into M [i] [j] (S806). The specific color object determination unit 818 obtains the static / movement determination information for each macroblock from the macroblock
A specific color object is determined. FIG. 24 is a specific flowchart. As shown in FIG. 4, the processing content includes two processes: a specific color detection process (S901) and a specific color object inclusion process (S902). In the specific color detection processing (S901), in order to prevent a small object in the background image from fluctuating or noise at the time of capturing the image, it is erroneously detected as a moving macroblock.
8 Moving macroblocks in which all macroblocks are still are removed. The specific color object inclusion process (S902) performs a process of detecting a minimum rectangle that includes an adjacent region where a moving macroblock exists, from the still / moving determination result after removing noise. Through this processing, a rectangle that includes the specific color object can be found.

FIG. 25 shows the specific processing contents of the specific color detection processing. As in FIG. 23, i and j represent the addresses of the vertical and horizontal macroblocks in the frame, respectively, and V_NMB and H_NMB represent the numbers of the vertical and horizontal macroblocks in the frame. The two-dimensional array M [i] [j] here is an array for storing information as to whether each macroblock is a specific color macroblock, and if TRUE, indicates a specific color macroblock.

First, the result of the static / dynamic determination for each macroblock is checked (S1003). In the two-dimensional array M [i] [j], the static / movement determination result is stored by the macroblock static / movement determination process (S701). If TRUE, a dynamic macroblock, FALS
E indicates a still macroblock. If the value of the two-dimensional array M [i] [j] is FALSE, that is, if it is a still macroblock, the macroblock does nothing and goes to the next macroblock. If the two-dimensional array M [i] [j]
Is TRUE, that is, if it is a moving macroblock, it is checked whether the pixel value of the macroblock is within the range of the color space specified by the specific color specifying unit.
(S1005) If not, it is determined that there is no specific color in the moving macroblock, that is, the newly rewritten macroblock, and rewritten to FALSE (S1006).

The specific color specifying section 819 specifies the range of the color space of the color to be detected. If the color is known in advance, it can be specified directly in the YCbCr color space.
Here, D1 to D6 are threshold values.

[0093]

[Formula 1] D1 <Y <D2 and D3 <Cb <D4 and D5 <Cr <D6 As another method of specifying a specific color, the user is asked to specify a color range on the RGB color space, and the range is specified. May be converted to YCbCr using Equation 2 and determined.

[0094]

[Equation 2] EY = 0.299R + 0.587G + 0.114BY = 219 (EY) / 256 + 16 Cb = 126 (B-EY) / 256 + 128 Cr = 160 (R-EY) / 256 + 128 As a method of designating a specific color of, there is also a method of converting the YCbCr color space being encoded into an RGB space by using Expression 3 and detecting it in the form of Expression 4.

[0095]

[Equation 3] R = (298Y + 408Cr-56993) / 256 G = (298Y-100Cb-207Cr + 34495) / 256 B = (298Y + 518Cb-72085) / 256

[0096]

[Formula 4] D1 <R <D2 and D3 <G <D4 and D5 <B <D6 In this way, if a range that can be distinguished in an appropriate color space can be specified, any method can be used. Can be done

FIG. 26 shows the specific processing contents of the specific color object inclusion processing. Here, n is a counter indicating the number of specific color objects, S1 to S4 are parameters indicating a search range of a rectangle including the specific color object, and S1 and S2 are a starting point of a vertical address. This is the end point, and S3 and S4 are the start and end points of the address in the horizontal direction.

In the flowchart of FIG. 26, initialization is performed so that the entire frame is designated as a search range (1101). Next, a function Rectangular for searching for the smallest rectangle including the specific color object in the specified search range is called (S1102).

The flowcharts of FIGS. 27 and 28 show the functions Rectangular
The content of the processing of (S1102) is shown. Function Rectangula
r is the search range S1 to S4 and n indicating the number of specific color objects
And the two-dimensional array M in which the results of the static / dynamic determination of each macroblock are stored, and the one-dimensional arrays B1 to B4 in which the rectangular addresses of the search results are stored and n indicating the number of specific color objects are output. . Here, the one-dimensional array HV is a working array for creating a histogram of the number of moving macroblocks in the vertical direction, and the one-dimensional array HH is used for creating a histogram of the number of moving macroblocks in the horizontal direction. It is a working array. The variable VFLAG is TRUE when the value of the horizontal histogram is not 0, and FALSE when the value is 0.
And the variable HFLAG is
TRUE if the vertical histogram value is not 0
This is a flag that changes to FALSE when the value is 0.

First, the range of S1 to S2, which is the search range of the work array HV for creating a histogram of the number of moving macroblocks in the vertical direction, is initialized to a value of 0. (S1201) Next
In the double loop of LOOP1 and LOOP2, a histogram of the number of vertical moving macroblocks in the search range is created. That is, the static / dynamic determination result M [i] for each macroblock
The value of [j] is compared (S1204). If the value is TRUE, that is, if the value is a moving macroblock, HV [i] is incremented by one (S1205).
), If FALSE, the action is to do nothing. Next, from the vertical histogram,
Search for contiguous parts that are not. First, set the flag VFLAG
Set to FALSE. (S1208) Next, from the search range S1
In order of S2, histogram HV is not 0 and VFLAG is FALS
Check if it is E. (S1210) The condition that satisfies this condition is the start point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the starting point in the vertical direction of the rectangle being searched, the address i is stored in the one-dimensional array B1 [n], and VFLAG is set to TRUE. (S1211)
Next, it is checked whether the histogram HV is 0 or VFLAG is TRUE at the end point of the search range. (S1212)
This condition is applied to the end point of a continuous portion where the histogram is not zero. Therefore, if the histogram HV is 0, the address i-1 is stored in the one-dimensional array B2 [n] because it is a candidate for the vertical end point of the rectangle being searched (S1214). Stores the address i in the one-dimensional array B2 [n]. (S1215) And
Set VFLAG to FALSE again. (S1211) FIG. 28 is a continuation of the flowchart of FIG. This time, the range of S3 to S4, which is the search range of the work array HV for creating a histogram of the number of moving macroblocks in the horizontal direction, is initialized to a value of 0 (S1217). In the next double loop of LOOP4 and LOOP5, a histogram of the number of moving macroblocks in the horizontal direction in the search range is created. That is, the values of the static / movement determination result M [i] [j] for each macroblock are compared (S1204),
If the value is TRUE, that is, if it is a moving macroblock, HH [i]
Is incremented by 1 (S1205), and in the case of FALSE, an operation is performed in which nothing is performed. Next, a non-zero continuous portion is searched from the horizontal histogram. First, the flag HFLAG is set to FALSE. (S1224) Next, in the order of the search ranges S3 to S4, it is checked whether the histogram HH is not 0 and HFLAG is FALSE (S1226).
). This condition is applied to the starting point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the horizontal start point of the rectangle being searched, the address j is stored in the one-dimensional array B3 [n], and HFLAG is set to TRUE (S1227). Next, it is checked whether the histogram HH is 0 or HFLAG is TRUE at the end point of the search range (S1228). This condition is applied to the end point of a continuous portion where the histogram is not zero. Therefore, since it is a candidate for the horizontal end point of the rectangle being searched, if the histogram HH is 0, the address j-1 is stored in the one-dimensional array B4 [n] (S1230), otherwise. Stores the address j in the one-dimensional array B4 [n] (S1231). Then, HFLAG is set to FALSE again (S1232).

Here, the search by one type of histogram in the vertical direction and the horizontal direction is completed, but the search result B1 [
It is checked whether n] to B4 [n] matches the search range S1 to S4 (S1233). If they match, there is no more search range, so it can be determined that the smallest rectangle has been found. (S1234). And n representing the number of specific color objects
Is increased by 1 (S1235), and the process proceeds to the search for the next specific color object. The search results B1 [n] to B4 [n] are in the search range S1 to S4
If they do not match, since there are still a plurality of specific color objects in the range of the search result, the search results B1 [n] to B4 [
n] to S1 to S4 (S1237), and again the function Rect
Call angular (S1237).

FIG. 29 shows an example of the judgment result of the specific color object judging section 818. Here, two specific color objects were determined.
In this example, as shown in the figure, the origin of the macro book address is at the upper left of the frame, as shown in the figure, such as B1 [0] to B4 [0] and B1 [1] to B4 [1]. The specific color object is determined in an appropriate arrangement.

The specific color object synthesizing display unit 817 creates an image obtained by synthesizing the reproduction color image signal and the area of the specific color object determined by the specific color object determination unit 818. FIG. 30 is a specific flowchart. Here, n indicates the number of specific color objects in the frame obtained by the specific color object determination unit 818. The processing flow is as follows. For each specific color object, B1 [i] to B4 [i] represent the four corner macroblocks of the specific color object, so draw a rectangular white line surrounding it and play it back. The image is synthesized with the image (S1202).

FIG. 31 is an example of an image created by the specific color object synthesis display section.

As described above, in the embodiment, an example in which processing is performed in units of macroblocks has been described. However, it goes without saying that the same method can be applied to processing in a smaller unit or processing in a larger unit. Also, as an example of a moving picture coding method, a method using DCT has been described. However, a similar method can be used in other transform methods such as Wavelet transform.

[Embodiment 2] (corresponding to claim 5) FIGS. 2 and 3 are block diagrams showing the configuration of an embodiment of the system of the present invention.

FIG. 2 is a system block diagram. Basically, the moving picture coded data and the still picture coded data of the coding block 125 of the system block diagram of FIG. Multiplexed transmission means
The system can be monitored remotely by transmitting via 123.

The system block diagram of FIG. 3 basically uses the decoding block 126 of the system block diagram of FIG. 1, and inputs the data transmitted from the encoding block from the transmission means 123 and demultiplexes the data. By returning the original encoded video data, the encoded still image data, and the encoded control data to the decoding block via the means 124, a system similar to that of FIG. 1 can be remotely monitored. ing.

[Embodiment 3] (corresponding to claim 6) FIG. 32 is a flowchart of an embodiment of a fire monitoring system according to claim 6. In fire monitoring using the moving image data S1501, moving object detection S1502 is performed on the input moving image data S1501. If there is no object in the detection by the detected object judgment S1503, the operation moves to the moving object detection S1502. If there is an object in the detection by the detected object judgment S1503, the specific color detection S1504 is performed on the object. The color of the object is determined in S1505. If it does not correspond to a preset fire or smoke, moving object detection S1502
Return to and perform processing. If it corresponds to fire or smoke, the proportion of the color in the moving object is determined, and the fire status is determined in fire status determination S1506. In fire condition judgment S1506, the size of fire and smoke in the screen is judged. If the fire is small, an alarm / contact S1507 is issued. Performs moving image data recording S1508. Performs the composite video display S1509 of the specific color detection image and the video. If the fire is large, obtain a still image S1510
And send the still image data created in S1510 at the same time as alarm / contact S1507. At the same time, it is a system that transmits still images to related departments S1511. Naturally, it is possible to provide a fire monitoring system capable of rearranging processes such as alarming and communication according to the use environment and the like.

FIG. 33 shows an embodiment of the fire monitoring system according to claim 6.

[Embodiment 4] (corresponding to claim 7) FIG. 34 is a flowchart of an embodiment of a costume changing system according to claim 7. In monitoring using the moving image data S1601, moving object detection S1 is added to the input moving image data S1601.
Perform 602. If there is no object detected by the detected object determination S1603, the operation returns to the moving object detection S1502. If there is an object in the detection by the detected object determination S1603, the object position determination S1604 is performed using the object as a target person. In object position determination S1604, position determination is performed so that the detected person comes near the center position in the moving image frame. From the position determined in object position determination S1604, each indication unit S1605 to S1608
To adjust the position of the person. After the position adjustment, still image acquisition S1609 and moving image data recording S1610 are performed. The size of the person is determined from the diagonal line of the rectangle obtained by the moving object detection, the size of the costume and the like is adjusted by the size adjustment S1613 of the image data of the costume and the like, and the combining process is performed in the image data combining process S1611. A composite moving image display S1612 is performed to check whether the costumes and the like match. Size adjustment of image data such as costumes S1
By repeating the processing after 613, it is possible to provide a costume changing system that can easily match various costumes and the like.

FIG. 35 shows an embodiment of the outfit changing system according to claim 7, in the state a, the target person is off the center of the screen. An instruction is issued to the target person, and the target person is moved to the center of the screen as shown by b. In this state, adjust the clothes as shown in c. The purpose of bringing a person to the center of the screen is to prevent the display from appearing on the screen depending on the costume to be matched.

[Embodiment 5] (corresponding to claim 8) FIG. 36 is a flowchart of an implementation of a nursing care guide system according to an embodiment of claim 8. The purpose of the nursing care building guidance system is to automatically move in the hospital with a wheelchair.
A consultation department input S1701 using a consultation ticket or the like is performed. In a hospital, a wheelchair is automatically driven by a moving image monitoring device attached to a wheelchair along with a color line for each department displayed on the floor.

The specific color detection S170 is performed using the moving image data S1702 input to the moving image monitoring device attached to the wheelchair.
According to 3, the color line to each department in the hospital is detected, and the wheelchair is caused to travel S1706 along the designated color line. When the color line to each department in the hospital is lost, the direction to the wheelchair is changed on the spot and the designated color line is searched, thereby guiding to the target department.

It is also assumed that there are objects and obstacles crossing the passage while moving the wheelchair. Therefore, the moving object detection S1707 is performed even while the wheelchair is running, and an object or obstacle crossing ahead is detected from the object detection determination S1708. When an object is detected, the wheelchair is stopped (S1709), and an object detection determination (S1710) is performed to check whether the object passes. If the object does not pass even after the lapse of the predetermined time, a warning is issued in S1711. If the object does not move even if the number of times exceeds the set number, the direction of the wheelchair is changed in S1712, and the vehicle advances while avoiding the object.

FIG. 37 shows an embodiment of the nursing care building guidance system according to the eighth embodiment.

[Embodiment 6] (corresponding to claim 9) FIG. 38 is a flowchart of an intersection and road monitoring system according to an embodiment of claim 9 for monitoring an intersection. In the intersection monitoring, a moving image monitoring system is installed using a traffic light installed at the intersection. Each traffic light is configured to monitor the traffic light facing each other and the direction in which the traffic light is located.

In the description of the intersection monitoring, the traffic light 1
Will be described. The moving object detection S1802 is performed using the moving image data S1801 of the moving image monitoring system installed in the traffic light 1. Moving object detection S1802 is also performed when moving object data information is transmitted from another traffic signal.

When an object is detected in the detected object determination S1803, the moving direction of the object is monitored, the moving direction is specified, and the moving object data information is transmitted to the moving direction signal (S1820).

Further, the color of the moving object is detected by a specific color detection S180.
Then, the moving object detection color judgment S1805 is performed. From the result, the size of the moving object is confirmed from the size of the macroblock in which the specific color is detected (S1806). If there is a moving object, moving image data is recorded in moving image data recording S1807.

At the same time as the detection of the specific color of the moving object S1804, the signal colors of the traffic signals facing each other are determined at the traffic light detection color judgment S1809.
Do with. It is determined whether the moving object is observing the signal based on the determined color of the traffic light. If the traffic light is blue, the process returns to S1812 for moving object detection S1802.

If the traffic light is yellow, a warning / caution is given by voice in S1811 (S1814), and data is transmitted to the traffic light in the traveling direction to perform monitoring. When the signal color is red, S1
In 810, it is determined that the signal is ignored.
1814, and at the same time acquire a still image of the license plate S
1815 and recording of moving image data. The moving image is combined with the moving object detection image and displayed S1808, and the still image is transmitted to the relevant department (S1818).

With this system, it is possible to prevent traffic signals and accidents at intersections from being ignored unintentionally, to control traffic signals, etc., and to monitor and record data on accidents at intersections by monitoring moving objects. it can.

FIG. 39 shows an embodiment of an intersection monitoring system according to claim 9.

The moving image input unit provided at each traffic light is installed so as to monitor the inside of the intersection. Each traffic light shares data of cars and people entering the intersection, and the signal color of the traffic light is determined only by the image. can do.

[Embodiment 7] (Corresponding to claim 10) A monitoring system for a road with poor visibility is described with reference to FIG.

In general, a curved mirror is often used on a road with poor visibility, but the state of the opposite side may not be accurately monitored. By using this video monitoring system, an object approaching from the opposite direction can be notified instead of a curved mirror.

Moving object detection S1902 is performed from the moving image data S1901 of the moving image monitoring system installed at the corner with poor visibility, and as a result of the detected object judgment S1903, if there is a moving object, a lamp or the like is displayed on the opposite side. The status of a place with poor visibility can be surely known by the notification.

Further, in this system, by installing a moving image input device at a corner depending on the width of the road to be monitored, both roads having poor visibility can be monitored by one monitoring system.

FIG. 41 shows an embodiment of an intersection monitoring system according to claim 10.

The left and right roads are monitored by a moving image input unit installed at a corner where the other side cannot be seen. On each of the unseen roads, if there is an object approaching on the opposite side, a signal is given by a traffic light to notify the vehicle or the person. Can prevent accidents at the meeting.

[Embodiment 8] (corresponding to claim 11) FIG. 42 illustrates an embodiment of the system of the present invention for monitoring the speed at a place where the speed is easy to get out or on a downhill. Moving object detection S2002 is performed on the moving image data S2001 input to the moving image monitoring system, and if no moving object is detected in the detected object determination S2003, the process returns to the moving object detection. If a moving object is detected, the speed is calculated S2004 from the change in the size of the moving object, and a speed determination S2005 is performed. As a result of the speed judgment, if the speed is within the legal speed, moving image data recording S2006 is performed, and a composite image display S2007 with the moving object detection result is performed. When the speed exceeds the legal speed, an alarm / caution S2008 is performed, a still image acquisition S2009 is performed so that the license plate can be recognized, and a still image transmission S2010 to the related department is performed.

FIG. 43 shows an embodiment of the intersection monitoring system according to the eleventh aspect. Monitoring is performed by obtaining the speed from the change in the size of the approaching vehicle by the moving image input unit installed on the road. The speed is obtained from the size of the moving image of the vehicle traveling from the front. In FIG. 43, the vehicle moving from a to b looks like a picture below it on the screen. At this time, the moving object is detected, and the macro block is formed into a square. This system can determine the speed from the change in the size of this macro block and the time.In the past, the speed measuring device and the photographing camera for control were separated, but in this system the speed is determined. The measuring device and the photographing device can be integrated, and the size of the system can be reduced.

[Embodiment 9] (Corresponding to claim 12) FIG. 44 is a flow chart illustrating an example of a sports monitoring system according to claim 12, taking baseball batting as an example. FIG. 45 shows an embodiment of a sports monitoring system. Stickman has head, shoulders, elbows, hips, knees and butt,
Apply the determined color to the root. In baseball batting, the moving image input unit of the moving image monitoring system is set at a position where a batter enters through the home base.

The moving image data S2101 is input. Moving object detection S2102 is performed on the input moving image data S2101, and detection object determination S2103 is performed. If there is no moving object, the process returns to the moving object detection S2103. If there is a moving object, perform specific color detection S2104 on the moving image data,
If the specific color is not determined in the detection color determination S2105, the process returns to the moving object detection S2102 and monitoring is continued. If a specific color is detected, moving image data recording S2106 is performed,
The display of the moving image data is performed. At the same time, the locus based on the color of the batter's body or the bat is processed S2109, and the synthesized moving image display S2108 can be displayed.

Further, the display S2111 of the sample moving image data can be performed using the moving image data serving as the sample. The locus of each color is also processed S2112 from the moving image data, and the display S2110 of the synthesized moving image can be performed. The composite moving image display is performed by combining the sample moving image data, the recorded moving image data of the batter, and the trajectory.
By doing, the batter's batting form,
You can monitor your body movements.

Of course, a similar system can be used for other sports such as golf and tennis.

[Embodiment 10] (Corresponding to Claims 13 and 14) As shown in FIG. 46, a moving image 90 near a door on each floor of an elevator.
4 is input to the moving image monitoring apparatus 930. The moving object detection unit 128 detects the movement of the object, and adds the output movement detection 931 to the information filter unit 908B. Movement detection 93
1 may be output multiple times depending on the content of the moving image.

The information filter unit 908B obtains the approximate position of the object based on the input movement detection. When the movement position stops near the elevator door, the call signal 932 is transmitted to the elevator control unit 906 via the host communication unit 906. Notify the device. Further, when there are a plurality of objects stopped near the elevator door, the information filter unit 908B notifies the elevator control device 935 of the number as the gathered number of persons 933 via the higher-level communication means 906. The elevator control device 935 controls the elevator car device based on the call signal 932 and the number of people 933 gathered.

[Embodiment 11] (corresponding to claim 15) FIG. 47 is a flowchart showing an embodiment of the passer-by monitoring system according to claim 15. As an example of a passer monitoring system, a player passing system at a marathon event will be described. At the present, it is very difficult to manage the progress of the marathon, because many players run.

The moving image monitoring system uses a moving object detection and a specific color detection so that a passing time of a player and a passing person can be grasped. Athletes run with bib numbers of specific colors for each athlete. The moving image data S2201 of the players passing through the moving image monitoring system installed at each check point is input. Performs moving object detection S2202 on the input video data and determines the detected object
S2203. The monitoring that repeatedly performs the moving object detection S2202 is continued until there is a detected object. If there is a detected object, a specific color detection S2204 is performed for the object, and a detected color determination S2205 is performed. If the specific color is not detected, the monitoring with the moving object detection S2202 is continued. If a specific color is detected, a notification is sent to the database together with the moving image data at the time of passing (S2206).

The moving image data is recorded in moving image data S2208 and displayed as moving image data S2209. On the other hand, in the notified database, the player is identified by the specific color, and the time embedded in the moving image data of the player is recorded as the passage time of the check point. By similarly notifying the database from other checkpoints, it is possible to grasp whether each player has passed the checkpoint and which checkpoint has passed at what time.

FIG. 48 shows an embodiment of a passing person monitoring system according to claim 15.

[Embodiment 12] (corresponding to claim 16) FIG. 49 is a flowchart showing an embodiment of an entry / exit area monitoring system according to claim 16. A batch is attached at the access control office for the purpose of monitoring persons entering and leaving the area (S2301). The other party is recognized by this batch. The moving image data S2302 of the entrance is input, and the moving object detection S2303 is performed. The detection object determination S2304 is repeatedly performed on the input image until the detection object is found in the detection object determination S2304. If there is a detected object, a specific color detection S2305 is performed, and a detection color determination S2
If it is determined in step 306 that a specific color batch has not been added, the process returns to the moving image data S2303.

When there is a person who attaches a batch of a specific color, a notification is sent to the database together with the moving image data at the time of passing, a moving image data recording S2308 is performed, and a displaying of the moving image data is performed S2309. The database that has received the notification analyzes the moving image data and the ratio of the time embedded in the moving image and the ratio of the color in the moving image data, and performs S2310 embedding in the database.

The same processing as described above is performed when leaving the area, and it is determined whether the clothing at the time of entering the area is the same based on the ratio of colors in the moving image in the database (S2311). If it is determined that the color ratios are different and the clothes are different, a still image is acquired, and reconfirmation is performed at the entry / exit management place (S2312).

FIG. 50 shows an embodiment of an access control / monitoring system according to the present invention.

[Thirteenth Embodiment] (Corresponding to Claim 17) FIG. 51 is a flowchart of a shelf monitoring system according to claim 17.

The shelf monitoring system is a system for monitoring shelves in a store, a warehouse, or the like, and performs a monitoring process for a target portion by setting a shelf portion of a display image in advance.

First, only the shelf portion of the display image is set as a monitoring target S2401. Moving object detection S2403 is performed on the designated portion of the moving image data S2402, and detection object determination S2404 is performed. The monitoring process is performed until a detected object is found, and if there is a detected object, specific color detection S2405 is performed. Step S2406 is performed until a specific color is detected, and after detecting the specific color, moving image data recording S2408 and display of moving image data S2409 are performed. At the same time, it notifies the database. The notified database records the number and time of the objects taken out of the shelf from the time data in the moving image.

FIG. 52 shows an embodiment of a shelf monitoring system according to claim 17.

[Embodiment 14] (corresponding to claim 18) FIG. 53 is a configuration diagram showing an embodiment of a water quality monitoring system according to claim 18. In FIG. 53, a moving image input unit 902
Is a video encoding unit 105 via a video input interface.
Connected to The output of the moving image encoding unit 105 is composed of mode information, motion vector information, and DCT coefficient information, and is input to the moving image specific color object detection unit 120.

The moving image specific color object detecting section 120 receives the specific color specifying information 90 in addition to the input from the moving image encoding section 105.
3 is entered. The output from the moving image specific color object detection unit 120 is a color detection output 910 and a detection range output interface 91.
1 is connected to the information filter unit 908. The result output of the information filter unit 908 is connected to the recording unit 907 and the higher-level communication unit 906. The upper communication means 906 is a communication line
It is connected to a higher-level monitoring system via 905.

As shown in FIG. 39, the moving image specific color object detecting section 120 includes a macroblock still / moving determining section 811 and a specific color object determining section 818. The input mode information, motion vector information, and DCT coefficient information are It is added to the block static motion judging unit 811. The output of the static / movement determination result and the input specific color designation information are applied to the specific color object determination unit 818.
The specific color object determination unit 818 outputs a color detection output 910 and a detection range output 911. Next, the operation will be described. The moving image input unit 902 outputs a sequence of digitized still images of, for example, about 30 screens per second for the monitoring target. The moving image encoding unit 105 receiving the continuous still images
Divides a still image into a group of pixels called macroblocks, detects the motion of an object in an image based on the similarity of macroblocks at different positions in the preceding and following images, and outputs the motion vector information. Further, DCT coefficient information is output by performing DCT (discrete cosine transform) processing on the macroblock. The macroblock static / moving determining unit 811 in the specific color object detecting unit 120 determines whether there is a moving object in the macroblock based on the input mode information, motion vector information, and DCT coefficient information, and determines a static / moving determination result. Is output.

The specific color object determination unit 818 in the specific color object detection unit 120 determines whether or not the macroblock has a color corresponding to the specific color designation information 903 based on the result of static / movement determination and the pixel value of the macroblock. I do. The specific color designation information 903 can designate from a single color to a color having a width, and is used properly according to a required condition. Specific color designation information, for example, when the change in brightness of the input image 904 is large, or when the characteristics of the moving image input unit 902 itself changes by switching cameras, etc.
It is possible to respond by widening the detection width of 903. The determination result is output from the specific color object detection unit 120 as a color detection output 910. In addition, the macroblocks having colors are connected to obtain the range of the object, and the detection range is output.
Output to 911 interface. The information filter unit 908 detects a color that is not applicable based on requirements such as whether the detection is continuous in time, has a spatially effective size, or matches the shape. Is removed. The required conditions differ depending on the application system, and can be changed depending on the difference in the output destination, such as the output to the recording unit 907 and the output to the upper communication unit 906.

The recording unit 907 records the color detection output and the detection range output from the information filter unit 908 together with time information on a recording medium such as a memory or a disk device.
Since the recording capacity is limited, new information is preferentially recorded, and the oldest information is deleted. The recorded color detection information can be read out via the higher-level communication unit 906 at the request of the higher-level monitoring device. The upper communication means 906 is connected to a higher monitoring device via a communication medium such as LAN or WAN or a wireless medium such as PHS or mobile phone,
The color detection output and the detection range output from the information filter unit are immediately notified to a higher-level monitoring device. Conversely, when there is a recording / reading request from a higher-level monitoring system, the storage unit 90
It also has a function to transfer the recorded information to the higher-level monitoring system in cooperation with 7.

FIG. 54 shows an example of a configuration in which coded moving image information is transmitted to a remote place by a communication means, and a specific color is detected from information extracted at the time of decoding.

The operation when the above basic operation is applied to water quality monitoring will be described. As shown in FIG. 54, a wide-range moving image 904 of the lake surface or the sea surface is captured by the moving image input device. The water surface is gradually moving due to the ebb and flow of the tide, the blowing wind, the flow of the tide, and the flow of the river. Even when there is water pollution such as blue-green algae or red tide, the position is gradually moving, and the moving image encoding unit 105 detects the movement. For a macroblock that has moved, a color corresponding to water pollution such as blue-green algae or red tide is set as specific color designation information 903, and a color detection operation is performed. The presence or absence of the output color and the detection range are recorded, and the filter unit 908 determines the presence or absence of contamination on the condition that the output value of the detection range exceeds a predetermined threshold. The presence / absence of contamination is notified to the higher-level monitoring device 920 via the transmission path 905 as an alarm via the higher-level communication means 906.

[Embodiment 15] (Corresponding to claim 19) An embodiment in which the present invention is applied to astronomical moving image monitoring will be described. As shown in FIG. 55, a wide-range celestial moving image 904 is input using a high-sensitivity camera.

The camera outputs the position on the earth where the camera is installed and the range information 912 on the celestial sphere where the camera is being taken. A color corresponding to a supernova or a comet is set as the specific color designation information 903, a color detection operation is performed, and a star is detected based on the presence or absence of a color to be output. In the information filter unit 908, based on the database 913 recording known stars and the output information 912 from the camera, it is compared whether the color detection result is a new discovery. If it does not exist in the database, a new discovery is output to the higher-level monitoring device 920 via the higher-level communication means 906.

[Embodiment 16] (Corresponding to claim 20) Next, an embodiment in which the present invention is applied to platform moving image monitoring at a railway station will be described. As shown in FIG. 56, a partial moving image or a whole moving image 904 of the station platform is added as an input image. As the specific color designation information 903, the color of the white line of the home is designated and the color detection operation is performed. For example, if a person blocks the white line, the person is detected by the movement detection, and the white color specified by the macroblock in the outline of the person is detected. Information filter section for specific color range to be output
At 908, a change is detected, and if there is a change, it is determined that a person has approached the track from the platform, and
Notify one of the people approaching the track.

[Embodiment 17] (Corresponding to claim 21) An embodiment in which the present invention is applied to aircraft moving image monitoring at an airport will be described. As shown in FIG. 57, a moving image 904 from a camera installed beside the runway and taxiway as an input image
Add. As the specific color designation information 903, a plurality of colors of the airplane of each airline designated in advance are designated, and a plurality of color detection operations are performed. When the aircraft passes in front of the camera, an object is detected by movement detection. If the color of the object matches the designated color, the detection result 91 from the specific color object detection unit 120 is output.
0 and the color range 911 are output. Regarding the range 911 of the specific color to be output, the information filter unit 908 detects incorrect detection depending on the size and the moving direction of the object, for example, a premises vehicle, a small animal, and an airplane on a close runway included in a moving image. Then, the result of the removal, that is, the passage of the aircraft and the color type thereof, are notified to the control room 920 of the airport by the host communication means 906.

[Embodiment 18] (Embodiment 22) An embodiment in which the present invention is applied to moving image control of a production line in a steel mill will be described. As shown in FIG. 58, a moving image 904 of hot iron on the production line is added as an input image. As the specific color designation information 903, a color at a required temperature is designated in advance, and a color detection operation is performed. When the temperature of the iron changes, its color changes. If the color of the object matches the color of the specified required temperature, the detection result 91 from the specific color object detection unit 120 is output.
0 and the color range 911 are output. The information filter unit 908 removes local detection of the output specific color range, and notifies the manufacturing control device 922 of the color range by the higher-level communication unit. In the production control device 922 shown in FIG. 59, for example, a temperature control instruction 923 is issued so that the iron temperature is stabilized at the required temperature.

[Embodiment 19] (Corresponding to claim 23) An embodiment in which the present invention is applied to a landslide moving image monitoring apparatus will be described. As shown in FIG. 60, a moving image 904 of earth and sand, such as a landslide danger area or a river bank, is added as an input image. As the specific color specification information 903, a color detection operation is performed by specifying the color of earth and sand in advance. Since the color of earth and sand varies depending on the area, specify the actual color when installing the camera. When soil having a specific color moves or is newly exposed, the specific color object detection unit 120 outputs a detection result 910 and a color range 911. When the detection range of the specific color range to be output exceeds the threshold value specified in advance, the information communication unit 907 sends an alarm 905 to the upper monitoring system 920.

[Embodiment 20] (Corresponding to Claims 24 and 25) FIG. 61 is a flowchart showing an embodiment of an on-vehicle road monitoring system according to claim 24. The on-vehicle road surveillance system is mounted on a car and linked with a car navigation system using GPS to correct position information without relying on D-GPS, etc. Using a vehicle speed sensor, the state of a traffic signal ahead when the vehicle is stopped, etc. It is a system that helps to ensure safe driving.

The operation flowchart will be described.
The moving image monitoring system is operated based on information from the vehicle speed sensor (S2501). The specific color detection S2503 is performed using the moving image data S2502, and the signal state of the signal ahead is checked (S250
2504). The position of the traffic signal ahead is notified to the car navigation, and the position is corrected using the traffic signal (2502). Voice instructions are given for each signal color based on the determined traffic light color S2505 (S2506 to S2510). If the traffic light is blue, the process in S2505 is instructed to proceed by voice (S2506). If the traffic light is yellow, S2507 will be alerted by voice (S250
8). If the traffic light is red, a stop guidance is given by voice to S2509 (S2510).

Further, a specific color is detected in response to a request from the car navigation (S2513), and the specific color on the map is confirmed.
(S2514) to correct the position. By recording a moving image during traveling, it is possible to make a driving record like an aircraft, and if there is no accident, it is possible to record a video cyclically.

[0168]

As described above, according to the moving picture monitoring system of the present invention, detection of a moving object in a moving picture and detection by color can be performed relatively easily, and if necessary, Since a fine still image can be immediately acquired and used, it can be easily applied to a wide range of uses.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of a moving image monitoring system according to a first embodiment of the present invention.

FIG. 2 is a block diagram showing a configuration on an encoding side of a video monitoring system according to a second embodiment of the present invention.

FIG. 3 is a block diagram showing a configuration on a decoding side of a video monitoring system according to a second embodiment of the present invention.

FIG. 4 is a block diagram illustrating a configuration of an encoding unit according to the present invention.

FIG. 5 is a block diagram illustrating a configuration of a moving image encoding unit according to the present invention.

FIG. 6 is a block diagram illustrating a configuration of a still image encoding unit according to the present invention.

FIG. 7 is a block diagram illustrating a configuration of a decoding unit according to the present invention.

FIG. 8 is a block diagram illustrating a configuration of a video decoding unit according to the present invention.

FIG. 9 is a block diagram illustrating a configuration of a still image decoding unit according to the present invention.

FIG. 10 is a block diagram illustrating a configuration of moving image moving object detection according to the present invention.

FIG. 11 is a flowchart showing the operation of the moving object detection unit of the present invention.

FIG. 12 is a flowchart showing the operation of the macroblock still / moving determination section of the present invention.

FIG. 13 is a flowchart showing the operation of the moving object determination unit of the present invention.

FIG. 14 is a flowchart showing an operation of noise processing of the moving object determination unit of the present invention.

FIG. 15 is a flowchart showing an operation of a moving object inclusion process of the moving object determination unit of the present invention.

FIG. 16 is a flowchart showing the operation of the moving object inclusion processing of the moving object determination unit of the present invention.

FIG. 17 is a flowchart showing an operation of a moving object inclusion process of the moving object determination unit of the present invention.

FIG. 18 is a flowchart illustrating an example of a determination result of a moving object determination unit according to the present invention.

FIG. 19 is a flowchart showing the operation of the moving object synthesis display unit of the present invention.

FIG. 20 is a diagram illustrating an example of a display result of the moving object determination unit according to the present invention.

FIG. 21 is a block diagram illustrating a configuration of specific color object detection according to the present invention.

FIG. 22 is a flowchart showing the operation of the specific color object detection unit of the present invention.

FIG. 23 is a flowchart showing the operation of the macroblock still / moving determination section of the present invention.

FIG. 24 is a flowchart showing the operation of the specific color object determination unit of the present invention.

FIG. 25 is a flowchart showing an operation of noise processing of the specific color object determination unit of the present invention.

FIG. 26 is a flowchart showing an operation of a moving object inclusion process of the specific color object determining unit of the present invention.

FIG. 27 is a flowchart showing an operation of a moving object inclusion process of the specific color object determination unit of the present invention.

FIG. 28 is a flowchart showing an operation of a moving object inclusion process of the specific color object determining unit of the present invention.

FIG. 29 is a flowchart illustrating an example of a determination result of a specific color object determination unit according to the present invention.

FIG. 30 is a flowchart showing the operation of the specific color object synthesis display unit of the present invention.

FIG. 31 is a diagram illustrating an example of a display result of the specific color object determination unit of the present invention.

FIG. 32 is a flowchart illustrating the operation of the fire monitoring system according to the embodiment of FIG. 3 of the present invention.

FIG. 33 is a diagram illustrating an example of a fire monitoring system according to a third embodiment of the present invention.

FIG. 34 is a flowchart showing the operation of the costume changing system according to the fourth embodiment of the present invention.

FIG. 35 is a diagram illustrating an example of a costume changing system according to a fourth embodiment of the present invention.

FIG. 36 is a flowchart showing the operation of the nursing care building guidance system according to the fifth embodiment of the present invention.

FIG. 37 is a view showing an example of a nursing care building guidance system according to a fifth embodiment of the present invention.

FIG. 38 is a flowchart showing the operation of the intersection monitoring system according to the sixth embodiment of the present invention.

FIG. 39 is a diagram illustrating an example of an intersection monitoring system according to a sixth embodiment of the present invention.

FIG. 40 is a flowchart showing the operation of the road monitoring system with poor visibility according to the seventh embodiment of the present invention.

FIG. 41 is a diagram illustrating an example of a road monitoring system with poor visibility according to a seventh embodiment of the present invention.

FIG. 42 is a flowchart showing the operation of the speed monitoring system according to the eighth embodiment of the present invention.

FIG. 43 is a diagram showing an example of a speed monitoring system according to an eighth embodiment of the present invention.

FIG. 44 is a flowchart showing an operation of the sports monitoring system according to the ninth embodiment of the present invention.

FIG. 45 is a diagram illustrating an example of a sports monitoring system according to a ninth embodiment of the present invention.

FIG. 46 is a flowchart showing the operation of the elevator self-answer control system according to the tenth embodiment of the present invention.

FIG. 47 is a flowchart showing the operation of the passer-by monitoring system according to the eleventh embodiment of the present invention.

FIG. 48 is a diagram illustrating an example of a passer monitoring system according to an eleventh embodiment of the present invention.

FIG. 49 is a flowchart showing the operation of the access monitoring system according to the twelfth embodiment of the present invention.

FIG. 50 is a diagram illustrating an example of an access monitoring system according to a twelfth embodiment of the present invention.

FIG. 51 is a flowchart showing an operation of the shelf monitoring system according to the thirteenth embodiment of the present invention.

FIG. 52 is a diagram illustrating an example of a shelf monitoring system according to a thirteenth embodiment of the present invention.

FIG. 53 is a block diagram showing a configuration of a water pollution degree monitoring system according to a fourteenth embodiment of the present invention.

FIG. 54 is a configuration diagram of a water pollution degree monitoring system according to a fourteenth embodiment of the present invention.

FIG. 55 is a block diagram showing an overall image of the astronomical moving image monitoring system according to the fifteenth embodiment of the present invention.

FIG. 56 is a block diagram showing another configuration of the station platform monitoring system according to the sixteenth embodiment of the present invention.

FIG. 57 is a block diagram showing an overall image of an airplane runway passage monitoring system according to a seventeenth embodiment of the present invention.

FIG. 58 is a block diagram showing a configuration of a high-temperature iron monitoring system according to an eighteenth embodiment of the present invention.

FIG. 59 is a block diagram showing a configuration of a high-temperature iron monitoring system according to an eighteenth embodiment of the present invention.

FIG. 60 is a block diagram showing a dangerous area monitoring system according to a nineteenth embodiment of the present invention.

FIG. 61 is a block diagram showing a dangerous area monitoring system according to a nineteenth embodiment of the present invention.

[Explanation of symbols]

101 image input means 102 time adding means 103 input image recording means 104
Image adjustment means, 105 ... moving picture coding means,
109: image combining means, 110: moving image display means, 111: specific color adjusting means, 112: specific color detecting means, 113: moving object adjusting means, 1
14 moving object detecting means 115 still image coding means 116 coding control means 11
7: Still image coding synchronization means, 118: Still image decoding means, 119: Still image display means, 1
20: recording means, 121: detection result output means, 122: multiplexing means, 123: transmission means, 124: demultiplexing means, 125: coding block, 126 ..Decoding blocks, 12
8: Moving object detection block, 129: Specific color detection block, 201: A / D unit, 202:
Switch, 203: Format conversion unit, 20
4 image size conversion unit 205 moving image coding unit 206 input image buffer 207
..Switch, 208... Still image coding unit, 2
09: encoding control unit, 301: blocking unit,
302: subtraction machine, 303: switch,
304: DCT unit, 305: Quantization unit, 3
06: inverse quantization unit, 307: IDCT unit,
308 ... adder, 309 ... frame memory,
310: motion compensation unit, 311: switch,
312: Mode selection unit, 313: Variable length coding unit, 401: Blocking unit, 402: D
CT unit 403 Quantizer 404 Variable length encoder 601 Variable length encoder 602
... Inverse quantization unit, 603 ... IDCT unit, 60
4 Adder 605 Frame memory 6
06: motion compensation unit, 607: switch, 7
01: variable length decoding unit, 702: inverse quantization unit, 703: IDCT unit, 811: macroblock static / dynamic judgment unit, 812: subtractor, 813
··· Moving object determination unit 814 ··· Moving object synthesis display unit
Specified color designation unit, 820: Specific color object composite display unit, 901: Moving image color monitoring device, 901C ...
Moving image celestial body monitoring device 901C ··· Moving image multi-color simultaneous monitoring device 902 ··· Moving image input unit 903 ··· Specific color designation information 904 · Input moving image · 905 ···
Communication line, 906 ... upper-level communication means, 907 ...
Recording means 908... Information filter section 909
... Communication means, 910 ... Color detection output, 911
... Detection range output, 912 ... Camera installation position and shooting range information on the celestial sphere, 913 ... Known nova database, 920 ... Host monitoring device 921 ... Close-up train, 922 ... Manufacturing control Device, 930 ... moving image movement monitoring device, 931 ...
-Movement detection, 932 ... call signal, 933 ...
Number of people gathered, 935 ... elevator control device. Applicant Toshiba Corporation Patent Attorney Saichi Suyama

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] December 28, 1999 (1999.12.
28)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] Claims

[Correction method] Change

[Correction contents]

[Claims]

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0006

[Correction method] Change

[Correction contents]

[0006]

Means for Solving the Problems] moving image monitoring system according to claim 1 of the present invention includes an image input means for inputting an image signal by photographing a monitoring object, macro said image signal blocks
Video encoding means for dividing video data into video data , encoding the video data as video data, video decoding means for decoding encoded video data, and macroblocks decoded by the video decoding means. Moving object detecting means for judging whether the image in the macroblock is still or moving from the decoding result and detecting a moving object from the still / moving judgment result for each macroblock; At least a synthesizing means for synthesizing the reproduced image signal created by the converting means, and displaying or outputting the result by determining the moving object.

[Procedure amendment 3]

[Document name to be amended] Statement

[Correction target item name] 0007

[Correction method] Change

[Correction contents]

[0007] According to the present invention, in the moving image monitoring system, moving determination of determining whether the image is a still state or dynamic conditions in the macro block from the decoded result of each macroblock is decoded for each macro-block Since the moving object is determined from the result, in the detection of the moving object, the amount of calculation can be reduced, and the calculation time and cost can be reduced.
A quick and efficient video monitoring system can be configured, and a wide range of applications can be obtained.

[Procedure amendment 4]

[Document name to be amended] Statement

[Correction target item name] 0008

[Correction method] Change

[Correction contents]

[0008] moving image monitoring system according to claim 2 of the present invention includes an image input means for inputting an image signal by photographing a monitoring object, the moving image data by dividing the image signal into macroblocks
A moving image encoding means for encoding as data, encoded with video decoding means for decoding the moving picture data, macroblock from the decoded results each macroblock decoded by said video decoding means Moving object detecting means for determining whether the image in the still state or the moving state and determining the moving object from the static / moving determination result for each macroblock , specific color detecting means for detecting an object of a specific color, An image synthesizing unit for synthesizing a display of the area where the specific color is detected and a reproduced image created by the moving image decoding unit; and an area of the moving object determined by the moving image decoding unit. synthesis of synthesizing said reproduced image signal
Means for determining and displaying or outputting a moving object and a specific color object.

[Procedure amendment 5]

[Document name to be amended] Statement

[Correction target item name] 0009

[Correction method] Change

[Correction contents]

According to the invention of claim 2, in moving image monitoring system, moving determination of determining whether the image is a still state or dynamic conditions in the macro block from the decoded result of each macroblock is decoded for each macro-block Since the moving object is determined from the result and the specific color is detected, the amount of calculation, the calculation time and the cost can be reduced in the detection of the moving object and the detection of the specific color object. A video surveillance system can be configured, and a wide range of applications can be obtained.

[Procedure amendment 6]

[Document name to be amended] Statement

[Correction target item name] 0010

[Correction method] Change

[Correction contents]

[0010] moving image monitoring system according to claim 3 of the present invention includes an image input means for inputting an image signal by photographing a monitoring object, the moving image dividing the image signal into macroblocks
A moving image encoding means for encoding as data, and still image coding means for coding the still image data from the image signal, the still image data storage means for storing still image data, the moving image encoded Moving image decoding means for decoding data, still image decoding means for decoding coded still image data, and macro decoding of each macro block decoded by the moving image decoding means from a decoding result. A moving object detecting unit that determines whether an image in a block is in a stationary state or a moving state, and determines a moving object from a static / moving determination result for each block , a specific color detecting unit that detects an object of a specific color,
An image synthesizing unit for synthesizing a display of a moving object and a region where a specific color is detected and a reproduced image created by the moving image decoding unit; and a moving object region determined by the moving image decoding unit. Moving image display means for displaying the synthesized image with the reproduced image signal, and still image display means for displaying a still image; and output means for processing and outputting the determination result of the moving object. The means is characterized by comprising means for calling and transmitting the stored still image data upon request, decoding, and displaying it. According to the third aspect of the present invention, in addition to the moving object detecting means and the specific color object detecting means, there is provided a means for retrieving, transmitting, decoding, and displaying the still image data stored in response to the request. It is possible to retrieve detailed information by calling a still image with good image quality.

[Procedure amendment 7]

[Document name to be amended] Statement

[Correction target item name] 0011

[Correction method] Change

[Correction contents]

According to a fourth aspect of the present invention, there is provided a moving image monitoring system, comprising: an image input unit for photographing a monitoring target and inputting an image; an input image recording unit for recording the input image; Time adding means for managing, image adjusting means for adjusting an image, and an image signal from the image adjusting means.
A moving image encoding unit that divides the image into macroblocks and encodes the image as a moving image; an encoding control unit that controls the encoding of each image; and a method that selects a specific image and encodes a still image into the encoding control unit A still image encoding / synchronizing means for issuing a command;
A still image encoding means for inputting and encoding a specific still image from an input image recording means according to an instruction of the encoding control means, a still image decoding means for decoding encoded still image data, Still image display means for displaying the encoded still image, moving image decoding means for decoding the encoded moving image data, recording means for storing the encoded moving image data using the recording means, Mak from the decoding results for each macro block decoded by the video decoding means
B and moving object detection means for determining the moving object from the movement determination result image is determined or a still state or dynamic conditions for each block in the block, the object of a specific color from the result detected by said moving object detecting means A specific color detecting unit for detecting, a moving object and an image synthesizing unit for synthesizing a display of an area where the specific color is detected and a reproduction image created by the moving image decoding unit, and a moving image for displaying the synthesized image There is provided a display unit including an image display unit and a still image display unit for displaying a decoded still image, and monitors a moving image.

[Procedure amendment 8]

[Document name to be amended] Statement

[Correction target item name] 0012

[Correction method] Change

[Correction contents]

According to the fourth aspect of the present invention, the time of the input image is
To add time and adjust the image further.
Thus, the moving picture monitoring of the present invention can be efficiently and effectively performed.

[Procedure amendment 9]

[Document name to be amended] Statement

[Correction target item name] 0053

[Correction method] Change

[Correction contents]

The image adjusting means 104 also serves as condition setting means such as image size, image compression ratio, color correction and the like. It is input to the moving picture coding means 105 and coded. Further, a still image encoding unit is provided. The still image has a structure in which the still image is branched from the input image recording means 103 and direct output data is input to the still image encoding means 115 for encoding to obtain an image close to the acquired original image. Each encoding unit performs encoding according to a command from the encoding control unit 116 . The encoding control means 116 re-acquires the specified time image data from the decoding block 126
To order. Upon receiving the instruction, the still image encoding unit 115 acquires the image data of the designated time from the input image recording unit 103, encodes the still image, and transmits it to the decoding block 126 .

[Procedure amendment 10]

[Document name to be amended] Statement

[Correction target item name] 0055

[Correction method] Change

[Correction contents]

[0055] moving image encoded data video decoding means
If the monitoring image data input to 121 is required, the encoded data is recorded in the recording means 120 as it is. Store each species a detected based on the data specified color detection unit 112 and the moving object detecting means 114 required for automatic monitoring to record remains encoded data obtained by the video decoding unit 121 to the recording unit 120 This is because it is performed from the data obtained.

[Procedure amendment 11]

[Document name to be amended] Statement

[Correction target item name] 0056

[Correction method] Change

[Correction contents]

[0056] encoded data input to the video decoding unit 121 in the case of using the moving object detection unit 114 is input to the moving object detecting means 114 enters the data in the decoding process to a moving object detection block 128, the moving object adjustment The moving object in the image is detected under the condition of the means 113. The moving object adjusting means 113 is provided with settings of a minimum detection macroblock, a maximum detection macroblock, a detection position, a detection sensitivity, and the like of the moving object.

[Procedure amendment 12]

[Document name to be amended] Statement

[Correction target item name] 0057

[Correction method] Change

[Correction contents]

The output of the moving object detecting means 114 is input to the specific color detecting means 112 in the specific color detecting block 129 as required, and the specific color is detected under the conditions of the specific color adjusting means 111. The specific color adjustment unit 111 sets a specific color, determines whether or not to perform a specific color detection on the macroblock detected by the moving object detection unit 114, searches for colors in the macroblock detected by the moving object detection, and specifies a specific color. Calculation of the specific color ratio in the macroblock detected by the detection can be set.

[Procedure amendment 13]

[Document name to be amended] Statement

[Correction target item name] 0058

[Correction method] Change

[Correction contents]

The moving object detecting means 114 and the specific color adjusting means 11
The macroblock information detected in step 1 is the detection result output means.
The data is output from the 121 and can be registered in the database and calculated as needed. Further, by combining the display of macro block of the moving object detection result and the specific color detection result on the decoded image by the image combining means 109 for displaying by reflecting the output image of the video decoding unit 121 at the same time, the moving image display The system can be displayed by means 110.

[Procedure amendment 14]

[Document name to be amended] Statement

[Correction target item name] 0059

[Correction method] Change

[Correction contents]

Here, a description will be given of the still image acquisition as an explanation of the characteristic functional unit. FIG. 4 is a block diagram showing an example of the configuration of the encoding unit 125. The input video signal sent from the image input means 101 is first converted by an A / D unit 201 from an analog signal to a digital signal. The digitized video signal passes through a switch 202 for determining whether or not to encode the frame, and is then input to a format conversion unit 203. The format converter 203 converts the RGB signals into YCbCr
The signal is converted into a signal and input to the image size conversion unit 204 and the input image buffer 206. The image size conversion unit 204 applies a low-pass filter to the input video signal, down-samples the image signal to reduce the image size, and inputs the reduced image size to the video encoding unit 205. The moving image encoding unit 205 encodes the input video signal as a moving image, and inputs the encoded data to the multiplexing unit 210. On the other hand, the encoding control unit 209 turns on the switch 207 when receiving a retransmission request signal from the feedback path 106,
A frame that matches the time code of the retransmission request signal is selected from the input image buffer, encoded by the still image encoding unit 208, and sent to the decoding unit. The coding control unit 209 determines coding parameters using the coding results of the moving picture coding unit 205 and the still picture coding unit 208, and controls the code amount.

[Procedure amendment 15]

[Document name to be amended] Statement

[Correction target item name] 0062

[Correction method] Change

[Correction contents]

FIG. 8 is a block diagram showing an example of the configuration of the moving picture decoding unit 121. In the variable length decoding unit 601, if the mode of the macroblock is INTRA, the mode changeover switch 607 is turned off, and the variable length decoding unit 601
The quantized DCT coefficient information decoded in
The IDCT unit 603 performs inverse discrete cosine transform processing to generate a reproduced image signal. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the display unit 110. In the variable length decoding unit 601, the mode of the macroblock is set to INTER.
And NOT_CODED, the mode changeover switch 607 is turned on, and the quantization decoded by the variable length decoding unit 601 is performed.
The DCT coefficient information is inversely quantized by the inverse quantization unit 602, and the
The inverse discrete cosine transform process is performed by the unit 603, and the motion compensation unit 606 performs motion compensation on the reference image based on the motion vector information decoded by the variable length decoding unit 601. An image signal is generated.
The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the display unit 110.

[Procedure amendment 16]

[Document name to be amended] Statement

[Correction target item name] 0066

[Correction method] Change

[Correction contents]

In the variable length decoding section 601, if the mode of the macroblock is INTER and NOT_CODED, the mode changeover switch 607 is selected to be ON , and the variable length decoding section 601 is turned on.
The quantized DCT coefficient information decoded in
The IDCT unit 603 performs inverse discrete cosine transform processing, and based on the motion vector information decoded by the variable length decoding unit 601, a motion compensation unit 606 performs motion compensation on the reference image, and At 604, a reproduced image signal is generated. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the moving object synthesis display unit 814 in the moving object detection unit 128.

[Procedure amendment 17]

[Document name to be amended] Statement

[Correction target item name]

[Correction method] Change

[Correction contents]

In the variable length decoding unit 601, if the mode of the macroblock is INTER and NOT_CODED, the mode changeover switch 607 is selected to be ON , and the variable length decoding unit 601
The quantized DCT coefficient information decoded in
The IDCT unit 603 performs inverse discrete cosine transform processing, and based on the motion vector information decoded by the variable length decoding unit 601, a motion compensation unit 606 performs motion compensation on the reference image, and At 604, a reproduced image signal is generated. The reproduced image signal is stored in the frame memory 605 as a reference image, and is input to the specific color object synthesis display unit 820 in the specific color object detection unit 129.

[Procedure amendment 18]

[Document name to be amended] Statement

[Correction target item name] 0085

[Correction method] Change

[Correction contents]

On the other hand, in the specific color object detecting section 129,
As shown in the flowchart of FIG. 22, the macroblock static / moving determination unit performs three processes of macroblock static / movement determination (S701), specific color object determination (S702), and specific color object composite display (S703) for each frame. 811, the specific color object determination unit 818 and the specific color object synthesis display unit 820 perform the processing.

[Procedure amendment 19]

[Document name to be amended] Statement

[Correction target item name] 0089

[Correction method] Change

[Correction contents]

If MODE is INTRA, the sum of absolute differences (SAD) between the reproduced image signal of the macroblock and the frame memory one frame before is calculated and compared with the threshold value T0 (S804). If it is larger than the threshold value T0, the macroblock is determined to be a moving macroblock, and M [i] [j]
Is set to TRUE (S806). If MODE is INTER, the motion vector information from variable length decoding section 601 and DC
From the T coefficient information, the sum of the absolute value of the motion vector of the macroblock Σ | MV | and the sum of the absolute values of the DCT coefficients Σ | COF | are calculated and compared with the respective thresholds T1 and T2 (S805).
If the sum of the absolute value of the motion vector and the sum of the absolute values of the DCT coefficients are equal to or larger than the threshold, the macroblock is determined to be a moving macroblock, and TRUE is substituted into M [i] [j] (S806). The specific color object determination unit 818 obtains the static / movement determination information for each macroblock from the macroblock
A specific color object is determined. FIG. 24 is a specific flowchart. As shown in FIG. 24 , the processing content includes two processes, a specific color detection process (S901) and a specific color object inclusion process (S902). In specific color detection process (S901), in order to prevent erroneously detected a moving macro block by the noise at the time fluctuations or image capture of small objects in the background image, dynamic macroblocks surrounding the macroblock is stationary all Has been removed. The specific color object inclusion processing (S902)
From the result of the still / movement determination after removing the noise, processing is performed to detect the smallest rectangle that includes the region where the adjacent moving macroblock exists. Through this processing, a rectangle that includes the specific color object can be found.

Continued on the front page (72) Inventor Takeshi Kenjo 1 Kosuka Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa Inside the Toshiba R & D Center (72) Inventor Shun Ikeda 8 Shinsugita-cho, Isogo-ku, Yokohama-shi, Kanagawa Shares (72) Inventor Yoshihiro Kikuchi 1 Kodamu Toshiba-cho, Saiwai-ku, Kawasaki-shi, Kanagawa F-term (reference) 5C054 AA01 CA04 CC05 CD03 CG02 CH02 DA01 EA01 EF06 EG10 FA01 FB03 FC07 FC12 FE28 FF03 FF06 GB02 GB15 GB17 HA02 HA03 HA16 HA18 HA20 HA26 HA28 HA35 HA39 5C055 AA01 AA03 BA06 BA07 CA01 CA03 EA01 FA22 GA44 HA18 HA31

Claims (25)

[Claims] 1. An image input unit for photographing a monitoring target and inputting an image signal, a moving image encoding unit for encoding the image signal as a moving image, and decoding the encoded moving image data. A moving image decoding unit, and for each block decoded by the moving image decoding unit, determining whether an image in the block is in a still state or a moving state based on a decoding result, and detecting a moving object from the still / moving determination result for each block. A moving object detecting unit, and at least a unit for combining and displaying the determined moving object region with the reproduced image signal created by the moving image decoding unit; determining the moving object; and displaying or outputting a result. A moving picture monitoring system characterized by the following. 2. An image input means for photographing an object to be monitored and inputting an image signal, a moving image encoding means for encoding the image signal as a moving image, and a moving image for decoding encoded moving image data An image decoding unit, and a moving unit that determines whether an image in the block is in a still state or a moving state based on a decoding result of each block decoded by the moving image decoding unit and determines a moving object based on a still / moving determination result for each block. An object detection unit and a specific color detection unit that detects an object of a specific color; an image synthesis unit that displays a display of an area where the moving object and the specific color are detected and a reproduction image created by the moving image decoding unit Means for combining and displaying the determined area of the moving object with the reproduced image signal created by the moving picture decoding means. A video surveillance system characterized in that the video is monitored or displayed. 3. An image input unit for photographing a monitoring target and inputting an image signal, a moving image encoding unit for encoding the image signal as a moving image, and a still image for encoding a still image from the image signal. Encoding means and still image data storage means, moving image decoding means for decoding encoded moving image data, still image decoding means for decoding encoded still image data, A moving object detection unit and a specific color image for each block decoded by the moving image decoding unit, which determine whether the image in the block is still or moving from the decoding result, and determine a moving object from the still / moving determination result for each block. Specific color detection means for detecting an object; image synthesis means for synthesizing a display of a moving object and an area where the specific color is detected and a reproduced image created by the moving image decoding means; Means for combining and displaying the area of the object with the reproduced image signal created by the moving image decoding means, and still image display means for displaying a still image,
The moving image further includes an output unit that processes and outputs the determination result of the moving object, and the still image display unit includes a unit that calls, transmits, decodes, and displays the still image data stored in response to the request. Image monitoring system. 4. An image input unit for photographing a monitoring target and inputting an image, an input image recording unit for recording the input image, a time adding unit for managing a time of the input image, and an image for adjusting the image Adjusting means, a moving image encoding means for encoding an image signal from the image adjusting means as a moving image,
Encoding control means for controlling each encoding,
A still image encoding synchronizing unit for selecting a specific image and instructing the encoding control unit to encode a still image, and inputting a specific still image from an input image recording unit according to an instruction of the encoding control unit to perform encoding. A still image encoding unit, a still image decoding unit for decoding the encoded still image data, a still image display unit for displaying the decoded still image, and a decoding of the encoded moving image data. Video decoding means for decoding, a recording means for storing encoded video data using a recording means, and an image in a block in a still state based on a decoding result of each block decoded by the video decoding means. A moving object detection unit that determines whether a moving state and determines a moving object from a static / movement determination result for each block, a specific color detection unit that detects an object of a specific color from a result detected by the moving object detection unit, Transfer Image synthesizing means for synthesizing the display of the area where the object and the specific color are detected and the reproduced image created by the moving image decoding means; moving image display means for displaying the synthesized image; A moving image monitoring system comprising a display unit including a still image display unit for displaying an image, and monitoring a moving image. 5. The moving picture monitoring system according to claim 1, wherein the moving picture monitoring system comprises: a multiplexing means for multiplexing the coded moving picture data and the still picture data; a transmitting / receiving stage for transmitting / receiving the multiplexed data; 2. The moving picture monitoring system according to claim 1, further comprising demultiplexing means for separating. 6. A moving object in a moving image by performing a moving image monitoring on an image input of an object photographed by using the moving image monitoring system according to any one of claims 1 to 5. The fire and the smoke are detected to detect the occurrence of a fire and / or the situation of the fire is determined, and the notification and the warning of the fire are automatically performed. 7. A target person is photographed using the moving image monitoring system according to any one of claims 1 to 5, an image is input, and moving image monitoring is performed to reduce the size of the person. A costume changing system characterized by the size of the costume being enlarged or reduced in accordance with it, and displayed over the person to match the costume. 8. Using the video monitoring system according to any one of claims 1 to 5, performing video monitoring on an image input of an object captured from an image input device mounted on a wheelchair. A wheelchair guidance system characterized in that the wheelchair can automatically move to a required place by following a moving object or a specific color to automatically drive the wheelchair. 9. Automatically acquiring warning and still images by photographing an intersection, inputting an image, monitoring the moving image, and monitoring the movements and signals of people and passing objects in the intersection. An intersection monitoring system using the moving image monitoring system according to any one of claims 1 to 5, wherein the intersection monitoring is performed. 10. The method according to claim 1, wherein:
Using the moving image monitoring system described in the paragraph, shooting a road with no visibility and inputting an image, by performing video monitoring, detecting a moving object on the road with no visibility,
A line-of-sight road monitoring system characterized by providing necessary warnings. 11. A method for monitoring a speed of a road vehicle by capturing an image of a traveling vehicle using the moving image monitoring system according to claim 1, inputting an image, and monitoring the moving image, A vehicle speed monitoring system, which warns a vehicle exceeding a speed limit to automatically acquire a still image. 12. The method according to claim 1, wherein
By using the moving image monitoring system described in the paragraph, photographing a player with a specific color mark attached to each part of the body and, if necessary, equipment, inputting an image, and performing the moving image monitoring, the sports form of the sports form Sports form analysis system characterized by performing comparative analysis. 13. The method according to claim 1, wherein:
Using the moving image monitoring system described in the section, photographing the vicinity of the door of each floor of the elevator and inputting an image, and performing the moving image monitoring, detects the stop of the moving object near the door and detects the stop of the elevator. An automatic elevator control system characterized by input as a call signal. 14. The automatic elevator control system according to claim 13, wherein the number of moving objects stopped near doors on each floor of the elevator is detected, and the elevator is preferentially controlled in accordance with the number of moving objects. 15. The method according to claim 1, wherein:
By shooting a person passing through a specific area using the video monitoring system described in the section and inputting an image, and performing video monitoring,
A passer management system, wherein the passer management system automatically identifies a passer of the specific area and performs pass management. 16. The method according to claim 1, wherein:
The entrance / exit area is managed by performing video surveillance on an image input taken at the entrance / exit of the area specified using the video surveillance system described in the section. Monitoring system. 17. The method according to claim 1, wherein:
A shelf monitoring system characterized by managing a shelf by monitoring a moving image in response to an image input photographing an article on the shelf and a periphery of the shelf using the moving image monitoring system described in the section. 18. The method according to claim 1, wherein:
By performing the video monitoring for the image input photographing the water surface using the video monitoring system according to the item, manages the degree of water pollution, and issues an alarm when the pollution exceeds a predetermined threshold. Water pollution degree monitoring system characterized by the emission. 19. The method according to claim 1, wherein:
Performs video monitoring for astronomical video input using the video monitoring system described in the section, specifies the color corresponding to the supernova, performs the color detection operation, and detects the star based on the presence or absence of the output color. An astronomical moving image monitoring system using a moving image monitoring system, which performs a comparison with a database that records known stars and outputs a notification output if the star does not exist in the database. 20. Any one of claims 1 to 5
For the partial moving image of the station platform or the whole moving image input using the moving image monitoring system described in the section, the moving image monitoring is performed using the white line of the platform as the designated color, and the monitoring result is notified to the train by the communication means. A home monitoring system characterized by: 21. Any one of claims 1 to 5
In response to an image input of the airport runway and taxiway using the video surveillance system described in the section, by performing the video surveillance, an airline airplane having a pre-specified color passes through the runway. A runway passage detection system for an airplane characterized by detecting. 22. Any one of claims 1 to 5
The moving image of hot iron on the production line of the steel mill is input using the moving image monitoring system according to the item, and the moving image monitoring is performed by using the color of the hot iron as a designated color, and the monitoring result is obtained. Is notified to the production control device using communication means. 23. Any one of claims 1 to 5
The moving image of the area at risk of landslide using the moving image monitoring system described in the paragraph is input, the moving image monitoring is performed with the color of the previously specified landslide as the specified color, and the monitoring result value exceeds the specified threshold value A hazardous area monitoring system characterized by outputting an alarm in some cases. 24. Any one of claims 1 to 5
By performing the video monitoring with the color of the traffic light as a designated color for the video input from the image input device installed in the vehicle using the video monitoring system according to the item, the signal state of the traffic signal is monitored and the signal is monitored. A road guidance monitoring system characterized by providing traffic guidance. 25. A road guidance monitoring system according to claim 24, wherein the position accuracy and the guidance accuracy are improved by interlocking with car navigation using GPS.