JP2004032814A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To record and reproduce a moving image while monitoring it. <P>SOLUTION: The image processing apparatus comprises a video recorder, a scaling section 63, a color converter 62 and a direct memory access (DMA) transfer controller 61. The video recorder separates a video signal into two field of interlaced signals. The scaling section 63 scales the separated two field signals independently from each field. The color converter 62 converts in color of the two field signals independently from each field. The DMA transfer controller 61 transfers respective two field signals which have undergone scaling and color conversion to different storage devices. <P>COPYRIGHT: (C)2004,JPO

Description

The present invention relates to an apparatus that captures, records, and reproduces an input moving image, and particularly relates to an apparatus that reproduces a moving image from a free scene (position).

The conventional image recording / reproducing apparatus has a configuration shown in FIG.

As shown in FIG. 10, the image recording / reproducing apparatus includes an information processing device 21 and an image input device 22. The image input device 22 is a unit for inputting an analog moving image (video), and is realized by a video camera, a VTR, a television, or the like. Here, an example of a video camera is shown. The information processing apparatus 21 includes a unit for digitizing an analog moving image input from the image input device 22 and a unit for recording a moving image digitized by the unit. This is a device for recording a moving image input from the control unit 22.

Next, an outline of processing of the conventional image recording / reproducing apparatus having the apparatus configuration shown in FIG. 10 will be described with reference to FIG.

FIG. 11 is a flowchart of a process for identifying a portion where a scene has changed in a series of continuous images (moving images) input from the image input device 22 in order to perform cue reproduction. .

In this process, first, the capture of a moving image is started (step 501), and an analog image of one frame or one field input from the image input device 22 is digitized (step 502). Here, the moving image input from the image input device 22 has a standardized format such as NTSC, PAL, or SECAM.

Next, in the above process, the digitized image is stored in the main storage device of the information processing device 21 (step 503). Then, the image is compared with the image stored immediately before (step 504), and it is determined whether or not the scene has changed (step 505). At this time, whether or not the scene has changed is determined by comparing the two images on a pixel-by-pixel basis, calculating the difference between them, and obtaining the sum of the differences for all the pixels. For example, if the sum exceeds an arbitrary threshold, it is determined that the scene has changed.

If it is determined in step 505 that the scene has changed, the time from the start of capturing the moving image of the image is stored in the main memory (step 506). Then, it is determined whether or not the capture has been completed (step 507). If the capture has not been completed, the process returns to step 502 and the process is repeated. If the capture has been completed, a series of uncompressed moving images is stored in the auxiliary storage device of the information processing device 21 (step 508), and the process ends.

Thereafter, the conventional image recording / reproducing apparatus displays a list of images at positions where scenes have changed on the display device of the information processing apparatus 21, and displays the list instructed by the user from the input apparatus of the information processing apparatus 21. At step 506, the time stored in the main memory corresponding to an arbitrary image is extracted, and at step 508, the moving image stored in the auxiliary storage device is cue-reproduced from the image (position) corresponding to the time. Do it.

JP-A-6-133305 discloses a technique of automatically extracting a scene change of a moving image and encoding the moving image including the information.

A technique for displaying a moving image on a display device or the like in real time is described on pages 102 to 109 of the April 1996 issue of Interface Magazine, published by CQ Publishing Company.

However, in the prior art as shown in FIG. 10, a series of uncompressed moving images is captured to detect a change in a scene, and is stored in an uncompressed auxiliary storage device. No consideration has been given to the amount of data

Further, in Japanese Patent Application Laid-Open No. Hei 6-133305, since information of a scene change is included in a compressed moving image, it is necessary to search the moving image from the beginning to the end before reproduction every time before reproduction. Not considered.

The technology described in the interface magazine does not consider displaying a moving image on a display device and recording the moving image in an auxiliary storage device or the like.

An object of the present invention is to convert a moving image input from an image input device into an uncompressed moving image and a moving image compressed using a compression technique capable of reproducing at a relatively high compression rate and high image quality in real time. Another object of the present invention is to provide an image capturing device that captures image information into an information processing device, an image storage / reproducing device equipped with the image capturing device, and an image input device.

Further, by associating an arbitrary image in the non-compressed moving image with an arbitrary image in the compressed moving image from information obtained based on the uncompressed moving image, recording and reproducing and displaying the moving image To provide an image storage / reproducing apparatus for performing image recording / reproducing, which makes it possible to easily perform image recording / reproducing.

Still another object is to capture the uncompressed moving image into the information processing device, and convert the odd field data and the even field data in the interlace signal input from the image input device into different sizes and color formats. An image capturing device for converting and transferring the image to a different area or device, and mounting the image capturing device, simultaneously monitoring the moving image input from the image input device on a display device included in the information processing device, and simultaneously performing An object of the present invention is to provide an image storage / reproduction device for recording / reproducing a moving image.

The present invention relates to a video decoder for inputting an analog moving image and digitizing the input moving image in an image capturing device mounted on an image recording / reproducing device for recording a moving image having a main memory and a display device. A moving image compression unit for compressing the digitized moving image into a compressed moving image; and transferring the digitized moving image output from the video decoder to the main storage and the display device, simultaneously with the moving image compression. And a moving image input interface unit for transferring a compressed moving image output from the unit to the main memory.

The present invention relates to an image recording / reproducing apparatus for recording a moving image having a main memory and a display device, wherein an analog moving image is input and a video decoder for digitizing the input moving image is provided. A moving image compression unit for compressing a moving image into a compressed moving image, and a compressed moving image output from the moving image compression unit while transferring the digitized moving image output from the video decoder to the main storage and the display device. A moving image input interface unit for transferring an image to the main memory, wherein the moving image input interface unit separates the digitized moving image into two field signals in an interlace signal, and converts one of the field signals into the main memory. And the other field signal is transferred to the display device, and the display device is connected to the moving image input interface. Displaying a moving image by a field signal transferred from the memory device, and the main memory stores the field signal transferred from the moving image input interface unit at the same time as the display device displays the moving image. Device.

Further, preferably, the image recording / reproducing apparatus has a frame memory for holding at least one screen image of the moving image digitized by the video decoder, and the moving image input interface unit is input from the video decoder. A video signal input / output unit that divides a digital moving image into odd field data and even field data and stores them in the frame memory; and a scaling unit that sequentially takes out odd field data and even field data from the frame memory and performs scaling. A color conversion unit that converts data input from the scaling unit into a predetermined color format, a compressed image input / output unit that captures data input according to a predetermined protocol from the video compression unit, Input from the above color converter A DMA transfer control unit for transferring data to be transferred and a compressed moving image input from the compressed image input / output unit to the main storage or the display device, and data to a bus generally employed in an information processing device. And a bus input / output unit for receiving data flowing through the bus.

Preferably, information extracting means for extracting arbitrary information based on the uncompressed moving image captured by the image capturing means, and information extracted from the information extracting means and captured by the image capturing means. Correspondence information generating means for associating the compressed video with the compressed video, wherein the information extracting means captures a video of the series of non-compressed video captured from the uncompressed video capture device at any point in time. A function for extracting a time from the start, wherein the correspondence information generating means extracts an image (position) having the same time as the time in the compressed moving image captured from the compressed image capturing device, and It has a function to add.

Preferably, the image recording / reproducing apparatus further includes an input device for a user to give an instruction, and a compressed image reproducing device for reproducing the compressed moving image, wherein the compressed image reproducing device includes the input device. An image recording / reproducing apparatus having a function of cueing and reproducing the compressed moving image from an image associated with the image or an image near the image before or after the image in response to an instruction input from a unit is also conceivable. .

好 ま し く Preferably, the image capturing device and the compressed image reproducing device may be combined.

The image recording / reproducing apparatus according to the present invention performs, for example, the following operation. In response to an instruction input by the user from the input unit, the image capturing device digitizes a moving image input from the image input device and captures the moving image in an uncompressed state, and a moving image input from the image input device. The image is digitized and converted into a compressed moving image that can be reproduced with a high compression rate and high image quality, and is captured. At this time, the two moving images are generated from the same video source and are captured simultaneously and in real time.

Then, for example, when detecting a change in a scene, the information extracting means may include, at least, an image immediately after a scene obtained by analyzing the uncompressed moving image and a start of moving image capturing of the image. The correspondence information generating means extracts an image (position) having the same time as the time in at least the compressed moving image captured from the image capturing device, and extracts the two images. Generate correspondence information for associating.

Further, the uncompressed moving image is transferred to the main storage and the display device while being divided into odd field data and even field data, respectively, so that the moving image input from the image input device can be monitored by the display device. It is possible to detect scene changes.

Further, the display device displays at least an image immediately after the scene change, and when the user instructs the image by the input unit, the compressed image reproduction device, from the image and the correspondence information, An image to start cueing reproduction in the compressed moving image is specified and reproduced.

According to the present invention, an uncompressed moving image and a compressed moving image can be simultaneously captured (recorded) in the information processing apparatus, and further, as a result of analysis using the uncompressed moving image, Any image in the compressed moving image corresponding to the image can be specified, and by correlating these two images, the cue reproduction of the compressed moving image from the position specified by the user or the like can be easily performed. Thus, an image recording / reproducing device can be provided.

Furthermore, the moving image input from the image input device is processed in real time and simultaneously as an uncompressed moving image and a moving image compressed using a compression technique capable of reproducing at a relatively high compression rate and high image quality. When the compressed moving image is taken into the information processing device, an image recording / reproducing device that takes in the compressed moving image preferentially can be provided.

Further, when the uncompressed moving image is taken into the information processing device, the odd field data and the even field data in the interlace signal input from the image input device are converted into the same or different size and color format. By transferring the image to the same or different area or device, the image recording / reproducing that simultaneously records the moving image while monitoring the moving image input from the image input device on a display device provided in the information processing device. An apparatus can be provided.

Hereinafter, the best mode for carrying out the present invention will be described.

Hereinafter, Embodiment 1 according to the present invention will be described with reference to the drawings.

FIG. 14 is a hardware configuration diagram of an information processing apparatus such as a personal computer to which the image recording / reproducing apparatus of the first embodiment can be applied.

As shown in FIG. 14, the device includes a CPU 1, a main memory 2, an auxiliary storage device 3, an input device 4, a display device 5, an image input device 6, an image capturing device 12, a compressed image reproducing device, The device 9 is provided. The components other than the image input device 6 are connected by a bus 10 so that necessary information can be transmitted between the components.

The image input device 6 is connected to the image capturing device 12 and configured to be able to transmit video information from the image input device 6 to the image capturing device 12. The image capturing device 12 is configured to be mounted on an information processing device 21 such as a PC constituted by the CPU 1 and the like connected via a bus, or to be built in the image input device 6 as shown in FIG. The image capturing device shown in FIG. 22 has the same configuration as the image capturing device 12 shown in FIG. 15 except that a CCD camera 56 is incorporated.

The main memory 2 is a means for functioning as a work area and storing necessary programs, and can be realized by a RAM for the former and a ROM for the latter. The auxiliary storage device 3 is a means for storing a program for controlling the operation of the device, a compressed moving image, and the like, and can be realized by, for example, a flexible disk, a hard disk, a memory card, a DVD, or the like. The input device 4 is a unit for inputting necessary commands and information, and can be realized by a pointing device such as a keyboard and a mouse, for example. The display device 5 is a unit that displays various information such as a reproduction display of a compressed moving image, and can be realized by, for example, a CRT, an EL display, a plasma display, a liquid crystal display, or the like.

The image input device 6 is a unit for inputting an analog moving image (video), like the image input device 22 in the above-described conventional technology, and is realized by a video camera, a VTR, a television, a TV tuner, or the like. The image capturing device 12 is means for digitizing an analog moving image input from the image input device 6 and compressing the digitized moving image. The compression (encoding) means is realized by, for example, MPEG (Moving Picture Experts Group) capable of reproducing with high compression rate and high image quality. It can also be realized by the compression technology described in "Point Illustrated Latest MPEG Textbook", pp. 28-29, edited by the Multimedia Communications Research Group, ASCII Publishing Bureau. The details of the MPEG compression technique are described in "Point Illustrated Latest MPEG Textbook", P89-P165, edited by Multimedia Communications Research Group, ASCII Publishing Bureau.

The compressed image reproducing device 9 is a unit that reproduces a moving image that has been compressed using the compression technology in the image capturing device 12 and displays the reproduced image on the display device 5. For example, if the compression technology is MPEG. For example, it can be realized by an MPEG decoder.

The CPU 1 performs a predetermined operation according to a program stored in the main memory 2 and the auxiliary storage device 3 in advance.

Next, FIG. 15 shows a block diagram of the image capturing device 12.

As shown in FIG. 15, the image capturing device 12 includes a moving image input interface unit 50, a frame memory 51, a moving image compression unit 52, and a video decoder 53.

The video decoder 53 is connected to the image input device 6, and digitizes and outputs an analog moving image input from the image input device 6. For example, if the analog moving image input from the image input device 6 is an NTSC signal, the video decoder 53 can be realized as an NTSC decoder. In this case, the odd field signal and the even field signal in the interlaced signal of the NTSC format are digitized and output in time series. The output moving image is input to the moving image compression unit 52 and the moving image input interface unit 50 at the same time. When the signal output from the image input device 6 to the image capturing device 12 is not an analog signal but a digital signal, the image capturing device 12 does not require the video decoder 53.

The image compression unit 52 compresses the digitized moving image output from the video decoder 53 by using a compression technique such as MPEG. Then, the compressed moving image is output to the moving image input interface unit 50. For example, if the compression technique is MPEG, the compressed moving image is an MPEG stream.

The frame memory 51 is for holding an image of at least one screen (frame) of a moving image digitized by the video decoder 53, and is connected to the moving image input interface unit 50.

The moving image input interface unit 50 is connected to the bus 10, temporarily stores a digital moving image input from the video decoder 53 in the frame memory 51, and further stores the image stored in the frame memory 51 through the bus 10. DMA transfer to the storage 2 or the auxiliary storage device 3 or the display device 5 is performed. At the same time, the compressed moving image input from the image compression unit 52 is also DMA-transferred to the main storage 2 or the auxiliary storage device 3 through the bus 10.

FIG. 16 is a block diagram of the moving image input interface unit 50.

As shown in FIG. 16, the moving image input interface unit 50 includes a bus input / output unit 60, a DMA transfer control unit 61, a color conversion unit 62, a scaling unit 63, a video signal input / output unit 64, and a compressed image input / output unit. An output unit 65 is provided.

The bus input / output unit 60 is realized in accordance with a bus protocol such as PCI or ISA generally employed in information processing devices, and sends data to the bus 10 or acquires data flowing in the bus 10.

The compressed image input / output unit 65 outputs data input from the moving image compression unit 52 according to a predetermined protocol to the DMA transfer control unit 61.

The video signal input / output unit 64 stores the digitized moving image input from the video decoder 53 in the frame memory 51. When the video decoder 53 is an NTSC decoder, the odd field data and the even field data in the moving image, that is, the interlace signal are input in chronological order, and the video decoder 53 determines whether the currently input data is odd field data. Since a signal indicating even field data is also sent, each field data is separately stored in a predetermined area in the frame memory 51 according to the signal. If only one screen worth of data (data obtained by combining odd field data and even field data) can be stored in the frame memory 51, the previously stored field data is overwritten and stored.

The scaling unit 63 sequentially retrieves one field of odd field data and one screen of even field data from the frame memory 51, scales the field data to a predetermined size, and outputs the data to the color conversion unit 62.

The color conversion unit 62 converts the data input from the scaling unit 63 into a predetermined color format and outputs the data to the DMA transfer control unit 61. Usually, the color format of the image data of the NTSC signal is the YUV format, and the frame memory 51 stores the data in the YUV format. However, the color format of the data to be transferred to the main storage 2 or the display device 5 is often the RGB format. Therefore, the color conversion unit 62 has a function of converting at least from the YUV format to the RGB format.

The DMA transfer control unit 61 transfers the data input from the color conversion unit 62 and the compressed moving image input from the compressed image input / output unit 65 to the main storage via the bus input / output unit 60 and the bus 10. 2 or the auxiliary storage device 3 or the display device 5 by DMA transfer. In addition, since the video decoder 53 and the moving image compression unit 52 operate asynchronously, data may be input from the color conversion unit 62 and the compressed image input / output unit 65 at the same time. In this case, the compressed moving image from the compressed image input / output unit 65 is preferentially DMA-transferred. This is because if any part of the compressed moving image is missing data, the entire data becomes meaningless.

FIG. 17 shows an example of a register configuration of the moving image input interface unit 50.

The register includes at least an input resolution 301, an output resolution 302, a color format 303, and a transfer destination address 304.

The input resolution 301 specifies the resolution of data input from the video decoder 53. The output resolution 302 specifies the resolution of data output on the bus 10. The color format 303 specifies a color format of data output on the bus 10. The transfer destination address 304 specifies an address that can be addressed by the CPU 1 to which the data output on the bus 10 is finally transferred.

{Each register is separately prepared for odd field data 310, even field data 320, and compressed moving image 330. However, for the compressed moving image 330, only the transfer destination address 304 is provided. The moving image input interface unit 50 operates according to the value set in each register in advance. The setting of a value in each register is a program for driving the image capturing device 12. The program is stored in the main storage 2 or the auxiliary storage device 3 and executed by the CPU 1. The user sets a desired set value in each register via the program.

Next, the set values and operations of each register in the image capturing device 12 will be described with reference to FIG. Here, a case where the set value of each register has the contents shown in FIG. 17 will be described.

に お い て In FIG. 18, area 1 and area 2 are areas individually secured in the main memory 2. The area 3 is a part of a VRAM (display memory) included in a general display device. The data stored in the VRAM is displayed on a CRT or the like. The area 3 is, for example, an area on the VRAM corresponding to the display area of the window when a moving image is displayed in the window.

In FIG. 17, the contents of the register of the transfer destination address 304 of the odd field data 310 are the start address of the area 2, the contents of the register of the transfer destination address 304 of the even field data 320 are the start address of the area 3, If the contents of the register of the transfer destination address 304 indicate the start address of the area 1, the odd field data and the compressed moving image are transferred to the main memory 2 and the even field data is transferred to the display device 5. .

Regarding the odd field data 310, since the contents of the register of the input resolution 301 are 640 × 240 and the contents of the register of the output resolution 302 are 160 × 120, the scaling unit 63 outputs 1/4 horizontally and vertically. The size is reduced to そ れ ぞ れ in each direction. Further, since the contents of the register of the color format 303 are RGB24, the color conversion unit 62 converts the data in the YUV format into the RGB format in which all the R, G, and B components are expressed by 8 bits. Then, the DMA transfer control unit 61 transfers the scaled and color-converted odd field data to the area 2 indicated by the contents of the register of the transfer destination address 304.

Regarding the even field data 320, since the contents of the register of the input resolution 301 are 640 × 240 and the contents of the register of the output resolution 302 are 320 × 240, the scaling unit 63 reduces the size in the horizontal direction to １ ／. I do. It is not enlarged or reduced in the vertical direction. Further, since the content of the register of the color format 303 is RGB8, the color conversion unit 62 converts the data in the YUV format into the RGB format expressed by 8 bits for each of the R, G, and B components. Then, the DMA transfer control unit 61 transfers the scaled and color-converted even field data to the area 3 indicated by the contents of the register of the transfer destination address 304.

Regarding the compressed moving image 330, the compressed moving image transmitted from the moving image compression section 52 is transferred to the area 1 indicated by the contents of the register of the transfer destination address 304.

As described above, the image capturing device 12 can divide a moving image input from a video camera, a VTR, or the like into odd field data and even field data and transfer the divided fields to different devices and areas in the information processing device. At the same time, data obtained by compressing the input moving image can be transferred to a different device or area in the information processing device. Furthermore, the odd field data and the even field data can be transferred after being converted into different sizes and color formats, respectively. Further, depending on the setting value of the register of the transfer destination address 304, the odd field data and the even field data can be transferred to the same device or area. In this case, the input moving image is recorded or displayed as it is. Will be.

FIG. 2 is a functional block diagram of the image recording / reproducing apparatus according to the first embodiment.

In FIG. 2, reference numeral 101 denotes a control unit for controlling each functional block in the image recording / reproducing apparatus of the first embodiment, 102 denotes a non-compressed image analysis unit having a function of analyzing an uncompressed image, and 103 denotes a non-compressed image. Correspondence information generating means 104 having a function of generating information for associating a compressed image with a compressed image is a compressed image reproducing device 104 having a function of reproducing and displaying a compressed moving image. This is realized by the CPU 1 executing a program stored in the storage device 3.

Next, the operation of the image storage / reproduction device of the present invention will be described with reference to the drawings.

Hereinafter, MPEG will be described as an example of a compression technique capable of reproducing at a high compression rate and high image quality, and a scene change will be described as information extracted from a digitized moving image, but the present invention is not limited thereto.

FIG. 3 is a flowchart showing the processing contents of the control means 101. A program for realizing the present process is stored in the main memory 2 or the auxiliary storage device 3. The program is executed by the CPU 1 in response to some event, for example, an instruction from the user input using the input device 4. Be executed.

As shown in FIG. 3, the control means 101 first starts capturing a moving image (step 121). Specifically, a program for driving the image capturing device 12 is executed, a value input by the user via the input device 4 or the like is set in a register, a work area is secured in the main memory 2, and the like. And preparing for acquiring a moving image. Also, as an example to be described, the set values of the registers are as shown in FIG. 17, and accordingly, each data is transferred as shown in FIG. That is, the work area on the main memory 2 includes the area 1 and the area 2.

Next, the control means 101 acquires an uncompressed image input from the image input device 6 and digitized by the image capturing device 12 and stores it in the area 2 (step 122). At this time, the data stored in the area 2 is odd field data, and is automatically DMA-transferred by the image capturing device 12. Further, at the same time as the transfer of the odd-numbered field data, the moving image input from the image input device 6 and subjected to the MPEG compression by the image capturing device 12 is transferred to the area 1 and the even-numbered field data is transferred to the area 3.

Next, the non-compressed image analysis means 102 is activated to detect whether or not the scene has changed (step 123). If the capturing of the moving image is continued, steps 122 and 123 are repeatedly executed (step 124). Details of the processing content of the uncompressed image analysis means 102 will be described later.

In step 124, for example, if the capturing of the moving image has been completed by an instruction from the user or the like, the moving image input from the image input device 6 and digitized and MPEG-compressed by the image capturing device 12 (the main storage 2) (Stored in the upper area 1) in the auxiliary storage device 3 (step 125). At this time, the compression performed in the image capturing device 12 is such that the compression of the non-compressed image is simultaneously performed while the processing from step 122 to step 124, that is, the analysis processing of the non-compressed image is performed, The processing speed of the entire control means 101 does not decrease. Further, instead of compressing the uncompressed moving image captured in step 122, the moving image input from the image input device 6 is compressed (this compression processing is performed during the analysis processing of the uncompressed image). Therefore, this is different from the above-described related art in which an uncompressed image is input and then compressed.

Then, the control means 101 activates the correspondence information generation means 103 to generate information for associating the uncompressed image at the time when the scene has changed with the image at the time in the compressed moving image (step 126). Details of the processing contents of the correspondence information generating means 103 will be described later.

Finally, the compressed image reproducing device 104 is started up, and the compressed moving image is reproduced and displayed, for example, from the time pointed by the user (step 127). Returning to step 121, the process is repeated (step 128). In step 128, for example, when there is a termination instruction or the like from the user, the processing of the control unit 101 is terminated. Details of the processing content of the compressed image reproducing device 104 will be described later.

FIG. 4 is a flowchart showing the processing contents of the uncompressed image analysis means 102. A program for realizing this processing is stored in the main memory 2 or the auxiliary storage device 3, and the program is executed by the CPU 1. As shown in FIG. 4, the uncompressed image analysis unit 102 determines the content of the uncompressed image stored in the area 2 in step 122 of the control unit 101 and the current processing in the repetition processing from step 122 to step 124. The content of the uncompressed image previously stored in the area 2 is compared (step 131). At this time, the comparison is performed by comparing the two non-images on a pixel-by-pixel basis, calculating the difference between the two non-images, and calculating the sum of the differences for all the pixels, as in the above-described related art. Then, it is determined whether the scene has changed (step 132). At this time, if the sum of the differences is larger than a predetermined threshold, it can be determined that the scene has changed.

If it is determined in step 132 that the scene has changed, the time from the start of capturing the moving image of the uncompressed image in which the scene has changed is obtained (step 133), and the time is stored in the work area. (Step 134), and the process ends. At this time, the time from the start of capturing the moving image can be obtained as follows. In other words, if the program for driving the above-described image capturing device 12 has a function of returning the number of images captured from the start of moving image capture (including the number of images that could not be captured due to time issues), To get. If the moving image input from the image input device 6 is in the NTSC format, the frame rate is 29.97 frames / second, and the quotient obtained by dividing the number of the images by 30 is the time. If the above-described program for driving the image capturing device 12 does not have a function of returning the number of images captured from the start of moving image capturing, 1 / 29.97 seconds (moving image (In the case of the NTSC format), an interval timer may be started to count the number of captured images.

If it is not determined in step 132 that the scene has changed, the uncompressed image stored in the area 2 before the current processing is deleted from the area 2 (step 135), and the processing ends. A specific example in which the scene changes will be described later.

FIG. 5 is a flowchart showing the processing contents of the correspondence information generating means 103. A program for realizing this processing is stored in the main storage 2 or the auxiliary storage device 3, and the program is executed by the CPUI.

As shown in FIG. 5, the correspondence information generating means 103 first determines whether or not a scene change has occurred during the capture of a series of moving images (step 141), and if not, ends the processing. Whether or not a scene change has occurred is determined by providing a counter in the processing of the uncompressed image analysis means 102 and incrementing the counter when it is determined in step 132 by the uncompressed image analysis means 102 that the scene has changed. Then, by checking whether or not the counter is 1 or more, it can be determined whether or not a scene change exists.

If it is determined in step 141 that there is a scene change, only one time stored in step 134 of the non-compressed image analysis means 102 is obtained from the work area (step 142). An image (position) at the same time as the time in the MPEG compressed moving image stored in the storage device 3 is calculated (step 143). The calculation method at this time will be described with reference to FIG.

FIG. 6A shows, as an example, a part of the non-compressed moving image taken in step 122 in the control means 101 arranged in chronological order, and time progresses from left to right on the page. Assume that That is, of the uncompressed images A1 to A9, A1 is the image that has been captured earliest in time. Also, A1 to A4 indicate information that the object a is moving, and can be said to be the same scene. However, in A4 to A5, the object a disappears from the image, and a new object b appears in the image of A5. This can be said to be a change in the scene. Similarly, the scene changes between A6 and A7. That is, in step 132 in the non-compressed image analysis means 102, it is determined that the scene has changed during the analysis of A5 and A7.

FIG. 6B is a diagram in which a part of the MPEG compressed moving image captured in step 125 of the control means 101 is arranged in time series (original screen order), and time progresses from left to right on the paper. Let's assume that That is, of the compressed images B1 to P2, B1 is the image that is fetched earliest in time and is MPEG-encoded. Further, A1 and B1, A2 and B2, A3 and 11, A4 and B3, A5 and B4, A6 and P1, A7 and B5, A8 and B6, and A9 and P2 are temporally coincident images. Since the non-compressed image and the compressed image are based on the same image input from the image input device 6, for example, the content of A1 matches the content of the image when B1 is decoded. Here, I, P, and B represent I picture, P picture, and B picture in MPEG, respectively. The details of the actual encoding method are described in "Point Illustrated Latest MPEG Textbook", P89-P165, edited by Multimedia Communication Research Group, ASCII Publishing Bureau.

FIG. 6C shows a part of the MPEG compressed moving image taken in step 125 in the control means 101 arranged in the order (bit stream order) actually stored in the auxiliary storage device 3. 6B and 6C indicate the same contents. Accordingly, for example, the first B1 in FIG. 6B and the second B1 in FIG. 6C have the same data. Also assume that 11 is the beginning of a GOP.

Now, assuming that the time obtained in step 142 is the time at which the uncompressed image A5 is captured (this is referred to as T1), since the image at the same time as T1 in the MPEG compressed moving image is B4, step 143 is executed. Then, the position of B4 is calculated. The calculation method is described below.

In the MPEG bit stream, header information is stored at the head of each GOP, and the header information includes the time of the head image of each GOP from the head of the bit stream (this is TC). . Therefore, by examining the TC of each GOP, the GOP including T1 (this is referred to as G1) can be specified. That is, the time of B1 in the MPEG bit stream (G1) in FIG. 6C is TC. The header information includes the frame rate of the moving image (this is referred to as PR). Further, header information is also stored in each image, and the header information includes a frame number from the beginning of a GOP (Group Of Picture: structure for randomly accessing an MPEG bit stream) including each image (this Is TR). Therefore, the result of multiplying PR (PICTURE RATE: representing the display period of the image of the sequence layer) by TR (TEMPORAL REFERENCE: the remainder of 1024 reset at the beginning of the GOP with the consistent picture of the picture layer) is the top of G1. (This is T2). Then, it can be calculated by searching for an image (B4) where TC + T2 = T1. Although the above description has been made with respect to video data, the same can be calculated for audio.

Also, depending on the capability of the compressed image reproducing device 104 described later, reproduction in the middle of a GOP or from a B picture may not be possible. Therefore, instead of B4, the vicinity of the head B1 or 11 of the GOP or the vicinity of B4 may be used. It is also conceivable to use P1 or the like.

Now, returning to the description of FIG. 5, subsequently, the correspondence information generation unit 103 creates the correspondence table 200 shown in FIG. 7 from the various information calculated in step 143 (step 144). If the correspondence table 200 has already been created, an entry is added to the correspondence table 200.

FIG. 7 shows the contents of the correspondence table 200. The correspondence table 200 associates an uncompressed image with a compressed image, and is stored in the main storage 2 or the auxiliary storage device 3. Further, the header information stored at the head of each GOP has an area for storing free information as a user data area, and the correspondence table 200 may be stored in the area.

The correspondence table 200 has items of an ID 210, a pointer 220 to a non-compressed image, a compressed file name 230, a time code 240, and an index 250. For example, for an image immediately after or near a scene change, Is composed of a set of data sets corresponding to When the correspondence table 200 is stored in the user data area, the correspondence table 200 may include at least a pointer 220 to an uncompressed image and an index 250.

An identifier given to each scene change is set in the ID 210, and the identifier is numerical data uniquely given in the image recording / reproducing apparatus. As the pointer 220 to the non-compressed image, a pointer indicating the non-compressed image stored in the main memory 2 (the one stored in step 122 in the control unit 101) is set. The content of the pointer 220 to the non-compressed image may be anything as long as a change in the scene can be specified, such as a comment input by the user. The file name of the compressed moving image stored in the auxiliary storage device 3 is set in the compressed file name 230 in step 125 of the control unit 101. In the time code 240, the time from the head of the bit stream of the first image of the GOP, which is present in the header information of the GOP including the image whose scene has changed, is set. This corresponds to the TC described above. In the index 250, an index (frame number) in the GOP of the image in which the scene has changed is set (the index starts from 0).

{For example, in the example shown in FIG. 6, a correspondence table 200 ′ as shown in FIG. 8 is configured. In FIG. 6A, the data sets 301 and 302 of the images A5 and A7, respectively, immediately after the scene change. In the data set 301, the set value of the pointer 220 to the non-compressed image is the address of the non-compressed image A5 stored in the main memory 2 in the main memory 2, and the data set 302 The set value is the address of the uncompressed image A7 stored in the main memory 2 on the main memory 2.

Returning to the description of FIG. 5, finally, when there is another scene change, the correspondence information generation unit 103 returns to step 142 and repeats the processing (step 145). If there is no other scene change, the process ends. Whether or not another scene change exists may be determined by checking the above-described counter.

FIG. 9 is a flowchart showing the processing contents of the compressed image reproducing apparatus 104. A program for realizing this processing is stored in the main memory 2 or the auxiliary storage device 3, and the program is executed by the CPU 1.

As shown in FIG. 9, first, the compressed image reproducing device 104 displays various information on the display device 5 in a list according to the contents of the correspondence table 200 generated by the correspondence information generating means 103 (step 151). At this time, the content of the ID 210 in the correspondence table 200 and the uncompressed image indicated by the pointer 220 to the uncompressed image are displayed. Also, the compressed file name 230 in the correspondence table 200 may be displayed together. For example, FIG. 19 shows an example of a display on the display device 5 in step 151.

19, reference numeral 400 denotes a display screen of the display device 5, 410 denotes a window for displaying the contents of the correspondence table 200, and 410a to 410i are stored in the main storage 2 indicated by the pointer 220 to the uncompressed image in the correspondence table 200. Uncompressed image. In this case, the correspondence table 200 has at least 9 entries.

Returning to FIG. 9, the compressed image reproducing device 104 waits for an instruction from the user (step 152), and waits for an instruction from the user (step 153). At this time, if there is an instruction from the user (in step 151, an instruction to select one uncompressed image from the uncompressed images (410a to 410i) displayed in a list), the instruction corresponds to the designated uncompressed image. The position (reproduction position) in the compressed moving image can be specified (step 154). This process can be performed as follows.

First, the correspondence table 200 is searched using the designated ID of the non-compressed image (the content of the ID 210 in the correspondence table 200) as a key. As a result, a data set having the ID can be found. Then, by extracting the contents of the compressed file name 230, the contents of the time code 240, and the contents of the index 250 from the data set, the reproduction position can be specified.

Next, the compressed image reproducing device 104 reproduces the compressed moving image, which is the content of the compressed file name 230, from the reproduction position and displays it on the display device 5 (Step 155). Since the method of reproducing the compressed moving image after the reproduction position is determined and the method of displaying the compressed moving image on the display device 5 are already known, detailed description will be omitted. For example, FIG. 20 shows an example of a display on the display device 5 in step 155.

In FIG. 20, reference numeral 420 denotes a window for reproducing and displaying a compressed moving image as the content of the compressed file name 230 from the reproduction position.

Finally, if the processing of the compressed image reproducing device 104 is continued, the process returns to step 151 and repeats the process (step 156). In step 156, for example, when there is a termination instruction or the like from the user, the processing of the compressed image reproducing device 104 is terminated.

As described above, according to the present invention, a scene change or the like can be detected while recording a compressed moving image, and the scene change can be associated with a scene change point in the compressed moving image. A recording / reproducing device can be provided.

Further, according to the present invention, an uncompressed moving image and a compressed moving image can be simultaneously captured (recorded) in the information processing apparatus. Further, as a result of analysis using the non-compressed moving image, An arbitrary image in the compressed moving image corresponding to an arbitrary image can be specified, and furthermore, by correlating these two images, the cue reproduction of the compressed moving image from the position specified by the user or the like can be easily performed. It is possible to provide an image recording / reproducing apparatus that can perform the above.

According to the present invention, it is possible to provide an image recording / reproducing apparatus that takes in a compressed image preferentially when an uncompressed moving image and a compressed moving image are simultaneously taken into an information processing device.

Further, according to the present invention, from a recorded or recorded uncompressed moving image, information related to a compressed moving image is created, and a compressed moving image is reproduced based on the related information, such as a moving image album. An image recording / reproducing system can be provided.

Further, according to the present invention, when capturing an uncompressed moving image into an information processing device, it can be captured separately in odd field data and even field data in an interlaced signal. , Can be transferred to the same or different areas and devices in the same or different size and color format, for example, it is possible to detect a change in the scene while monitoring the input moving image on the display device of the information processing device A simple image recording / reproducing device can be provided.

In the present embodiment, a change in a scene is described as an example of information to be extracted from an uncompressed moving image, but other information may be used. For example, if the user designates image information to be detected in advance and causes the uncompressed image analysis means 102 to recognize an image matching the image information, a simple moving image search system can be constructed.

In the above description, the image capturing device 12 and the compressed image reproducing device 9 are configured as different devices. However, the image capturing device 12 may be configured as the image capturing and reproducing device 13 including the compressed image reproducing device 9. FIG. 21 shows a block configuration of the image capturing / reproducing device 13 at this time.

As shown in FIG. 21, the image capturing / reproducing apparatus 13 includes a moving image input interface unit 50, a frame memory 51, a moving image compression / expansion unit 54, and a video decoder encoder 55.

The video decoder encoder 55 is connected to the image input device 6, and digitizes and outputs an analog moving image input from the image input device 6. For example, if the analog moving image input from the image input device 6 is an NTSC format signal, the video decoder encoder 55 can be realized as an NTSC decoder. In this case, the odd field signal and the even field signal in the interlaced signal of the NTSC format are digitized and output in time series. The output moving image is input to the moving image compression / expansion unit 54 and the moving image input interface unit 50 at the same time.

Further, the video decoder encoder 55 is connected to the image output device 14 implemented by a TV monitor, a VTR, or the like, and the video decoder encoder 55 includes a moving image expanded by the image compression / expansion unit 54 and an information processing device. The data stored in the main memory 2 or the auxiliary storage device 3 or the display device 5 is output to the image output device 14. At this time, if the image output device 14 handles an NTSC format signal, the data to be output to the image output device 14 is converted into an NTSC format signal. Further, the video decoder encoder 55 may be configured to be divided into a video decoder and a video encoder.

The image compression / decompression unit 54 compresses the digitized moving image output from the video decoder encoder 55 by using a compression technique such as MPEG, and outputs the compressed moving image to the moving image input interface unit 50. Further, the compressed moving image transmitted via the moving image input interface unit 50 is decompressed and output to the video decoder encoder 55 and / or output to the video signal input / output unit 64 (see FIG. 16) of the moving image input interface unit 50. I do. The moving image output to the video signal input / output unit 64 of the moving image input interface unit 50 is DMA-transferred to the main memory 2, the auxiliary storage device 3, or the display device 5 as described above.

The frame memory 51 and the moving image input interface unit 50 are the same as those shown in FIG.

Next, a second embodiment will be described with reference to the drawings.

FIG. 1 is a hardware configuration diagram of an information processing apparatus such as a personal computer to which the image recording / reproducing apparatus of the second embodiment can be applied.

As shown in FIG. 1, the device includes a CPU 1, a main memory 2, an auxiliary storage device 3, an input device 4, a display device 5, an image input device 6, an uncompressed image capturing device 7, a compression device It comprises an image capturing device 8 and a compressed image reproducing device 9. The components other than the image input device 6 are connected by a bus 10 so that necessary information can be transmitted between the components.

Further, the image input device 6 is connected to the non-compressed image capturing device 7 and the compressed image capturing device 8, and the same information is simultaneously transmitted from the image input device 6 to the non-compressed image capturing device 7 and the compressed image capturing device 8. It is configured to be able to transmit.

{The functional block diagram of the image recording / reproducing apparatus of the second embodiment is the same as the functional block diagram of the first embodiment shown in FIG.

The uncompressed moving image is fetched from the uncompressed image capturing device 7, and the compressed moving image is fetched from the compressed image capturing device 8. However, both the uncompressed image capturing device 7 and the compressed image capturing device 8 are connected to the image input device 6. Thus, the same (video) information is input, so that the processing of the second embodiment is not essentially different from the processing of the first embodiment.

If the non-compressed image capturing device 7 has a configuration in which the image compression unit 52 is removed from the configuration of the image capturing device 12, the same function as that of the first embodiment can be realized. Further, since the configuration is separated from the compressed image capturing device 8, an image recording / reproducing device having a relatively high degree of freedom in system construction can be provided.

Next, a third embodiment will be described with reference to the drawings.

In the first or second embodiment, the non-compressed image and the compressed image are captured in real time and simultaneously, but only the non-compressed image is captured in real time, and the uncompressed image used for the detection is compressed while detecting a change in the scene. Such a configuration may be adopted. In this case, the hardware configuration of the information processing apparatus to which the image storage device can be applied may be the configuration shown in FIG. 12, and the control means 101 may be changed (control means 101 ') as shown in FIG.

In FIG. 12, the components other than the image compression device 11 are the same as the components shown in FIG. The image compression device is a device for compressing an uncompressed moving image passed from the CPU 1 into the main memory 2 or the auxiliary storage device 3 by, for example, MPEG compression.

As shown in FIG. 13, the control means 101 'first starts capturing a moving image (step 161). Specifically, the respective programs for driving the non-compressed image capturing device 7 are executed, and preparations for acquiring a moving image, such as securing a work area on the main memory 2, are performed.

Next, the control means 101 'acquires an uncompressed image input from the image input device 6 and digitized by the uncompressed image capturing device 7, and stores it in the work area (step 162). Subsequently, the non-compressed image analysis means 102 is activated to detect whether the scene has changed (step 163). Further, the non-compressed image stored in the work area is compressed using the image compression device 11 (step 164). If the capturing of the moving image is continued, steps 162 and 164 are repeatedly executed (step 165).

In step 165, for example, if the capturing of the moving image has been completed by an instruction from the user or the like, the moving image MPEG-compressed by the image compression device 11 is acquired and stored in the work area or the auxiliary storage device 3. (Step 166).

Then, the control means 101 'activates the correspondence information generation means 103 to generate information for associating the uncompressed image at the time when the scene has changed with the image at the time in the compressed moving image (step 167). .

Finally, the compressed image reproducing apparatus 104 is started up, and the compressed moving image is reproduced and displayed from the time pointed by the user, for example (step 168). If the processing of the control means 101 'is continued, Returning to step 161, the process is repeated (step 169). In step 169, for example, when there is a termination instruction or the like from the user, the processing of the control unit 101 'is terminated.

As described above, according to the third embodiment, for example, the data input from the image input device 6 by the uncompressed image capturing device 7 is divided into odd field data and even field data, and the odd field data is mainly converted into a YUV color format. If the data is transferred to the storage 2, the even field data is converted into a color format that can be displayed on the display device 5 and transferred to the display device 5, and the odd field data in the YUV format is input to the image compression device 11 and compressed. Thus, a relatively inexpensive image recording / reproducing apparatus can be provided.

従 来 Conventional technology can be applied to the image compression device 11, and in many cases, the image compression device 11 of the conventional technology should normally receive YUV format data. If the image compression device 11 is realized by software, the function of transferring the data in the YUV color format to the main memory 2 while displaying the uncompressed moving image on the display device 5 is very effective.

According to the present invention, an uncompressed moving image and a compressed moving image can be simultaneously captured (recorded) in the information processing apparatus, and further, as a result of analysis using the uncompressed moving image, Any image in the compressed moving image corresponding to the image can be specified, and by correlating these two images, the cue reproduction of the compressed moving image from the position specified by the user or the like can be easily performed. Thus, an image recording / reproducing device can be provided.

Furthermore, the moving image input from the image input device is processed in real time and simultaneously as an uncompressed moving image and a moving image compressed using a compression technique capable of reproducing at a relatively high compression rate and high image quality. When the compressed moving image is taken into the information processing device, an image recording / reproducing device that takes in the compressed moving image preferentially can be provided.

Further, when the uncompressed moving image is taken into the information processing device, the odd field data and the even field data in the interlace signal input from the image input device are converted into the same or different size and color format. By transferring the image to the same or different area or device, the image recording / reproducing that simultaneously records the moving image while monitoring the moving image input from the image input device on a display device provided in the information processing device. An apparatus can be provided.

FIG. 7 is a hardware configuration diagram of an image recording / reproducing device according to a second embodiment of the present invention. FIG. 2 is a functional block diagram of the image recording / reproducing apparatus according to the first embodiment of the present invention. 5 is a processing flowchart of a control unit according to the first embodiment of the present invention. 5 is a processing flowchart of an uncompressed image analysis unit according to the first embodiment of the present invention. 7 is a processing flowchart of a correspondence information generating unit according to the first embodiment of the present invention. FIG. 4 is an explanatory diagram illustrating a method of generating correspondence information according to the first embodiment of the present invention. FIG. 3 is an explanatory diagram illustrating a data configuration of a correspondence table according to the first embodiment of the present invention. FIG. 3 is an explanatory diagram illustrating a data configuration of a correspondence table according to the first embodiment of the present invention. 3 is a processing flowchart of the compressed image reproducing device according to the first embodiment of the present invention. FIG. 1 is a configuration diagram of an image recording / reproducing device according to a conventional technique. 9 is a processing flowchart of an image recording / reproducing apparatus according to a conventional technique. FIG. 9 is a hardware configuration diagram of an image recording / reproducing device according to a third embodiment of the present invention. 9 is a processing flowchart of a control unit according to a third embodiment of the present invention. FIG. 1 is a hardware configuration diagram of an image recording / reproducing device according to a first embodiment of the present invention. FIG. 2 is a hardware configuration diagram of the image capturing device according to the first embodiment of the present invention. FIG. 2 is a hardware configuration diagram of a moving image input interface unit according to the first embodiment of the present invention. FIG. 2 is a configuration diagram of a register of the image capturing device according to the first embodiment of the present invention. FIG. 4 is an explanatory diagram of an operation of the image capturing device according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating a screen configuration during playback of a compressed moving image according to the first embodiment of the present invention. FIG. 3 is a diagram illustrating a screen configuration during playback of a compressed moving image according to the first embodiment of the present invention. FIG. 2 is a hardware configuration diagram of the image capturing device according to the first embodiment of the present invention. FIG. 1 is a configuration diagram of an image input device including an image capturing device according to a first embodiment of the present invention.

Explanation of reference numerals

1 ... CPU
2 Main memory 3 Auxiliary storage device 4 Input device 5 Display device 6 Image input device 9 Compressed image reproduction device 10 Bus 12 Image capture device

Claims (1)

An input / output unit for inputting a moving image,
A compression unit that compresses a moving image output from the input / output unit,
A register that can separately set a first transfer destination address of the moving image output from the input / output unit and a second transfer destination address of the moving image output from the compression unit;
A moving image output from the input / output unit is transferred to a first external device according to the first transfer destination address, and a moving image output from the compression unit is transferred to a second external device according to the second transfer destination address. An image processing apparatus comprising: a transfer control unit configured to transfer the data to a device.

Images