JP3661696B2 - Image processing device - Google Patents

Description

  The present invention relates to an apparatus that captures, records, and reproduces an input moving image, and more particularly to an apparatus that reproduces a moving image from a free scene (position).

  A conventional image recording / reproducing apparatus has the configuration shown in FIG.

  As shown in FIG. 10, the image recording / reproducing apparatus includes an information processing apparatus 21 and an image input apparatus 22. The image input device 22 is means for inputting an analog moving image (video), and is realized by a video camera, a VTR, a television, or the like. Here, an example of a video camera is shown. The information processing apparatus 21 includes means for digitizing an analog moving image input from the image input device 22 and means for recording a moving image digitized by the means. 22 is a device for recording a moving image input from 22.

  Next, an outline of processing of the conventional image recording / reproducing apparatus having the apparatus configuration shown in FIG. 10 will be described with reference to FIG.

  FIG. 11 is a flowchart of a process for identifying a part where a scene has changed in a series of continuous images (moving images) input from the image input device 22 in order to perform cue reproduction, for example. .

  In this process, first, capturing of a moving image is started (step 501), and an analog image for one frame or one field input from the image input device 22 is digitized (step 502). Here, the moving image input from the image input device 22 has a standardized format such as NTSC, PAL, or SECAM.

  Next, in the above process, the digitized image is stored in the main storage device of the information processing apparatus 21 (step 503). Then, the image is compared with the image stored immediately before it (step 504), and it is determined whether the scene has changed (step 505). At this time, whether or not the scene has changed is determined by comparing the two images pixel by pixel, calculating the difference, and calculating the sum of the differences at all pixels. For example, if the sum exceeds an arbitrary threshold value, it is determined that the scene has changed.

  If it is determined in step 505 that the scene has changed, the time from the start of capturing the moving image of the image is stored in the main memory (step 506). Then, it is determined whether or not the import has been completed (step 507). If not completed, the process returns to step 502 and the process is repeated. If the capturing has been completed, a series of uncompressed moving images are stored in the auxiliary storage device included in the information processing apparatus 21 (step 508), and the processing is terminated.

  Thereafter, the conventional image recording / reproducing apparatus displays a list of images at positions where the scene has changed on the display device included in the information processing device 21 and is currently displaying the list instructed by the user from the input device included in the information processing device 21. The time stored in the main memory in step 506 corresponding to an arbitrary image is extracted, and the moving image saved in the auxiliary storage device in step 508 is searched from the image (position) corresponding to the time. Do it.

  Japanese Patent Laid-Open No. 6-133305 discloses a technique for automatically extracting a scene change of a moving image and encoding the moving image including the information.

  Further, a technique for displaying a moving image on a display device or the like in real time is described in pages 102 to 109 of the April 1996 issue of Interface magazine, published by CQ Publishing Co., Ltd.

  However, in the prior art as shown in FIG. 10, since a change in the scene is detected by capturing a series of uncompressed moving images and stored in the auxiliary storage device without being compressed, it is stored in the auxiliary storage device. The amount of data that is not considered.

  In Japanese Patent Laid-Open No. 6-133305, information on scene change is included in a compressed moving image. Therefore, it is necessary to search the moving image from the beginning to the end each time before reproduction. Not considered.

  In addition, the technology described in the interface magazine does not take into account recording a moving image on a display device and recording the moving image in an auxiliary storage device or the like.

  An object of the present invention is to convert a moving image input from an image input device as a non-compressed moving image and a moving image compressed using a compression technique capable of reproducing at a relatively high compression rate and high image quality. Another object of the present invention is to provide an image capturing device that captures an information processing device at the same time, an image storage / playback device that mounts the image capturing device, and an image input device.

  In addition, from the information obtained based on the non-compressed moving image, the recording and reproduction display of the moving image is performed by associating the arbitrary image in the uncompressed moving image with the arbitrary image in the compressed moving image. It is an object of the present invention to provide an image storage / playback apparatus for performing image recording / playback that enables easy execution of the above.

  Furthermore, when the non-compressed moving image is taken into the information processing device, the other object is that the odd field data and the even field data in the interlace signal input from the image input device have different sizes and color formats. An image capturing device that converts and transfers the image to a different area or device, the image capturing device is mounted, and a moving image input from the image input device is monitored on the display device included in the information processing device, An object of the present invention is to provide an image storage / reproduction device for recording and reproducing moving images.

The present invention relates to a video decoder for inputting an analog moving image and digitizing the inputted moving image in an image capturing device attached to an image recording / reproducing apparatus for recording a moving image having a main memory and a display device. A moving image compression unit that compresses the digitized moving image into a compressed moving image, and simultaneously transfers the digitized moving image output from the video decoder to the main memory and the display device, and simultaneously compresses the moving image. And a moving image input interface unit for transferring a compressed moving image output from the unit to the main memory.

  The present invention relates to an image recording / reproducing apparatus for recording a moving image having a main memory and a display device, which inputs an analog moving image and digitizes the input moving image; A moving image compression unit that compresses a moving image into a compressed moving image; and a compressed moving image that is output from the moving image compression unit at the same time as the digitized moving image output from the video decoder is transferred to the main memory and the display device. A moving image input interface unit for transferring an image to the main memory, wherein the moving image input interface unit separates the digitized moving image into two field signals in an interlaced signal, and transmits one field signal to the main memory. The other field signal is transferred to the display device, and the display device receives the moving image input interface unit. The main memory stores the field signal transferred from the moving image input interface unit at the same time as the display device displays the moving image. Device.

  Further preferably, the image recording / reproducing apparatus has a frame memory for holding at least one screen image of the moving image digitized by the video decoder, and the moving image input interface unit is input from the video decoder. A video signal input / output unit that divides digital moving images into odd field data and even field data and stores them in the frame memory, and a scaling unit that sequentially extracts odd field data and even field data from the frame memory and performs scaling A color conversion unit that converts the data input from the scaling unit into a predetermined color format, a compressed image input / output unit that captures data input according to a predetermined protocol from the moving image compression unit, Input from the color converter The DMA transfer control unit for transferring the data to be compressed and the compressed moving image input from the compressed image input / output unit to the main memory or the display device, and the data to the bus generally used in the information processing device. Or a bus input / output unit that captures data flowing in the bus.

  Preferably, the information extracting unit extracts arbitrary information based on the uncompressed moving image captured by the image capturing unit, the information extracted from the information extracting unit, and the image capturing unit. Correspondence information generating means for associating with the compressed moving image, wherein the information extracting means captures a moving image of an image at an arbitrary time of a series of uncompressed moving images captured from the uncompressed image capturing device. A function of extracting a time from the start, and the correspondence information generating unit extracts an image (position) having the same time as the time from the compressed moving image captured from the compressed image capturing device It has a function to attach.

  Preferably, the image recording / reproducing apparatus further includes an input device for a user to give an instruction and a compressed image reproducing device for reproducing the compressed moving image, and the compressed image reproducing device includes the input device. An image recording / reproducing apparatus having a function of cuing and reproducing the compressed moving image from the associated image or images in the vicinity of the front and back of the image triggered by an instruction input from the means is also conceivable. .

  Preferably, the image capturing device and the compressed image reproducing device are combined.

  For example, the image recording / reproducing apparatus according to the present invention performs the following operations. In response to some instruction input from the input means by the user, the moving image input from the image input device by the image capturing device digitizes and captures the moving image input from the image input device and is input from the image input device The image is digitized and converted into a compressed moving image that can be reproduced with a high compression rate and high image quality. At this time, the two moving images are generated from the same video source and are simultaneously captured in real time.

  For example, when detecting a change in a scene, the information extraction unit includes at least an image immediately after a scene change obtained by analyzing the uncompressed moving image and a moving image capturing start of the image. The correspondence information generating means extracts at least the image (position) having the same time as the time in the compressed moving image captured from the image capturing device, and extracts the two images. Correspondence information for associating is generated.

  Further, since the uncompressed moving image is divided into odd field data and even field data and transferred to the main memory and the display device, the moving image input from the image input device is monitored by the display device. However, it is possible to detect changes in the scene.

  Further, the display device displays at least an image immediately after the scene changes, and when the user instructs the image by the input means, the compressed image reproduction device uses the image and the correspondence information, Reproduction is performed by specifying an image in which the cue reproduction is started in the compressed moving image.

  According to the present invention, an uncompressed moving image and a compressed moving image can be simultaneously captured (recorded) in an information processing apparatus, and further, as a result of analysis using the uncompressed moving image, an arbitrary in the uncompressed moving image An arbitrary image in the compressed moving image corresponding to the image can be identified, and by linking these two images, the cue playback of the compressed moving image from the position designated by the user or the like can be easily performed. An image recording / reproducing apparatus can be provided.

  Further, the moving image input from the image input device is simultaneously converted in real time as a non-compressed moving image and a moving image compressed using a compression technique capable of reproducing at a relatively high compression rate and high image quality. An image recording / reproducing apparatus that preferentially takes in the compressed moving image when being taken into the information processing apparatus can be provided.

  Further, when the uncompressed moving image is taken into the information processing device, the odd field data and the even field data in the interlace signal input from the image input device are converted into the same or different size and color format. Thus, by transferring to an equivalent or different area or device, the moving image input from the image input device is monitored on the display device included in the information processing device, and the moving image is recorded simultaneously. An apparatus can be provided.

  Hereinafter, the best mode for carrying out the present invention will be described.

  A first embodiment according to the present invention will be described below with reference to the drawings.

  FIG. 14 is a hardware configuration diagram of an information processing apparatus such as a personal computer to which the image recording / reproducing apparatus of the first embodiment can be applied.

  As shown in FIG. 14, the device includes a CPU 1, a main memory 2, an auxiliary storage device 3, an input device 4, a display device 5, an image input device 6, an image capturing device 12, and a compressed image reproduction. And a device 9. Each component other than the image input device 6 is connected by a bus 10 so that necessary information can be transmitted between the components.

  Further, the image input device 6 is connected to the image capturing device 12 and configured to transmit video information from the image input device 6 to the image capturing device 12. The image capturing device 12 is mounted on an information processing device 21 such as a PC configured by the CPU 1 connected via a bus, or built in the image input device 6 as shown in FIG. The image capturing device shown in FIG. 22 has the same configuration as the image capturing device 12 shown in FIG. 15 except that the CCD camera 56 is incorporated.

  The main memory 2 functions as a work area and stores necessary programs. The main memory 2 can be realized by a RAM for the former and a ROM for the latter. The auxiliary storage device 3 is means for storing a program for controlling the operation of the device, a compressed moving image, and the like, and can be realized by, for example, a flexible disk, a hard disk, a memory card, a DVD, or the like. The input device 4 is a means for inputting necessary instructions and information, and can be realized by a pointing device such as a keyboard or a mouse, for example. The display device 5 is a means for displaying various information such as reproduction display of a compressed moving image, and can be realized by, for example, a CRT, EL display, plasma display, liquid crystal display, or the like.

  The image input device 6 is a means for inputting an analog moving image (video) similarly to the image input device 22 in the prior art described above, and is realized by a video camera, a VTR, a television, a TV tuner, or the like. The image capturing device 12 is means for digitizing an analog moving image input from the image input device 6 and compressing the digitized moving image. The compression (encoding) means is realized by, for example, MPEG (Moving Picture Experts Group) capable of reproducing with a high compression rate and high image quality. It can also be realized by the compression technique described in "Point Diagram Illustrated Latest MPEG Textbook", P28-P29, edited by Multimedia Communications Research Group, ASCII Publishing Bureau. Details of the MPEG compression technique are described in “Point Diagram-Based Latest MPEG Textbook”, P89-P165, edited by Multimedia Communications Research Group, ASCII Publishing Bureau.

  The compressed image reproduction device 9 is means for reproducing a moving image compressed using the compression technology in the image capturing device 12 and displaying the reproduced image on the display device 5. For example, if the compression technology is MPEG. For example, it can be realized by an MPEG decoder.

  Further, the CPU 1 performs a predetermined operation according to a program stored in advance in the main memory 2 or the auxiliary storage device 3.

  Next, FIG. 15 shows a block diagram of the image capturing device 12.

  As shown in FIG. 15, the image capturing device 12 includes a moving image input interface unit 50, a frame memory 51, a moving image compression unit 52, and a video decoder 53.

  The video decoder 53 is connected to the image input device 6 and digitizes and outputs an analog moving image input from the image input device 6. For example, if the analog moving image input from the image input device 6 is an NTSC format signal, the video decoder 53 can be realized as an NTSC decoder. In this case, the odd field signal and the even field signal in the NTSC format interlace signal are digitized and output in time series. The output moving image is input to the moving image compression unit 52 and the moving image input interface unit 50 at the same time. Note that when the signal output from the image input device 6 to the image capturing device 12 is not an analog signal but a digital signal, the image capturing device 12 does not need the video decoder 53.

  The image compression unit 52 compresses the digitized moving image output from the video decoder 53 using a compression technique such as MPEG. Then, the compressed moving image is output to the moving image input interface unit 50. For example, if the compression technique is MPEG, the compressed moving image is an MPEG stream.

  The frame memory 51 is for holding an image for at least one screen (frame) of the moving image digitized by the video decoder 53, and is connected to the moving image input interface unit 50.

  The moving image input interface unit 50 is connected to the bus 10, stores the digital moving image input from the video decoder 53 once in the frame memory 51, and further stores the image stored in the frame memory 51 through the bus 10. DMA transfer is performed to the storage 2, the auxiliary storage device 3, or the display device 5. At the same time, the compressed moving image input from the image compression unit 52 is also DMA-transferred to the main memory 2 or the auxiliary storage device 3 through the bus 10.

  Next, FIG. 16 shows a block diagram of the moving image input interface unit 50.

  As shown in FIG. 16, the moving image input interface unit 50 includes a bus input / output unit 60, a DMA transfer control unit 61, a color conversion unit 62, a scaling unit 63, a video signal input / output unit 64, and a compressed image input unit. And an output unit 65.

  The bus input / output unit 60 is realized in accordance with a bus protocol such as PCI or ISA that is generally adopted in an information processing apparatus, and flows data to the bus 10 or acquires data flowing to the bus 10.

  The compressed image input / output unit 65 outputs data input from the moving image compression unit 52 according to a predetermined protocol to the DMA transfer control unit 61.

  The video signal input / output unit 64 stores the digitized moving image input from the video decoder 53 in the frame memory 51. When the video decoder 53 is an NTSC decoder, the odd field data and the even field data in the moving image, that is, the interlace signal, are input in time series, and the video decoder 53 determines whether the currently input data is the odd field data. Since a signal indicating even field data is also sent, each field data is separately stored in a predetermined area in the frame memory 51 according to the signal. In addition, when only one screen of data (data combining odd field data and even field data) can be stored in the frame memory 51, each previously stored field data is overwritten and stored.

  The scaling unit 63 sequentially takes out odd field data for one screen and even field data for one screen from the frame memory 51, scales them to a predetermined size, and outputs them to the color conversion unit 62.

  The color conversion unit 62 converts the data input from the scaling unit 63 into a predetermined color format and outputs the data to the DMA transfer control unit 61. Usually, the color format in the image data of the NTSC signal is the YUV format, and the YUV format data is stored in the frame memory 51. However, the color format of data to be transferred to the main memory 2 or the display device 5 is often the RGB format. Therefore, the color converter 62 has a function of converting at least from the YUV format to the RGB format.

  The DMA transfer control unit 61 stores the data input from the color conversion unit 62 and the compressed moving image input from the compressed image input / output unit 65 via the bus input / output unit 60 and the bus 10. 2, DMA transfer to the auxiliary storage device 3 or the display device 5. Further, since the video decoder 53 and the moving image compression unit 52 operate asynchronously, data may be input simultaneously from the color conversion unit 62 and the compressed image input / output unit 65. In this case, the compressed moving image from the compressed image input / output unit 65 is preferentially DMA transferred. This is because even if a part of the compressed moving image is lost, the whole data becomes meaningless.

  FIG. 17 shows an example of a register configuration included in the moving image input interface unit 50.

  The register includes at least an input resolution 301, an output resolution 302, a color format 303, and a transfer destination address 304.

  The input resolution 301 specifies the resolution of data input from the video decoder 53. The output resolution 302 specifies the resolution of data output on the bus 10. A color format 303 specifies the color format of data output on the bus 10. The transfer destination address 304 designates an address that can be addressed by the CPU 1 to which the data output on the bus 10 is finally transferred.

  In addition, each register is individually prepared for the odd field data 310, the even field data 320, and the compressed moving image 330. However, for the compressed moving image 330, only the transfer destination address 304 is provided. The moving image input interface unit 50 operates in accordance with a value set in advance in each register. In addition, it is a program for driving the image capturing device 12 that sets a value in each register. The program is stored in the main memory 2 or the auxiliary storage device 3 and is executed by the CPU 1. The user sets a desired set value in each register through the program.

  Next, setting values and operations of each register in the image capturing device 12 will be described with reference to FIG. Here, a case will be described in which the set values of each register have the contents shown in FIG.

  In FIG. 18, areas 1 and 2 are areas individually secured on the main memory 2. The area 3 is a part on a VRAM (display memory) included in a general display device. Data stored in the VRAM is displayed on a CRT or the like. For example, when a moving image is displayed in a window, the area 3 is an area on the VRAM corresponding to the display area of the window.

  In FIG. 17, the contents of the register of the transfer destination address 304 of the odd field data 310 are the start address of the area 2, the contents of the register of the transfer destination address 304 of the even field data 320 are the start address of the area 3, and the compressed moving image 330. Assuming that the contents of the register of the transfer destination address 304 indicate the start address of the area 1, the odd field data and the compressed moving image are transferred to the main memory 2, and the even field data is transferred to the display device 5. .

  As for the odd field data 310, since the contents of the register of the input resolution 301 are 640 × 240 and the contents of the register of the output resolution 302 are 160 × 120, the scaling unit 63 is 1/4 in the horizontal direction, Reduce to 1/2 in the direction. Further, since the content of the register of the color format 303 is RGB24, the color conversion unit 62 converts the data in the YUV format into the RGB format represented by 8 bits for each of the R, G, and B components. Then, the DMA transfer control unit 61 transfers the scaled and color-converted odd field data to the area 2 indicated by the contents of the register of the transfer destination address 304.

  As for the even field data 320, since the contents of the register of the input resolution 301 are 640 × 240 and the contents of the register of the output resolution 302 are 320 × 240, the scaling unit 63 reduces the horizontal direction by 1/2. To do. It is not enlarged or reduced in the vertical direction. Further, since the content of the register of the color format 303 is RGB8, the color conversion unit 62 converts the data in the YUV format into the RGB format represented by 8 bits in combination with the R, G, and B components. Then, the DMA transfer control unit 61 transfers the scaled and color-converted even field data to the area 3 indicated by the contents of the register of the transfer destination address 304.

  As for the compressed moving image 330, the compressed moving image sent from the moving image compression unit 52 is transferred to the area 1 indicated by the contents of the register of the transfer destination address 304.

  As described above, the image capturing device 12 can divide a moving image input from a video camera, a VTR, or the like into odd field data and even field data and transfer them to different devices and areas in the information processing device. At the same time, data obtained by compressing the input moving image can be transferred to a different device or area in the information processing apparatus. Furthermore, odd field data and even field data can be transferred after being converted into different sizes and color formats. Depending on the setting value of the register of the transfer destination address 304, odd field data and even field data can be transferred to the same device or area. In this case, the input moving image is recorded or displayed as it is. Will be.

  Next, FIG. 2 shows a functional block diagram of the image recording / reproducing apparatus of the first embodiment.

  In FIG. 2, 101 is a control means for controlling each functional block in the image recording / reproducing apparatus of the first embodiment, 102 is an uncompressed image analysis means having a function of analyzing an uncompressed image, and 103 is an uncompressed image. Corresponding information generating means 104 having a function of generating information for associating a compressed image with a compressed image, and 104 is a compressed image reproducing device having a function of reproducing and displaying a compressed moving image. This is realized by the CPU 1 executing a program stored in the storage device 3.

  Next, the operation of the image storage / playback apparatus of the present invention will be described with reference to the drawings.

  Hereinafter, MPEG will be described as a compression technique capable of reproducing with a high compression rate and high image quality, and a scene change will be described as an example of information extracted from a digitized moving image. However, the present invention is not limited thereto.

  FIG. 3 is a flowchart showing the processing contents of the control means 101. A program that realizes this processing is stored in the main memory 2 or the auxiliary storage device 3, and the program is executed by the CPU 1 in response to some event, for example, an instruction from the user input using the input device 4. Executed.

  As shown in FIG. 3, the control unit 101 first starts capturing a moving image (step 121). Specifically, a program for driving the image capturing device 12 is executed, a value input by the user via the input device 4 or the like is set in a register, or a work area is secured on the main memory 2. Prepare to acquire a moving image. Further, as an example to be described, the set value of the register is as shown in FIG. 17, and therefore each data is transferred as shown in FIG. That is, the work area on the main memory 2 includes area 1 and area 2.

  Next, the control means 101 acquires an uncompressed image input from the image input device 6 and digitized by the image capturing device 12, and stores it in the region 2 (step 122). At this time, the data stored in the area 2 is odd field data and is automatically DMA-transferred by the image capturing device 12. Further, simultaneously with the transfer of the odd field data, the moving image input from the image input device 6 and MPEG-compressed by the image capturing device 12 is transferred to the region 1 and the even field data is transferred to the region 3.

  Subsequently, the uncompressed image analysis unit 102 is activated to detect whether the scene has changed (step 123). If the moving image has been captured, step 122 and step 123 are repeatedly executed (step 124). Details of the processing contents of the uncompressed image analysis means 102 will be described later.

  In step 124, for example, if moving image capturing has been completed due to an instruction from the user or the like, the moving image (main memory 2) input from the image input device 6, digitized by the image capturing device 12, and MPEG-compressed. Is stored in the auxiliary storage device 3 (step 125). At this time, the compression performed in the image capturing device 12 is because the compression of the non-compressed image is simultaneously performed while the processing from step 122 to step 124, that is, the analysis processing of the non-compressed image is being performed. The processing speed of the entire control means 101 does not decrease. Further, instead of compressing the uncompressed moving image captured in step 122, the moving image input from the image input device 6 is compressed (this compression processing is performed during the analysis processing of the uncompressed image). Therefore, this is different from the above-described conventional technique in which an uncompressed image is input and then compressed.

  Then, the control unit 101 activates the correspondence information generation unit 103 to generate information that associates the uncompressed image at the time when the scene changes with the image at the time in the compressed moving image (step 126). Details of processing contents of the correspondence information generating unit 103 will be described later.

  Finally, the compressed image reproduction device 104 is activated to display and reproduce the compressed moving image from the time designated by the user (step 127), and if the processing of the control means 101 continues, step Returning to 121, the process is repeated (step 128). In step 128, for example, if there is an end instruction from the user, the processing of the control means 101 is ended. Details of processing contents of the compressed image reproduction apparatus 104 will be described later.

  FIG. 4 is a flowchart showing the processing contents of the uncompressed image analysis unit 102. A program for realizing this processing is stored in the main memory 2 or the auxiliary storage device 3, and the CPU 1 executes the program. As shown in FIG. 4, the uncompressed image analysis unit 102 performs the current processing in the contents of the uncompressed image stored in the area 2 in step 122 in the control unit 101 and the iterative processing from step 122 to step 124. The contents of the uncompressed image previously stored in the area 2 are compared (step 131). At this time, the comparison is performed by comparing the two non-images in pixel units, calculating the difference between them, and obtaining the sum of the differences in all pixels, as in the conventional technique described above. Then, it is determined whether the scene has changed (step 132). At this time, if the sum of the differences is larger than a predetermined threshold value, it can be determined that the scene has changed.

  If it is determined in step 132 that the scene has changed, the time from the moving image capturing start time of the uncompressed image in which the scene has changed is acquired (step 133), and the time is stored in the work area. (Step 134), the process is terminated. At this time, the time from the moving image capturing start time can be acquired as follows. That is, if the program for driving the image capturing device 12 described above has a function of returning the number of images captured from the moving image capturing start point (including the number of images that could not be captured due to a time problem), To get. If the moving image input from the image input device 6 is in the NTSC format, the frame rate is 29.97 frames / second, so the quotient obtained by dividing the number of images by 30 is the time. If the above-described program for driving the image capturing device 12 does not have a function of returning the number of images captured from the start of moving image capture, 1 / 29.97 seconds from the start of moving image capture (moving image (In the case of NTSC format) The interval timer may be activated to count the number of captured images.

  If it is not determined in step 132 that the scene has changed, the uncompressed image stored in area 2 before the current process is deleted from area 2 (step 135), and the process ends. A specific example of the scene change will be described later.

  FIG. 5 is a flowchart showing the processing contents of the correspondence information generating means 103. A program for realizing this processing is stored in the main memory 2 or the auxiliary storage device 3, and the program is executed by the CPUI.

  As shown in FIG. 5, the correspondence information generating unit 103 first determines whether or not there is a scene change by capturing a series of moving images (step 141), and if not, ends the processing. Whether there has been a change in the scene is determined by providing a counter in the processing of the uncompressed image analysis means 102 and incrementing the counter when it is determined in step 132 in the uncompressed image analysis means 102 that the scene has changed. Then, it can be determined whether or not there is a scene change by checking whether the counter is 1 or more.

  If it is determined in step 141 that there is a scene change, only one time stored in step 134 in the uncompressed image analysis means 102 is acquired from the work area (step 142). An image (position) at the same time as that in the MPEG compressed moving image stored in the storage device 3 is calculated (step 143). The calculation method at this time will be described with reference to FIG.

  FIG. 6A shows, as an example, a part of the non-compressed moving images captured in step 122 in the control unit 101 arranged in time series, and time advances from the left to the right of the page. Assume that That is, among the uncompressed images A1 to A9, A1 is an image taken in earliest in time. Further, A1 to A4 indicate information that the object a is moving, and can be said to be the same scene. However, in A4 to A5, the object a disappears from the image, and a new object b appears in the image of A5. This can be said to be a scene change. Similarly, the scene changes between A6 and A7. That is, in step 132 in the non-compressed image analysis unit 102, it is determined that the scene has changed while the analysis of A5 and A7 is being performed.

  FIG. 6B shows a part of the MPEG compressed moving images captured in step 125 in the control means 101 arranged in time series (original screen order), and time advances from the left to the right of the page. Assume that That is, among the compressed images B1 to P2, B1 is an image that is taken in earliest and is MPEG-encoded. Furthermore, A1 and B1, A2 and B2, A3 and 11, A4 and B3, A5 and B4, A6 and P1, A7 and B5, A8 and B6, and A9 and P2 are temporally coincident images. Since the uncompressed image and the compressed image are based on the same image input from the image input device 6, for example, the content of A1 matches the content of the image when B1 is decoded. Here, I, P, and B represent I picture, P picture, and B picture in MPEG, respectively. Details of the actual coding method are described in “Point Graphical Newest MPEG Textbook”, P89 to P165, edited by Multimedia Communications Research Group, ASCII Publishing Bureau.

  FIG. 6C shows a part of the MPEG compressed moving images captured in step 125 in the control means 101 arranged in the order actually stored in the auxiliary storage device 3 (bit stream order). 6 (b) and FIG. 6 (c) indicate the same contents. Accordingly, for example, the first B1 in FIG. 6B and the second B1 in FIG. 6C are data having the same contents. Also, 11 is assumed to be the head of the GOP.

  If the time acquired in step 142 is the time when the uncompressed image A5 is taken in (this is T1), the image at the same time as T1 in the MPEG compressed moving image is B4. Then, the position of B4 is calculated. The calculation method is shown below.

  In the MPEG bit stream, header information is stored at the head of each GOP, and the header information includes the time (referred to as TC) from the head of the bit stream of the head image of each GOP. . Therefore, by examining the TC of each GOP, it is possible to identify the GOP that includes T1 (this is referred to as G1). That is, the time B1 in the MPEG bit stream (G1) in FIG. 6C is TC. The header information includes the frame rate of the moving image (this is referred to as PR). Furthermore, header information is also stored in each image, and the header information includes a frame number from the top of a GOP (Group Of Picture: structure for randomly accessing an MPEG bit stream) including each image (this is the header information). Is TR). Therefore, the result of multiplying PR (PICTURE RATE: represents the display cycle of the image in the sequence layer) and TR (TEMPORAL REFERENCE: the remainder of 1024 that is reset at the beginning of the GOP with the picture consistent picture number in the picture layer) is the head of G1 It is the time from (this is T2). It can be calculated by searching for an image (B4) where TC + T2 = T1. Although the above has been described with respect to video data, it can be similarly calculated for audio.

  Further, depending on the capability of the compressed image playback apparatus 104 described later, there are some that cannot be played back in the middle of a GOP or from a B picture. Therefore, instead of B4, B1 or 11 at the head of the GOP, or the vicinity of B4 It is also conceivable to use P1 or the like.

  Now, returning to the description of FIG. 5, the correspondence information generating unit 103 subsequently creates the correspondence table 200 shown in FIG. 7 from various information calculated in Step 143 (Step 144). If the correspondence table 200 has already been created, an entry is added to the correspondence table 200.

  FIG. 7 shows the contents of the correspondence table 200. The correspondence table 200 associates an uncompressed image with a compressed image, and is stored in the main memory 2 or the auxiliary storage device 3. Furthermore, the header information stored at the head of each GOP includes an area for storing free information as a user data area, and the correspondence table 200 may be stored in the area.

  The correspondence table 200 includes items of ID 210, a pointer 220 to an uncompressed image, a compressed file name 230, a time code 240, and an index 250. It consists of a collection of data sets corresponding to. When the correspondence table 200 is stored in the user data area, the correspondence table 200 may be configured with at least a pointer 220 and an index 250 to an uncompressed image.

  In the ID 210, an identifier given to each scene change is set, and the identifier is numeric data uniquely given in the image recording / reproducing apparatus. In the pointer 220 to the uncompressed image, a pointer that points to the uncompressed image stored in the main memory 2 (stored in step 122 in the control means 101) is set. Further, the content of the pointer 220 to the uncompressed image may be anything as long as the change of the scene can be specified, such as a comment input by the user. In the compressed file name 230, the file name of the compressed moving image stored in the auxiliary storage device 3 in step 125 in the control means 101 is set. In the time code 240, the time from the head of the bit stream of the head image of the GOP, which is present in the header information of the GOP including the image whose scene has changed, is set. This corresponds to the TC described above. In the index 250, an index (frame number) in the GOP of an image in which the scene has changed is set (the index starts from 0).

  For example, in the example shown in FIG. 6, a correspondence table 200 ′ as shown in FIG. 8 is configured. In FIG. 6A, the data sets of images A5 and A7 immediately after the scene change are 301 and 302, respectively. In the data set 301, the setting value of the pointer 220 to the uncompressed image is the address on the main memory 2 of the uncompressed image A5 stored in the main memory 2, and the setting value of the pointer 220 to the uncompressed image in the data set 302. The set value is an address on the main memory 2 of the uncompressed image A7 stored in the main memory 2.

  Returning to the explanation of FIG. 5, finally, the correspondence information generating means 103 returns to step 142 and repeats the process when there is another scene change (step 145). If there is no other scene change, the process ends. Whether or not there is another scene change may be determined by checking the counter described above.

  FIG. 9 is a flowchart showing the processing contents of the compressed image reproduction device 104. A program for realizing this processing is stored in the main memory 2 or the auxiliary storage device 3, and the CPU 1 executes the program.

  As shown in FIG. 9, the compressed image reproduction device 104 first displays a list of various types of information on the display device 5 in accordance with the contents of the correspondence table 200 generated by the correspondence information generation means 103 (step 151). At this time, the contents of ID 210 in the correspondence table 200 and the uncompressed image indicated by the pointer 220 to the uncompressed image are displayed. Further, the compressed file name 230 in the correspondence table 200 may be displayed together. For example, an example displayed on the display device 5 in step 151 is shown in FIG.

  In FIG. 19, 400 is a display screen of the display device 5, 410 is a window for displaying the contents of the correspondence table 200, and 410 a to 410 i are stored in the main memory 2 pointed to by the pointer 220 to the uncompressed image in the correspondence table 200. It is an uncompressed image. In this case, there are at least nine entries in the correspondence table 200.

  Returning to FIG. 9, the compressed image reproduction apparatus 104 waits for an instruction from the user (step 152), and waits for an instruction from the user (step 153). At this time, if there is an instruction from the user (instruction to select one uncompressed image from the list-displayed uncompressed images (410a to 410i in step 151)), it corresponds to the instructed uncompressed image. The position (reproduction position) in the compressed moving image can be specified (step 154). This process can be performed as follows.

  First, the correspondence table 200 is searched using the instructed ID of the uncompressed image (the content of the ID 210 in the correspondence table 200) as a key. As a result, a data set having the ID can be found. Then, the playback position can be specified by extracting the contents of the compressed file name 230, the contents of the time code 240, and the contents of the index 250 from the data set.

  Next, the compressed image reproduction device 104 reproduces the compressed moving image as the content of the compressed file name 230 from the reproduction position and displays it on the display device 5 (step 155). Since the playback of the compressed video after the playback position is determined and the display method on the display device 5 are already known, detailed description thereof is omitted. For example, an example displayed on the display device 5 in step 155 is shown in FIG.

  In FIG. 20, reference numeral 420 denotes a window for reproducing and displaying a compressed moving image as the content of the compressed file name 230 from the reproduction position.

  Finally, if the process of the compressed image reproduction apparatus 104 continues, the process returns to step 151 and is repeated (step 156). In step 156, for example, when there is an end instruction from the user, the processing of the compressed image reproduction device 104 is ended.

  As described above, according to the present invention, a scene change or the like can be detected while recording a compressed moving image, and the image can be easily recorded and reproduced because it is associated with a scene change point in the compressed moving image. A recording / reproducing apparatus can be provided.

  Further, according to the present invention, the uncompressed moving image and the compressed moving image can be simultaneously captured (recorded) in the information processing apparatus, and further, as a result of analysis using the uncompressed moving image, An arbitrary image in a compressed moving image corresponding to an arbitrary image can be specified, and by linking these two images, cueing reproduction of the compressed moving image from a position designated by a user or the like can be easily performed. It is possible to provide an image recording / reproducing apparatus that can perform the operation.

  In addition, according to the present invention, it is possible to provide an image recording / reproducing apparatus that preferentially takes in a compressed image when the non-compressed moving image and the compressed moving image are simultaneously taken into the information processing apparatus.

  Further, according to the present invention, information related to a compressed moving image is created from a non-compressed moving image being recorded or recorded, and a compressed moving image is reproduced based on the related information. An image recording / reproducing system can be provided.

  In addition, according to the present invention, when an uncompressed moving image is captured into an information processing device, it can be separately captured into odd field data and even field data in an interlaced signal. Because it can be transferred to the same or different area and device in the same or different size and color format, for example, it is possible to detect scene changes while monitoring the input moving image on the display device of the information processing device An image recording / reproducing apparatus can be provided.

  In the present embodiment, a scene change is taken as an example of information extracted from an uncompressed moving image, but other information may be used. For example, if image information that the user wants to detect is specified in advance and the uncompressed image analysis unit 102 is made to recognize an image that matches the image information, a simple moving image search system can be constructed.

  In the above description, the image capturing device 12 and the compressed image reproducing device 9 are configured as different devices. However, the compressed image reproducing device 9 may be configured as an image capturing and reproducing device 13 included in the image capturing device 12. FIG. 21 shows a block configuration of the image capturing / reproducing apparatus 13 at this time.

  As shown in FIG. 21, the image capturing / playback apparatus 13 includes a moving image input interface unit 50, a frame memory 51, a moving image compression / decompression unit 54, and a video decoder encoder 55.

  The video decoder encoder 55 is connected to the image input device 6 and digitizes and outputs an analog moving image input from the image input device 6. For example, if the analog moving image input from the image input device 6 is an NTSC format signal, the video decoder encoder 55 can be realized as an NTSC decoder. In this case, the odd field signal and the even field signal in the NTSC format interlace signal are digitized and output in time series. The output moving image is simultaneously input to the moving image compression / decompression unit 54 and the moving image input interface unit 50.

  Further, the video decoder encoder 55 is connected to an image output device 14 realized by a TV monitor, a VTR, or the like. The video decoder encoder 55 is a moving image expanded by the image compression / decompression unit 54 and an information processing apparatus. The data held in the main memory 2, auxiliary storage device 3, or display device 5 is output to the image output device 14. At this time, if the image output device 14 handles NTSC format signals, the data to be output to the image output device 14 is converted to NTSC format signals. The video decoder encoder 55 may be divided into a video decoder and a video encoder.

  The image compression / decompression unit 54 compresses the digitized moving image output from the video decoder encoder 55 using a compression technique such as MPEG, and outputs the compressed moving image to the moving image input interface unit 50. Further, the compressed moving image sent via the moving image input interface unit 50 is decompressed and output to the video decoder encoder 55 and / or output to the video signal input / output unit 64 of the moving image input interface unit 50 (see FIG. 16). To do. The moving image output to the video signal input / output unit 64 of the moving image input interface unit 50 is DMA-transferred to the main memory 2, the auxiliary storage device 3, or the display device 5 as described above.

  The frame memory 51 and the moving image input interface unit 50 are the same as those shown in FIG.

  Next, Example 2 will be described with reference to the drawings.

  FIG. 1 is a hardware configuration diagram of an information processing apparatus such as a personal computer to which the image recording / reproducing apparatus of the second embodiment can be applied.

  As shown in FIG. 1, the apparatus includes a CPU 1, a main memory 2, an auxiliary storage device 3, an input device 4, a display device 5, an image input device 6, an uncompressed image capturing device 7, and a compression. An image capturing device 8 and a compressed image reproducing device 9 are included. Each component other than the image input device 6 is connected by a bus 10 so that necessary information can be transmitted between the components.

  The image input device 6 is connected to an uncompressed image capturing device 7 and a compressed image capturing device 8, and the same information is simultaneously transmitted from the image input device 6 to the uncompressed image capturing device 7 and the compressed image capturing device 8. It is configured to be able to transmit.

  The functional block diagram of the image recording / reproducing apparatus of the second embodiment is the same as the functional block diagram of the first embodiment shown in FIG.

  The uncompressed moving image is captured from the uncompressed image capturing device 7 and the compressed moving image is captured from the compressed image capturing device 8. Both the uncompressed image capturing device 7 and the compressed image capturing device 8 are connected to the image input device 6. Therefore, since the same (video) information is input, the processing of the second embodiment is not essentially different from the processing of the first embodiment.

  If the uncompressed image capturing device 7 has a configuration in which the image compression unit 52 is removed from the configuration of the image capturing device 12, a function equivalent to that of the first embodiment can be realized. Further, since the configuration is separated from the compressed image capturing device 8, it is possible to provide an image recording / reproducing device having a relatively high degree of freedom in system construction.

  Next, Example 3 will be described with reference to the drawings.

  In the first or second embodiment, the uncompressed image and the compressed image are captured simultaneously in real time, but only the uncompressed image is captured in real time, and the uncompressed image used for detection is compressed while detecting a change in the scene. Such a configuration may be adopted. In this case, the hardware configuration of the information processing apparatus to which the image storage device can be applied is configured as shown in FIG. 12, and the control means 101 may be changed (control means 101 ′) as shown in FIG.

  In FIG. 12, the components other than the image compression device 11 are the same as those shown in FIG. The image compression apparatus is an apparatus for compressing an uncompressed moving image delivered from the CPU 1 and storing the uncompressed moving image in the main memory 2 or the auxiliary storage device 3, for example, by MPEG compression.

  As shown in FIG. 13, the control unit 101 ′ first starts capturing a moving image (step 161). Specifically, each program for driving the non-compressed image capturing device 7 is executed, and preparation for acquiring a moving image is performed, such as securing a work area on the main memory 2.

  Next, the control means 101 ′ acquires an uncompressed image input from the image input device 6 and digitized by the uncompressed image capturing device 7, and stores it in the work area (step 162). Subsequently, the uncompressed image analysis means 102 is activated to detect whether the scene has changed (step 163). Further, the uncompressed image stored in the work area is compressed using the image compression device 11 (step 164). If the moving image has been captured, step 162 and step 164 are repeatedly executed (step 165).

  In step 165, for example, if the moving image has been captured by an instruction from the user, the moving image compressed by the MPEG in the image compression device 11 is acquired and stored in the work area or the auxiliary storage device 3. (Step 166).

  Then, the control unit 101 ′ activates the correspondence information generation unit 103, and generates information that associates the uncompressed image at the time when the scene changes with the image at the time in the compressed moving image (step 167). .

  Finally, the compressed image playback device 104 is activated to perform playback display of the compressed video from the time designated by the user (step 168), and if the processing of the control means 101 ′ is continued, Returning to step 161, the processing is repeated (step 169). In step 169, for example, when there is an end instruction from the user, the process of the control means 101 ′ is ended.

  As described above, according to the third embodiment, for example, the data input from the image input device 6 by the uncompressed image capturing device 7 is divided into odd field data and even field data, and the odd field data is mainly displayed in the YUV color format. If it is transferred to the memory 2, the even field data is converted into a color format that can be displayed on the display device 5 and transferred to the display device 5, and the odd field data in YUV format is further input to the image compression device 11 and compressed. A relatively inexpensive image recording / reproducing apparatus can be provided.

  Conventional techniques can be applied to the image compression apparatus 11, and many of the conventional image compression apparatuses 11 should normally receive YUV format data. If the image compression device 11 is realized by software, the function of transferring data in the YUV format color format to the main memory 2 while displaying an uncompressed moving image on the display device 5 is very effective.

  According to the present invention, an uncompressed moving image and a compressed moving image can be simultaneously captured (recorded) in an information processing apparatus, and further, as a result of analysis using the uncompressed moving image, an arbitrary in the uncompressed moving image An arbitrary image in the compressed moving image corresponding to the image can be identified, and by linking these two images, the cue playback of the compressed moving image from the position designated by the user or the like can be easily performed. An image recording / reproducing apparatus can be provided.

  Further, the moving image input from the image input device is simultaneously converted in real time as a non-compressed moving image and a moving image compressed using a compression technique capable of reproducing at a relatively high compression rate and high image quality. An image recording / reproducing apparatus that preferentially takes in the compressed moving image when being taken into the information processing apparatus can be provided.

  Further, when the uncompressed moving image is taken into the information processing device, the odd field data and the even field data in the interlace signal input from the image input device are converted into the same or different size and color format. Thus, by transferring to an equivalent or different area or device, the moving image input from the image input device is monitored on the display device included in the information processing device, and the moving image is recorded simultaneously. An apparatus can be provided.

The hardware block diagram of the image recording / reproducing apparatus in Example 2 of this invention. 1 is a functional block diagram of an image recording / reproducing apparatus in Embodiment 1 of the present invention. The processing flowchart of the control means in Example 1 of this invention. 5 is a processing flowchart of uncompressed image analysis means in Embodiment 1 of the present invention. The processing flowchart of the corresponding | compatible information production | generation means in Example 1 of this invention. Explanatory drawing which shows the production | generation method of the correspondence information in Example 1 of this invention. Explanatory drawing which shows the data structure of the corresponding | compatible table in Example 1 of this invention. Explanatory drawing which shows the data structure of the corresponding | compatible table in Example 1 of this invention. 3 is a process flowchart of the compressed image reproduction device according to the first embodiment of the present invention. The block diagram of the image recording / reproducing apparatus in a prior art. 10 is a processing flowchart of an image recording / reproducing apparatus according to the prior art. The hardware block diagram of the image recording / reproducing apparatus in Example 3 of this invention. The processing flowchart of the control means in Example 3 of the present invention. 1 is a hardware configuration diagram of an image recording / reproducing apparatus in Embodiment 1 of the present invention. 1 is a hardware configuration diagram of an image capturing device according to Embodiment 1 of the present invention. The hardware block diagram of the moving image input interface part in Example 1 of this invention. 1 is a configuration diagram of a register of an image capturing device in Embodiment 1 of the present invention. Explanatory drawing of operation | movement of the image capturing apparatus in Example 1 of this invention. FIG. 3 is a screen configuration diagram at the time of reproduction of a compressed moving image according to the first embodiment of the present invention. FIG. 3 is a screen configuration diagram at the time of reproduction of a compressed moving image according to the first embodiment of the present invention. 1 is a hardware configuration diagram of an image capturing device according to Embodiment 1 of the present invention. 1 is a configuration diagram of an image input device including an image capturing device according to Embodiment 1 of the present invention.

Explanation of symbols

1 ... CPU
2 ... main memory 3 ... auxiliary storage device 4 ... input device 5 ... display device 6 ... image input device 9 ... compressed image reproduction device 10 ... bus 12 ... image capturing device

Claims (1)

An input / output unit for inputting moving images;
A compression unit that compresses the moving image output from the input / output unit;
A register capable of separately setting a first transfer destination address of a moving image output from the input / output unit and a second transfer destination address of a moving image output from the compression unit;
The moving image output from the input / output unit according to the first transfer destination address is transferred to a first external device, and the moving image output from the compression unit according to the second transfer destination address is transferred to a second external device. An image processing apparatus comprising: a transfer control unit that transfers to the apparatus.

Images