JP4859974B2 - Information processing apparatus, print system, and image composition method - Google Patents

Description

  The present invention relates to a technique for creating one piece of print data from moving image data.

  Conventionally, various techniques for creating one still image from moving image data have been developed. For example, according to Patent Document 1, each data of a print target image in a moving image and a reference image one second before and one second after the print target image is taken into a frame memory in frame units. The matching processing unit performs local matching processing on the print target image data and the reference image data, and divides the print target image into a still area and a moving area. The still area weighted average unit weights and averages pixel data in the same position of the print target image and the reference image with respect to the still area. The moving region smoothing processing unit performs smoothing processing on the moving region using the image data of the print target image. The still image and moving region image data subjected to these processes are combined by the image combining unit.

  According to Patent Document 2, the detection angle of the angle detection unit is associated with each shooting cut and stored in the shooting angle storage unit. At the time of printing, panoramic shot image data including a plurality of shooting cuts is stored in the shooting angle storage unit. When outputting data of a plurality of shooting cuts identified as panoramic shooting specified from the shooting angle information, the shooting data reading means configures and outputs the composition processing of the joining shot portions of adjacent shooting cuts.

JP-A-10-83033 JP-A-11-331695

  By the way, if images with different subject sizes, movement directions, and inclinations are combined, there is a possibility that the viewer will feel uncomfortable (see FIG. 10). In addition, the combined shooting cut does not cover the entire range to be printed, and there are many portions where the image does not exist, resulting in poor appearance (see FIG. 11). The present invention has been made in view of such problems, and an object of the present invention is to provide a technique capable of obtaining one piece of print data having good appearance from moving image data.

  In order to solve the above problems, the present invention provides an imaging unit that outputs an image signal based on a subject image received through an imaging lens, and an image storage unit that stores a moving image based on an image signal output from the imaging unit. A state information detection unit that detects state information that is information related to a state of the digital camera at the time of shooting each frame of the moving image, and an accessory that includes at least state information at the time of shooting each frame of the moving image And an attached information storage unit that stores information in association with each frame of the moving image.

  According to the present invention, each frame of a moving image is associated with attached information including state information at the time of shooting. As a result, the print system described later can arrange images suitable for printing by referring to the attached information when creating one print image from each frame.

  Here, the state information detected by the state information detection unit includes the vertical and horizontal movement directions and amount of movement of the camera body, the inclination direction and amount of inclination of the camera body, or the zoom magnification of the subject image.

  The attached information may further include information that defines the association between the frame and a desired still image.

  In this way, for example, by arranging a still image associated with a frame in the vicinity of the frame, an unnatural blank can be prevented from occurring.

  The state information detection unit is preferably provided on the pan head.

  The state information detection unit may be provided in the camera body, but it is preferable to provide the state information detection unit on the pan head installed on the ground because the detection accuracy of the state information is improved.

  In order to solve the above problems, a print system according to the present invention includes a state information acquisition unit that acquires state information, which is information about the state of the digital camera at the time of shooting each frame of a moving image, from a predetermined storage medium; A determining unit that determines an arrangement position of the frame within a predetermined print range based on the state information, a combining unit that arranges the frame at the determining position determined by the determining unit and combines the frames into a single print image; A printer that prints the combined print image.

  Here, the state information includes the up / down / left / right movement direction and amount of movement of the camera body, the determination unit determines the frame arrangement path based on the up / down / left / right movement direction and movement amount of the camera body, and the combining unit Arrange each frame along the arrangement path.

  By arranging the frames in accordance with the movement of the camera, it is possible to create a single printed image with a sense of reality that matches the state at the time of shooting.

  The synthesizing unit may select and arrange frames in which overlapping areas are less than a predetermined amount among frames arranged along the arrangement route.

  In this way, it is possible to prevent waste of overlapping images having many overlapping portions.

  The state information includes the tilt direction and tilt amount of the camera body, and the combining unit corrects the tilt of the frame arranged based on the tilt direction and tilt amount of the camera body.

  In this way, the tilt of the subject caused by the tilt of the camera can be corrected.

  The state information includes the zoom magnification of the subject image, and the combining unit may scale each frame based on the zoom magnification of the subject image so that the magnification of the subject image of each arranged frame is unified.

  The attached information may further include related information that defines the association between the frame and a desired still image, and the combining unit may combine and synthesize a still image associated with the frame based on the related information in the vicinity of the arranged frame.

  By doing so, it is possible to prevent an unnatural blank from occurring in the printed image.

  The camera system according to the present invention associates each frame of a moving image with attached information including state information at the time of shooting. As a result, the print system according to the present invention can arrange the images suitable for printing with reference to the attached information when creating one print image from each frame.

1 is an external perspective view of a camera system according to the present invention. Block diagram of the camera system Block diagram of a printer according to the present invention Flow chart showing the flow of printing operation The figure which shows an example of the arrangement | sequence route of a frame image The figure which shows an example of the frame image selected according to the magnitude | size of an overlapping area The figure which shows an example of the print image obtained by arrange | positioning a frame image along an arrangement | sequence route The figure which shows an example of the print image obtained by arrange | positioning the frame image enlarged / reduced according to zoom magnification The figure which shows an example of the print image obtained by arrange | positioning the frame image by which inclination was corrected according to inclination movement information The figure which shows the prior art example which synthesize | combined several frame images The figure which shows the other conventional example which synthesize | combined several frame images

  Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings.

  As shown in FIG. 1, the camera system according to the present invention is configured by fixing a camera 100 to a tripod 140 with a pan head. The user sets the tripod 140 with the camera platform 200 with the camera 100 roughly facing the direction of the subject, and then moves the lever 140a provided on the tripod 140 up and down, left and right, or rotates it horizontally. Shooting that follows the subject is possible.

  FIG. 2 is a block diagram of the camera 100. The camera 100 includes a CPU 10, an imaging lens 11, a drive unit 12 that drives the imaging lens 11, a lens controller 13 that sends commands such as zoom demand and focus demand to the drive unit 12, and a solid-state imaging device such as a CCD (hereinafter referred to as CCD). 14, a memory controller 15 that controls storage of the image signal output from the CCD 14 in the built-in memory 16 or the memory card 17, a compression unit 18 that compresses captured image data, a setting of a shooting mode, and a start / An operation unit 19 that receives a manual instruction for termination, a communication unit 20 that performs communication with the pan head 200, and the like are provided.

  For example, an optical double zoom lens is used as the imaging lens 11, and the drive unit 12 moves the imaging lens 11 to the telephoto (tele) side or the wide angle (wide) side according to the zoom magnification change operation designated from the operation unit 19. The optical zoom magnification is changed by driving back and forth. The magnification of the imaging lens 11 is not limited to the above.

  The memory controller 15 as a recording device control unit receives a command from the CPU 10 and controls the memory card 17 attached to the digital camera 100, and “JPEG image data” compressed by the compression unit 18 using the JPEG compression method, R , G, B signals and uncompressed “RGB image data”, and Y, C signals, “YC image data” can be recorded (written) in the memory 16. These data formats are used in the still image shooting mode. In the moving image shooting mode, the image data of each frame (frame) constituting the captured moving image data is compressed by the compression unit 18, and the moving image data is stored in the memory 16 as a motion JPEG format in which independent JPEG images are arranged. To be recorded. Note that the compression format of the image data is not limited to JPEG.

  The image data in the memory 16 is added to the electronic zoom circuit 5 as necessary. The electronic zoom circuit 5 performs image enlargement / reduction processing based on the electronic zoom magnification designated by the operation unit 19. That is, a part of the image data is cut out, and the cutout range is enlarged / reduced to perform image enlargement / reduction processing. The enlarged / reduced image data is output to the memory 16. The cut-out range can be set at an arbitrary position on the image by operating the operation unit 19 such as a cross button, whereby the desired range of the image data can be enlarged / reduced.

  The camera platform 200 includes a motion detection circuit 21 that detects the motion of the camera 100 and sends the detected motion as motion information to the camera 100 via the communication unit 22.

  The motion detection circuit 21 includes a potentiometer, a gravity sensor, a tilt sensor, a posture sensor, an electronic compass, and the like. Specifically, the motion detection circuit 21 outputs a left-right motion information indicating the left-right motion direction and the amount of motion of the camera 100, and an up-down direction indicating the up-and-down motion direction and the amount of motion of the camera 100. A vertical motion detection unit 21b that outputs motion information and a tilt motion detection unit 21c that outputs tilt motion information indicating the tilt direction and the tilt amount of the housing of the camera 100 with respect to the horizontal direction are provided. The motion information includes left / right motion information, up / down motion information, and tilt motion information.

  The motion detection circuit 21 may be provided in the camera 100 itself. However, in order to obtain accurate motion information, it is preferable to provide the motion detection circuit 21 on the tripod 140 with the pan head 200 placed on the ground, rather than the camera 100 held by hand.

  The CPU 10 obtains the motion information obtained at the time when each frame constituting the moving image data is captured or designated at that time, the position of the optical axis of the lens 11, the electronic zoom magnification or the optical zoom magnification (summarizing these). The zoom magnification is stored in the memory card 17 in association with each frame image taken at that time.

  FIG. 3 is a block diagram showing the configuration of the printer 300 according to the present invention. As shown in the figure, the printer 300 mainly includes a recording medium insertion port 33, a media interface 32, a memory 34, a system memory 36, a touch panel monitor 39, an input controller 37, a display controller 38, a CPU 31, a print engine 35, and a bus 30. It is composed of

  The printer 300 has a recording medium insertion slot 33 into which the memory card 17 used in the camera 100 is inserted, and can capture moving image data from the recording medium inserted into the recording medium insertion slot 33. It has become.

  After the memory card 17 is inserted into the recording medium insertion slot 33, the moving image file recorded on the memory card 17 is sent to a necessary location via the media interface 32 and the bus 30 in accordance with instructions from the CPU 31. The memory 34 temporarily stores image data in order to generate display image data and print image data. The system memory 36 includes a ROM and the like and is a medium for storing system files. The touch panel monitor 39 has an operation unit and a display screen, and is displayed by a display controller 38. When the operation unit of the touch panel monitor 39 is operated, the input controller 37 is activated to perform input. In addition to performing various controls, the CPU 31 decompresses image data that has been compressed and recorded in the motion JPEG format. The print engine 35 controls the printing drive by the printer.

  Since this embodiment is a case where a moving image is recorded in a motion JPEG format shot with a digital camera, the moving image can be reproduced by switching the still image on the touch panel monitor 39 like a picture-story show.

  Hereinafter, the flow of the printing operation of the printer 300 according to the present invention will be described with reference to the flowchart of FIG.

  First, in S <b> 1, the CPU 31 reads moving image data and attached information from the memory card 17 inserted into the media insertion slot 33 into the memory 34.

  In S <b> 2, the CPU 31 determines an arrangement route of the frame image within the print range based on the movement path of the camera optical axis included in the attached information. As an example, this arrangement path appropriately enlarges / reduces the movement path of the camera optical axis so that the start point and the end point of the movement path of the camera optical axis are arranged at the start point P0 and the end point P1 of the arrangement path existing at a predetermined position in the print range. Can be determined.

FIG. 5 shows an example of the arrangement route in the print range. As shown in this figure, the arrangement route is arranged so as to pass through the inside of the outer peripheral edge X of the print range. However, the method of determining the arrangement path of the frame image is arbitrary, and may be determined based on the locus drawn by the light spot of the specific subject image, or may be arbitrarily specified by the user by operating the touch panel monitor 39. Also good.

  In S3, a frame image to be arranged along the arrangement route determined in S2 is selected, and the selected frame image is arranged along the arrangement route. The method of selecting the image to be arranged is arbitrary. For example, it may be arranged for each frame image separated by a predetermined time. Alternatively, as shown in FIG. 6, among the plurality of frame images F <b> 2 to F <b> 4 photographed at the photographing time close to the photographing time of a certain frame image F <b> 1, the frame image F <b> 4 having the smallest overlapping area with F <b> 1 may be arranged.

  In S4, a plurality of frame images arranged along the arrangement route are combined into one print image (see FIG. 7).

  In S5, the print engine 35 drives a printer (not shown) according to the combined print image, and prints the print image.

  In the print image synthesized in S4, when the blank in the print range exceeds a predetermined amount (for example, when the ratio of the blank to the print range exceeds 50%), a predetermined background color or background image, or a frame in advance A blank portion may be filled by arranging and synthesizing a still image associated with an image in the vicinity of the frame. The association between the frame image and the still image can be performed by the camera 100. For example, as shown in Japanese Patent Laid-Open No. 2003-158653 by the present applicant, any one of the still images continuously shot during the moving image shooting mode is used. , Can be associated with the video that was shot. Information indicating the relationship between the moving image and the still image is also included in the attached information. By filling in the blank part as much as possible, the appearance of the printed image can be further improved.

  Further, the size of each selected frame image may be changed according to the zoom magnification of the attached information corresponding to each selected frame image. FIG. 8 shows a state in which the size of each frame image is enlarged or reduced so as to be inversely proportional to the zoom magnification. In this way, a frame shot at a low magnification, that is, a frame on the wide-angle side can be enlarged, and a frame shot at a high magnification, that is, a frame on the close side can be reduced. You can get a good-looking print.

  Alternatively, the inclination of each selected frame image may be corrected according to the inclination movement information corresponding to each selected frame image. FIG. 9 shows a state in which the inclination of each frame image is corrected in accordance with the inclination movement information. In this way, frame images shot at an inclination can be combined with the inclination corrected, and a good-looking print can be obtained.

  In any case, according to the present invention, a plurality of frame images constituting a moving image can be unified into a single print image without a sense of incongruity, and a print with a wide visual field and good appearance can be obtained.

  100: camera, 200: pan head, 300: printer, 21: motion detector

Claims (5)

A state information acquisition unit that acquires state information, which is information related to the state of the digital camera, including the movement path of the optical axis of the digital camera, the vertical and horizontal movement directions, and the amount of movement at the time of shooting each frame of the moving image. When,
A determination unit that determines an arrangement route of frames within a predetermined range based on the state information;
A composition in which the frames are arranged along the arrangement route determined by the determining unit, and frames in which the overlapping area is less than a predetermined amount are selected from the frames arranged along the arrangement route and combined into one image. And
An information processing apparatus comprising: An information processing apparatus according to claim 1;
A printer that prints a combined image of the information processing apparatus;
A printing system comprising: An imaging unit that outputs an image signal based on a subject image received through an imaging lens;
An image storage unit for storing a moving image based on an image signal output from the imaging unit;
A state information detection unit that detects state information that is information about the state of the digital camera at the time of capturing each frame of the moving image;
An attached information storage unit that stores at least the attached information including state information at the time of shooting each frame of the moving image in association with each frame of the moving image in the predetermined storage medium;
A printing system according to claim 2.   The print system according to claim 3, wherein the state information detection unit is provided on a pan head. Information processing device
Obtaining state information, which is information about the state of the digital camera including the movement path of the optical axis of the digital camera, the vertical and horizontal movement directions, and the amount of movement at the time of capturing each frame of the moving image from a predetermined storage medium;
Determining an arrangement route of frames within a predetermined range based on the state information;
Arranging the frames along the determined arrangement route, selecting a frame having an overlapping area less than a predetermined amount from the frames arranged along the arrangement route, and combining the images into one image;
The image composition method to execute.

Images