JP6326891B2 - Image data generation system, image data generation method, image processing apparatus, and program - Google Patents

Description

  The present invention relates to an image data generation system that generates image data.

  Conventionally, in an event such as a concert or a television program, a plurality of cameras are used to capture images of singers and the like as subjects from different directions, while editing the images captured by the cameras, Data is being generated.

  Recently, by installing multiple cameras around the subject to capture the subject from different angles, and sequentially switching the cameras that capture the image in the angle changing direction. A method of generating an image in which the viewpoint appears to move at high speed while stopping the movement of the subject or in a slow motion state has been used (for example, Patent Document 1).

JP 2011-243205 A

  However, in the method described in Patent Document 1, when switching from one camera to another camera, the image continuity is ensured and smooth image switching is performed every time image interpolation is performed. The processing load becomes excessive and cannot be performed easily and easily.

  The present invention has been made to solve the above-described problems, and ensures continuity of images when switching images captured by a camera that captures a subject from different angles, and is easily and simply smooth. An object of the present invention is to provide an image data generation system capable of generating image data whose angle changes.

  In order to solve the above-described problems, an image data generation system or the like according to the present invention is installed on a circumference of a concentric circle centering on a subject with a predetermined interval, and each captures an image of the subject. A plurality of synchronized imaging devices having means, and an image processing device for synthesizing images taken by the imaging means, wherein the image processing device switches the imaging device and the imaging device. And generating means for generating, as image data, image data of a virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device. Combining the first image captured by the imaging unit of the switching source imaging apparatus and the second image captured by the imaging unit of the switching destination imaging apparatus, It has a configuration to generate image data of the virtual imaging device.

  With this configuration, when the image data generation system or the like according to the present invention switches each imaging unit, the image of the virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device. Since data can be generated, images can be switched smoothly when the imaging means is switched.

  Therefore, the image data generation system and the like according to the present invention can generate continuous image data without a sense of incongruity while ensuring continuity of images.

  In order to solve the above-described problem, an image processing apparatus or the like according to the present invention includes a switching unit that switches a plurality of synchronized imaging devices, and a switching source imaging device and a switching destination imaging when the imaging device is switched. Generating means for generating, as image data, image data of a virtual imaging device that is virtually formed with the device, wherein the generating means is imaged by the imaging means of the switching source imaging device. One image is combined with the second image captured by the imaging means of the switching destination imaging device to generate image data of the virtual imaging device.

  With this configuration, when the image processing apparatus or the like according to the present invention switches each imaging unit, the image data of the virtual imaging apparatus that is virtually formed between the switching source imaging apparatus and the switching destination imaging apparatus. Since the image can be generated, the image can be switched smoothly when the imaging means is switched.

  Therefore, the image processing apparatus and the like according to the present invention can generate continuous image data without a sense of incongruity while ensuring continuity of images.

  The image data generation system and the like according to the present invention can generate continuous image data without a sense of incongruity while ensuring continuity of images.

1 is a system configuration diagram showing a configuration in an embodiment of an image data generation system according to the present invention. It is a block diagram which shows the structure of the image data generation apparatus of one Embodiment. It is a figure which shows an example of the data stored in the camera management table recorded on the image data generation apparatus of one Embodiment. It is FIG. (1) for demonstrating the composite image data generation process performed in the image data generation apparatus of one Embodiment. It is FIG. (2) for demonstrating the synthetic | combination image data generation process performed in the image data generation apparatus of one Embodiment. It is a flowchart (the 1) which shows the operation | movement of the synthetic | combination image data generation process performed in the image data generation apparatus of one Embodiment. It is a flowchart (the 2) which shows the operation | movement of the composite image data generation process performed in the image data generation apparatus of one Embodiment. It is the figure which made the graph the change algorithm of the blending coefficient recorded on the image data generation apparatus of one Embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following embodiment, an image according to the present invention is provided for an image data generation system including a camera that captures an image of an object and an image data generation apparatus that acquires image data from the camera and performs editing processing. It is an embodiment when a data generation system, an image data generation method, an image processing apparatus, and a program are applied.

[1] Image Data Generation System First, the configuration and outline of the image data generation system 1 according to the present embodiment will be described with reference to FIG. FIG. 1 is a system configuration diagram showing the configuration of the image data generation system 1 in the present embodiment.

  The image data generation system 1 of the present embodiment continuously captures images of the subject O while switching cameras installed around the subject O, and synthesizes and edits the images captured by the cameras 10. Thus, the system generates image data corresponding to an image obtained by imaging the subject from 360 degrees in all directions (hereinafter referred to as “omnidirectional image”).

  Specifically, as shown in FIG. 1, the image data generation system 1 of the present embodiment is equidistant or predetermined on a concentric circle C having a subject O as a central point and a radius R (imaging distance). The plurality of cameras 10 are installed at different intervals (non-equal intervals), and are connected to each camera 10 to capture images of the subject O from different angles, and are controlled by each camera 10 and captured by each camera 10. An image data generation device 20 that generates image data corresponding to an omnidirectional image of the subject O by combining and editing the obtained images, thereby realizing the above-described system.

  In the image data generation system 1, the number of cameras installed around the subject O is arbitrary, but in the present embodiment, for the purpose of clarifying the explanation, as illustrated in FIG. The description will be made assuming that images are taken by six cameras 10-1 to 10-6.

  Each camera 10 has an imaging device (CCD image sensor: Charge Coupled Device Image Sensor) as an imaging means, and is a camera for capturing a moving image such as a digital video camera, a web camera, or a video camera attached to a smartphone. Functions as an imaging device of the present invention. Each camera 10 has various known functions such as an autofocus function and a camera shake correction function, and generates image data composed of a plurality of frames by imaging the subject O.

  In particular, each camera 10 of the present embodiment captures an image of the subject O while synchronizing under the control of the image data generation device 20, and the generated image data is transferred to the image data generation device 20. It has a configuration to supply.

  In addition, the cameras 10 are installed at equal intervals along a concentric circle C or a part of the circumference of the concentric circle C. In the present embodiment, six units are installed at equal intervals along the concentric circle C, but a plurality of cameras 10 may be used. However, when two adjacent cameras 10 (for example, cameras 10-1 and 10-2) and the angle formed by the subject O exceed 90 degrees, it is possible to ensure continuity of images. Since it becomes difficult, basically, it is desirable to use four or more cameras 10.

  The image data generation device 20 is formed by the first image formed by the first image data output from one camera 10 during a plurality of frame periods and the second image data output from another camera 10 adjacent thereto. The second image is combined with the second image and the display is switched from the first image to the second image.

  Specifically, the image data generation device 20 starts from the elapsed time from the switching start timing or the first frame during a period when switching the display from the first image to the second image (hereinafter referred to as “during the switching period”). In accordance with the number of frames, a process for synthesizing the images of the respective frames (hereinafter, referred to as “synthesis coefficient”) while changing a blending coefficient (hereinafter also referred to as “synthesis coefficient”) for synthesizing (blending) the first image and the second image "Composite image data generation process") is executed.

  In particular, the image data generation device 20 switches the camera 10 while synchronizing each frame output from each camera 10, and when switching the camera 10, the switching source camera 10 and the switching destination camera 10 The image data (that is, image data) of a plurality of virtual cameras M that are virtually formed in between are generated. Then, the image data generation device 20 outputs, as image data of the virtual camera 10, a composite image obtained by combining the first image captured by the switching source camera and the second image captured by the switching destination camera 10. It has a configuration.

  Further, the image data generation device 20 generates image data of the virtual camera 10 using a synthesis coefficient (hereinafter referred to as “blending coefficient”) indicating a ratio of synthesizing the first image and the second image, and the blending. The coefficient changes linearly according to the position between the switching source camera 10 and the switching destination camera 10 where the virtual camera 10 is formed.

  With such a configuration, the image data generation system 1 of the present embodiment can generate continuous image data while switching the captured images of the cameras that capture the subject O from different angles. The image can be switched smoothly while securing the property.

  In addition, the image data generation system 1 according to the present embodiment can improve the continuity of images when switching images by increasing the number of cameras installed, so that a smoother image can be generated. It has become. That is, for example, when the number of cameras is increased to about 16 and images are captured at a normal frame rate (about 60 fps), the switching of images cannot be recognized by the human eye, which makes the user feel uncomfortable. There is no need to switch images.

[2] Image Data Generation Device Next, the configuration of the image data generation device 20 of the present embodiment will be described with reference to FIGS. 2 and 3. 2 is a block diagram showing the configuration of the image data generation device 20 of the present embodiment, and FIG. 3 shows an example of data storage of the camera management table TBL recorded in the image data generation device 20 of the present embodiment. FIG.

  As shown in FIG. 2, the image data generation device 20 of the present embodiment includes an input / output (I / O) between the recording unit 200 in which data corresponding to various types of information is recorded, and the camera 10 and other external devices. O) Display unit 240 constituted by interface 210, liquid crystal element or panel of organic EL (Electro Luminescence) element, display control unit 250 for controlling display of image on display unit 240, and composite image data generation processing Each of the above-described units includes a data processing unit 260 that executes other processing, an operation unit 270 configured by a keyboard, a mouse, a touch panel, and the like, a device management control unit 280, and a timer 290 that specifies the current date and time. Are connected to each other by a bus B.

  The recording unit 200 includes, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and at least information for managing each camera 10 is recorded in the recording area. And an image data recording unit 203 for recording the image data after the image synthesis, and a RAM 204 functioning as a work area and an image buffer.

  The camera management table recording unit 201 records a camera management table TBL for managing each camera 10. Further, as shown in FIG. 3, the camera management table TBL stores a camera ID for identifying each camera 10 and camera information in association with each other.

The camera information of this embodiment includes
(1) installation position information indicating the installation position of each camera 10;
(2) frame rate information indicating a frame rate at the time of imaging of each camera 10;
It is included.

  FIG. 3 shows an example of a camera management table TBL when six cameras 10 are installed as shown in FIG. That is, FIG. 3 shows that the installation position of the camera ID “Camera 1” is set to “0 degree”, and “Camera 2 to 6” is stored for each “60 degrees”. Has been.

  In FIG. 3, the frame rates of all the cameras 10 are “60 fps”. However, when the frame rates of the cameras 10 are different, different frame rate information is stored.

  The image data recording unit 203 records image data generated by the combined image data generation process in association with predetermined information such as an image ID.

  The RAM 204 is used as a work area when the composite image data generation process is being executed, and also functions as an image buffer. In particular, when the composite image data generation process is executed, the RAM 204 sequentially records image data output from each camera 10 (that is, image data before composition) for each frame. Note that the image of each frame recorded in the RAM 204 is held for a certain period.

  The I / O interface 210 is an input / output interface such as USB (Universal Serial bus), wireless LAN (IEEE802.11a, b, n, ac), HDMI (registered trademark (High Definition Multimedia Interface)), and the like. The camera 10 is connected to each camera 10 by wire or wirelessly, and relays data exchange between each camera 10 and the bus B.

  Note that an optical disc recorder such as a BD (Blu-ray disc) can be connected to the I / O interface 210, and the generated composite image data can be recorded on a recording medium. Further, a printer for photo printing can be connected to the I / O interface 210, and an image can be printed out based on the composite image data.

  The display unit 240 is configured by a panel of a liquid crystal element or an EL (Electro Luminescence) element having an image display area of a predetermined size (W1920 × H1080 pixels), and based on display data generated by the display control unit 250 Display an image.

  The display control unit 250 generates drawing data necessary for causing the display unit 240 to draw a predetermined image under the control of the device management control unit 280 and the data processing unit 260, and the generated drawing data is displayed on the display unit 240. Output to.

  The data processing unit 260 is configured by an independent central processing unit (CPU), or is configured using the central processing unit (CPU) of the device management control unit 280 and is controlled by the device management control unit 280. The composite image data generation process is executed by executing the program below.

  Specifically, the data processing unit 260, as the composite image data generation process, performs another camera 10 adjacent to the first image formed by the first image data output from one camera 10 during a predetermined period. A composite image data generation process for switching the display from the first image to the second image is executed while combining the second image formed by the second image data output from.

  In particular, the data processing unit 260 switches the camera 10 while synchronizing each frame output from each camera 10, and switches the camera 10 between the switching source camera 10 and the switching destination camera 10 when switching the camera 10. Image data of a virtual camera M that is virtually formed between them is generated as image data.

  Then, the image data generation device 20 combines the first image captured by the switching source camera and the second image captured by the switching destination camera 10 to generate image data of the virtual camera 10.

  In particular, the data processing unit 260 performs a composite image data generation process by using a camera control unit 261 that controls each camera 10 by executing a predetermined application, and a blending coefficient used when performing image synthesis. A coefficient determination unit 262 to determine, an image composition processing unit 263 that performs image composition, and a composite image data generation unit 264 that generates composite image data are realized.

  For example, the image composition processing unit 263 of this embodiment constitutes a switching unit and a generation unit.

  The camera control unit 261 outputs a control command to each camera 10 via the I / O interface 210 based on the administrator's instruction input via the operation unit 270 to control the corresponding camera 10. Specifically, the camera control unit 261 has a configuration capable of performing various settings necessary for executing power ON / OFF, focus, frame synchronization setting, and other composite image data generation processing. ing.

  In addition, the camera control unit 261 acquires image data captured and generated by each camera 10 via the I / O interface 210, and associates the acquired image data with the camera 10 for each frame in the RAM 204. Record sequentially.

  The coefficient determination unit 262 calculates a blending coefficient based on an instruction of an administrator input via the operation unit 270 while interlocking with the display control unit 250 and the operation unit 270 during execution of the composite image data generation process. Then, the calculated blending coefficient is provided to the image composition processing unit 263.

  In particular, the coefficient determination unit 262 according to the present embodiment calculates a blending coefficient according to the elapsed time from the image switching start timing and the number of frames in the composite image data generation process, and the calculated blending coefficient is input to the image composition processing unit 263. provide. The calculation of the blending coefficient in the present embodiment will be described later.

  The image composition processing unit 263 performs the first image data and the second image data configured by the first image data recorded in the RAM 204 based on the blending coefficient determined by the coefficient determination unit 262 during the composite image data generation process. The second image constituted by is synthesized.

  Specifically, in the composite image data generation process, the image composition processing unit 263 includes a first image output from one camera 10 and a second image output from the camera 10 adjacent to the one camera 10. Based on the blending coefficient that changes during the period determined by the coefficient determination unit 262 while assuming the virtual camera M according to the switching elapsed time or the number of frames during the switching period of the image, Frame composite into two images. The frame composition of this embodiment will be described later together with the blending coefficient.

  The composite image data generation unit 264 generates composite image data in a format that can be viewed by the administrator based on the composite image combined by the image composition processing unit 263 and causes the image data recording unit 203 to record the composite image data. Then, the composite image data generation unit 264 interlocks with the display control unit 250 in accordance with the input operation performed on the operation unit 270 by the administrator using the composite image data recorded in the image data recording unit 203 in this way. While being displayed on the display unit 240.

  In addition, the composite image data generation unit 264 supplies the generated composite image data to the BD recorder connected to the I / O interface 210 in accordance with an input operation by the administrator, and records it on the recording medium.

  Further, when the administrator instructs the printout while selecting a predetermined frame included in the composite image data to the operation unit 270, the composite image data generation unit 264 is based on the image of the selected frame. Then, still image data is generated and output to a photographic printer connected to the I / O interface 210. At this time, the composite image data generation unit 264 outputs a print command command together with the generated still image data to the printer.

  The operation unit 270 is configured by a keyboard, a pointing device, and the like, and is used by the administrator of the image data generation apparatus 20 to receive an instruction input from the administrator when the subject O is imaged using the camera 10.

  The device management control unit 280 is mainly configured by a central processing unit (CPU) and includes various input / output ports such as a key input port and a display control port, and the overall functions and image of the image data generation device 20. General control of the overall process for the synthesis process.

[3] Principle of Composite Image Data Generation Processing Next, the composite image data generation processing executed in the data processing unit 260 of the present embodiment will be described using FIG. 4 and FIG. 4 and 5 are diagrams for explaining the composite image data generation process executed in the image data generation apparatus of the present embodiment.

  Normally, if the image output from the camera 10 arranged concentrically in a concentric circle C formed at the imaging distance R as in the present embodiment is simply switched sequentially, a sudden change occurs in the image. Will give the viewer a sense of discomfort.

  Therefore, in the present embodiment, during a predetermined period (that is, during an image switching period), the ratio of the first image is increased at the switching start timing at the time of switching or the first frame at that time, and the ratio is gradually increased. By increasing the ratio of the second image while reducing the image quality and generating an image that finally switches to the second image, it becomes possible to switch the image smoothly while ensuring the continuity of the image. ing.

  Specifically, as illustrated in FIG. 4, the data processing unit 260 assumes a virtual camera M based on an angle difference from one camera 10-1 to another camera 10-2, and performs switching in the switching period. A blending coefficient for blending the first image and the second image is determined based on the elapsed time from the start, the number of frames from the first frame in the switching period, or a predetermined output timing in the switching period, The first image and the second image are synthesized (blended) based on the determined blending coefficient and output as an image of the virtual camera M.

  Then, when the blending coefficient is “α”, the data processing unit 260 outputs the composite image “ImgOut” based on the first image “Img1” and the second image “Img2” according to the following (Equation 1). It has become.

  In particular, the data processing unit 260 performs the blending coefficient “α” for each predetermined timing during the image switching period, that is, for each output position of the composite image when it is assumed that there is the virtual camera M illustrated in FIG. The composite image “ImgOut” is output while changing. In this embodiment, the data processing unit 260 uses the angle change amount “θn” in order to calculate the blending coefficient “α”. In the present embodiment, the blending coefficient changes linearly according to the position of the virtual camera M.

  For example, in this embodiment, in order to determine the blending coefficient “α”, the data processing unit 260 determines an angle difference “from one camera 10 to another camera 10” as shown in (Formula 2) below. A blending coefficient “α” is determined based on a change amount “θn” of the angle with respect to θx.

  The angle change amount “θn” can also be expressed as “θn = ωΔt”, where “ω” is the angular velocity when the camera is switched and “Δt” is the elapsed time from the start of imaging.

  In the present embodiment, the data processing unit 260 determines the angle change amount “θn” based on the synthesis condition input under the direction of the administrator or the predetermined synthesis condition. It has become.

  Specifically, when generating and outputting a composite image for each frame, the data processing unit 260 uses the camera 10-1 as a composite condition input by the administrator or as a predetermined composite condition. The time for switching the image from the camera 10-2 to the camera 10-2, that is, the switching time required from the start to the end of the switching is detected.

  Further, the data processing unit 260 calculates the angle difference “θx” between the two target cameras based on the camera information stored in the camera management table recording unit 201. For example, the camera (hereinafter referred to as “reference camera” or “switching source camera”) 10-1 serving as a reference has an angle of “0 degrees” and is adjacent to the reference camera 10-1 (hereinafter referred to as “reference camera”). When the angle of the installation position of 10-2 is “60 degrees”, the data processing unit 260 calculates the angle difference “θx = 60 degrees”.

  Further, the data processing unit 260 calculates an angle “Δθ” that changes in one frame from the detected synthesis condition and the calculated angle difference “θx”, and calculates the change amount “θn” of each frame from the calculated angle “Δθ”. decide. Then, the data processing unit 260 calculates the blending coefficient “α” based on the change amount “θn” and the angle difference “θx” of each frame.

  For example, assuming that the frame rate is 60 fps, the data processing unit 260, for the switching time “T = 10 seconds” and the angle difference “θx = 60 degrees”, It is calculated as “0.1 degree”. In other words, the data processing unit 260 assumes that the virtual camera M is set between the switching source camera 10 and the switching camera every “0.1 degree”.

Then, the data processing unit 260 (1) calculates “1/60 seconds” when the change amount is “θ1”, and calculates the blending coefficient “α = 0.1 / 60”,
(2) When the change amount is “θ2”, “2/60 seconds” is obtained, and the blending coefficient “α = 0.2 / 60” is calculated.
(3) When the change amount is “θ60”, “1 second” is obtained, and the blending coefficient “α = 6/60” is calculated.
Finally, when the change amount is “θ600”, each change amount “θn” is calculated until “10 seconds” and the blending coefficient “α = 1”.

  On the other hand, for each frame, the data processing unit 260 combines the image of the frame, that is, the first image and the second image, based on the calculated blending coefficient “α”, and the combined image is the image of the virtual camera M. Output as. As a result, the data processing unit 260 can perform image composition of the first image Image1 and the second image Image2 as shown in FIG. 5, for example.

  On the other hand, in the present embodiment, it has been described that a composite image is generated and output for each frame. , Every 0.05 degrees), and based on the determined output timing interval or output angle interval, the blending coefficient “α” may be calculated to generate a composite image.

  In this case, the frame cannot be used using a frame that matches the determined output timing interval or the output timing calculated based on the output angle interval (timing determined by the output angle interval, angle difference, and switching time). Frames may be thinned out, or a composite image may be generated using the latest frame image recorded in the RAM 104 at the output timing.

  In the present embodiment, the composite image data generation processing is executed by detecting the output angle interval, the output timing interval, and the switching time for outputting the composite image. However, only the output angle interval or the output timing interval is executed. Only, i.e., without determining the switching time required from the start to completion of switching, only the angle or timing at which the synthesis is performed is determined, and each change amount "θn" is calculated, and the calculated change The composite image data generation process may be executed while calculating the blending coefficient α based on the amount.

[4] Composite Image Data Generation Processing Next, composite image data generation processing executed in the image data generation device 20 of the present embodiment will be described using FIGS. 6 and 7. 6 and 7 are flowcharts showing the operation of the composite image data generation process executed in the image data generation device 20 of the present embodiment.

  In this operation, it is assumed that the camera management table illustrated in FIG. 3 is already recorded in the recording unit 200, and the power is turned on for each camera 10 to start imaging, and the captured images are the same. It is assumed that the data is output to the image data generation device 20 at the frame rate.

  Further, it is assumed that image data sequentially output from the camera 10-1 and the camera 10-2 is recorded in the RAM 204 for each frame, and the image data of the latest frame is always held, and the composite image is stored for each frame. Is generated.

  Further, prior to this operation, the apparatus management control unit 280 is in a state of waiting for an instruction to generate a composite image on the subject O from the administrator while displaying a predetermined home screen on the display unit 240. To do.

  In this operation, the camera 10-1 adjacent to the reference camera (ie, the reference camera (switching source camera)) 10-1 (ie, the adjacent camera (switching destination camera)) 10- A case where the subject O is imaged using 2 will be described as an example, and a case where linear synthesis is performed for each frame (a case where a linearly changing blending coefficient is used) will be described.

  First, when the image composition processing unit 263 detects an input instruction indicating start of composition for the output composite image data generation processing for the subject O via the operation unit 270 (step S100), the camera that performs image composition from the camera management table TBL. The camera information relating to 10-1 and the camera 10-2 is read and recorded in the RAM 204 (step S101). In particular, the image composition processing unit 263 records installation position information indicating the installation positions of the cameras 10-1 and 10-2 in the RAM 204.

  Next, the image composition processing unit 263 detects the composition condition input based on the operation of the administrator in conjunction with the display control unit 250 (step S102). Specifically, the image composition processing unit 263 detects a time for switching an image from the camera 10-1 to the camera 10-2, that is, a switching time required from the start to the end of switching.

  Next, the image composition processing unit 263 calculates an angle difference “θx” between the reference camera 10-1 and the adjacent camera 10-2 based on the read camera information (step S103). Specifically, the image composition processing unit 263 extracts installation information from the camera information corresponding to the cameras 10-1 and 10-2 recorded in the RAM 204 in the process of step S <b> 103, and sets the difference value as the angle difference “θx”. ".

  Next, the image composition processing unit 263 calculates an angle “Δθ” that changes in one frame based on the composition condition and the angle difference detected in step S102. (Step S104). Note that when the output timing interval or the output angle interval of the composite image input by the administrator is used as the composite condition, the processing in step S104 is not executed.

  Next, when the image composition processing unit 263 detects the start of composition via the operation unit 270 (step S105), it sets the installation information “0 degree” of the camera 10-1 to “θn” (step S106), and the timer 190 Is started (step S107).

  Next, the image composition processing unit 263 obtains the first image and the second image from the image data of the camera 10-1 and the camera 10-2 held as the latest frame image in the RAM 204 (step S111). .

  Next, the coefficient determination unit 262 determines the blending coefficient “α” based on the number of frames from the start of counting (step S112). Specifically, the coefficient determination unit 262 determines the change amount “θn” of the corresponding frame based on the angle “Δθ” (that is, the position of the virtual camera M) that changes in one frame from the detected combination condition and angle difference. And blending coefficient “α” is calculated based on the determined change amount “θn” and angle difference “θx” of the corresponding frame.

  In addition, the coefficient determination unit 262 changes based on the output timing interval of the composite image, the angle difference, and the switching time when using the output timing interval or the output angle interval of the composite image input by the administrator as a composite condition. The amount “θn” may be determined, or the change amount “θn” may be determined from the output angle interval of the composite image, and the blending coefficient “α” may be calculated based on the determined change amount “θn”.

  Next, the image composition processing unit 263 synthesizes the acquired first image and second image based on the determined blending coefficient “α” (step S113), and interlocks with the display control unit 250 to display the display unit 240. The synthesized image is output as an image of the virtual camera M (step S114). Note that the image composition processing unit 263 records the synthesized image in the RAM 204 when recording is set.

  Next, the image composition processing unit 263 adds the angle “Δθ” (for example, “0.1 degree”) determined in the process of step S104 to “θn” and increments the predetermined angle (step S115). It is determined whether the incremented “θn” has become the angle difference “θx” (step S116).

  At this time, if it is not determined that “θn” is “θx”, the image composition processing unit 263 counts the time interval determined in step S104 based on the timer 109 and outputs the next output. The system waits until the timing (step S117), and repeats the processes of steps S111 to S116. On the other hand, when it is determined that “θn” is a predetermined angle, this operation is terminated.

  As described above, the image data generation system 1 according to the present embodiment sets a plurality of cameras 10 on the concentric circle C centered on the subject O, and switches the cameras 10 every predetermined period (10 seconds in the above example). In the case where the subject O is captured and the images captured by the adjacent cameras 10 are combined for each frame to generate composite image data, the subject O generates omnidirectional image data. The camera image can be switched while ensuring the continuity of the image.

  In addition, the image data generation system 1 of the present embodiment is configured to change the blending coefficient according to the elapsed time of the switching period, the number of frames from the first frame of the switching period, or a predetermined output timing. Therefore, when the camera 10 is switched, the composite image data can be generated so that the image of the front camera is gradually switched to the camera image of the rear side, so that the composite image data of the omnidirectional image is generated. The continuity of the images at the time can be ensured, and the smoothness of the images can be ensured.

[5] Modification [5.1] Modification 1
In the above embodiment, the composite image data generation process is executed using the image output from the camera 10, but the image is already recorded and synchronized, and is captured by the same camera system. The composite image data generation process may be executed using.

[5.2] Modification 2
In the above embodiment, the combined image data synthesized by the image data generation device 20 is displayed on the display unit 240 and recorded on a recording medium or printed out by a printer. It is good also as a structure delivered to the communication terminal device.

  In this case, the image data generation device 20 may be provided with a network communication unit for performing communication via the network, and the composite image data may be distributed in response to a request from the communication terminal device.

  For example, when the subject O is a human being, by adopting this configuration, composite image data of an omnidirectional image captured at an amusement park attraction or the like may be generated and distributed to the user's communication terminal device. It becomes possible.

[5.3] Modification 3
In the composite image data generation process of the above embodiment, the blending coefficient “α” is calculated based on the amount of change in angle and the angle difference between the cameras 10, but is calculated based on a predetermined algorithm. Also good.

  For example, in addition to the linear function algorithm Y = −AX + C (hereinafter referred to as “first algorithm” (see FIG. 8A)) that performs linear synthesis, the first and last when switching images are moderated. A function algorithm (hereinafter referred to as “second algorithm” (FIG. 8B)) or a quadratic function algorithm (hereinafter referred to as “third algorithm”) that changes rapidly with time. 8 (C))) may be used to execute the composite image data generation process.

  In this case, in each graph shown in FIG. 8, the vertical axis indicates the blending coefficient indicated by “α”, and the horizontal axis indicates the number of frames from the start frame when the image synthesis is started, the image synthesis. The elapsed time from the start of the image and the output angle changed from the start of the image composition. Then, when generating the composite image, the data processing unit 360 determines the blending coefficient “α” value from the graph, and generates the composite image using the determined blending coefficient “α” value.

[5.4] Modification 4
In addition to the above-described embodiment, the image composition processing unit 263 includes a plurality of virtual cameras M arranged between the switching source camera 10 and the switching destination camera 10. As the images are arranged closer to each other, the first image and the second image are combined to generate image data of the camera 10.

  Specifically, the image composition processing unit 263 generates a composite image at a fine interval (for example, Δθ = 0.1 degrees) at a position (angle) close to the switching source camera 10 or the switching destination camera 10. On the other hand, at a position far from the switching source camera 10 or the switching destination camera 10 (for example, when the angle difference is 60 degrees as described above, the position is 25 degrees to 30 degrees from the switching source camera 10). Then, a composite image is generated at a rough interval (for example, Δθ = 5 degrees).

  That is, the image composition processing unit 263 correlates with the blending coefficient. When there are many composition components in any of the images, the image composition processing unit 263 displays the composite image finely, and when the two images are synthesized at substantially the same composition ratio. , Display a rough composite image.

  In this case, since the images can be switched more smoothly, it is possible to ensure the continuity of the images when generating the composite image data of the omnidirectional images. In this case, a linear blending coefficient may be used as in the present embodiment, but it is more effective when the second algorithm shown in the third modification is used.

[5.5] Modification 5
In the above embodiment, the composite image data generation processing is executed by performing switching control on the image data output from each camera 10, but the camera control unit 261 turns ON and OFF the operation state of each camera. The combined image data generation process may be executed in conjunction with the switching of the above.

DESCRIPTION OF SYMBOLS 1 ... Image editing system 10 ... Camera 20 ... Image data generation apparatus 200 ... Recording part 201 ... Camera management table recording part 203 ... Image data recording part 204 ... RAM
210 ... I / O interface 240 ... display unit 250 ... display control unit 260 ... data processing unit 261 ... camera control unit 262 ... coefficient determination unit 263 ... image synthesis processing unit 264 ... composite image data generation unit 270 ... operation unit 280 ... apparatus Management control unit 290 ... timer

Claims (8)

A plurality of synchronized imaging devices installed on the circumference of a concentric circle centered on the subject with a predetermined interval and having imaging means for imaging the subject,
An image processing device for synthesizing images picked up by the image pickup means;
With
The image processing apparatus is
Switching means for switching the imaging device;
Generating means for generating, as image data, image data of a virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device when the imaging device is switched;
Have
The generating means is
Combining the first image captured by the imaging unit of the switching source imaging apparatus and the second image captured by the imaging unit of the switching destination imaging apparatus to generate image data of the virtual imaging apparatus ;
A plurality of virtual imaging devices arranged between the switching source imaging device and the switching destination imaging device, and arranged at finer intervals as they come closer to the switching source imaging device and the switching destination imaging device An image data generation system for generating image data of an apparatus . The image data generation system according to claim 1,
The generating means generates image data of the virtual imaging device using a synthesis coefficient indicating a ratio of synthesizing the first image and the second image;
The image data generation system, wherein the synthesis coefficient changes linearly according to a position between the switching source imaging device and the switching destination imaging device in which the virtual imaging device is formed. The image data generation system according to claim 1 or 2 ,
An image data generation system in which the generation means synthesizes an image while thinning out some frames. The image data generation system according to any one of claims 1 to 3 ,
The image data generation system, wherein the imaging means is installed so that the subject can be imaged 360 degrees at the predetermined interval. Computer
Switching means for switching a plurality of synchronized imaging devices each having an imaging means for imaging the subject, and installed on the circumference of a concentric circle centered on the subject, with a predetermined interval; and
Generating means for generating, as image data, image data of a virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device when the imaging device is switched;
Function as
As the generating means,
The first image captured by the imaging means of the switching source imaging device and the second image captured by the imaging means of the switching destination imaging device are combined to generate image data of the virtual imaging device
And
A plurality of virtual imaging devices arranged between the switching source imaging device and the switching destination imaging device, and arranged at finer intervals as they come closer to the switching source imaging device and the switching destination imaging device A program that functions to generate image data of a device . A switching step of switching a plurality of synchronized imaging devices each having an imaging means for imaging the subject, each being installed at a predetermined interval on a concentric circle centered on the subject,
A generation step of generating, as image data, image data of a virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device when the imaging device is switched;
Including
When generating the image data, the first image captured by the imaging unit of the switching source imaging apparatus and the second image captured by the imaging unit of the switching destination imaging apparatus are synthesized, and the virtual imaging is performed. Generate device image data ,
A plurality of virtual imaging devices arranged between the switching source imaging device and the switching destination imaging device, and arranged at finer intervals as they come closer to the switching source imaging device and the switching destination imaging device An image data generation method characterized by generating image data of a device . Switching means for switching a plurality of synchronized imaging devices;
Generating means for generating, as image data, image data of a virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device when the imaging device is switched;
With
The generating means is
Combining the first image captured by the imaging unit of the switching source imaging apparatus and the second image captured by the imaging unit of the switching destination imaging apparatus to generate image data of the virtual imaging apparatus ;
A plurality of virtual imaging devices arranged between the switching source imaging device and the switching destination imaging device, and arranged at finer intervals as they come closer to the switching source imaging device and the switching destination imaging device An image processing apparatus for generating image data of the apparatus. Computer
Switching means for switching a plurality of synchronized imaging devices; and
Generating means for generating, as image data, image data of a virtual imaging device that is virtually formed between the switching source imaging device and the switching destination imaging device when the imaging device is switched;
Function as
As the generating means,
Combining the first image captured by the imaging unit of the switching source imaging apparatus and the second image captured by the imaging unit of the switching destination imaging apparatus to generate image data of the virtual imaging apparatus ;
A plurality of virtual imaging devices arranged between the switching source imaging device and the switching destination imaging device, and arranged at finer intervals as they come closer to the switching source imaging device and the switching destination imaging device A computer that functions to generate image data of a device .