JP5321279B2 - Image processing apparatus, image processing system, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing system, and a program.

  There are cases where work or a meeting is performed while transferring image data between remote locations and sharing the image data between the two. For example, Patent Document 1 below describes a technique in which an image captured by a camera installed at a remote place is transmitted to a computer and displayed on a display connected to the computer.

JP 2003-168021 A

  When a partial image obtained by extracting at least a part of each of the image representing the outline and the image representing the details obtained by enlarging a part thereof is received from the other side, the received partial images are displayed between the partial images. The relationship is difficult to grasp.

  One of the objects of the present invention is to provide an image processing apparatus, an image processing system, and a program capable of overlooking the relationship between partial images extracted from an image representing an outline and an image representing a detail.

  In order to achieve the above object, the invention of the image processing apparatus according to claim 1 acquires a partial image obtained by extracting at least a part of each of a plurality of images having different magnifications, and acquired by the acquisition unit. Generating means for generating a three-dimensional object by arranging each partial image inside a cone, wherein the generating means determines each partial image from a vertex of the cone according to a magnification of each partial image. The three-dimensional object is generated by being arranged on a distance plane.

  According to a second aspect of the present invention, in the image processing device according to the first aspect, the generating means assigns each partial image having the same magnification to each positional relationship in the image from which the partial image is extracted. It arrange | positions based on, It is characterized by the above-mentioned.

  According to a third aspect of the present invention, in the image processing apparatus according to the first or second aspect, the generation unit includes a straight line connecting corresponding image elements for each partial image acquired by the acquisition unit. The three-dimensional object is generated so as to overlap the apex of the cone.

  According to a fourth aspect of the present invention, in the image processing device according to any one of the first to third aspects, the viewpoint set in a three-dimensional space in which the three-dimensional object generated by the generating unit is arranged. The image processing device further includes image generation means for generating an image of the three-dimensional object based thereon.

  According to a fifth aspect of the present invention, in the image processing device according to the fourth aspect, an image of a cross section of the three-dimensional object specified in the image generated by the image generating means is represented by the cross section and the cone. Means for generating based on a magnification based on the distance to the vertex of the image and each partial image arranged inside the three-dimensional object.

  The invention of an image processing system according to claim 6 includes a first image processing device and a second image processing device, wherein the first image processing device includes at least a plurality of images having different magnifications. The image processing apparatus includes transfer means for transferring a partial image extracted from a part to the second image processing apparatus, and the second image processing apparatus arranges each partial image transferred by the transfer means within a cone. Generating means for generating a three-dimensional object, wherein the generating means generates the three-dimensional object by arranging the partial images on a plane having a distance corresponding to a magnification of each partial image from the vertex of the cone. It is characterized by doing.

  According to a seventh aspect of the present invention, there is provided a program for acquiring a partial image obtained by extracting at least a part of each of a plurality of images having different magnifications, and acquiring each partial image acquired by the acquisition unit within a cone. Generating means for generating a three-dimensional object by arranging each partial image on a plane having a distance corresponding to a magnification of each partial image from a vertex of the cone. A program for causing a computer to function as generation means.

  According to the first, sixth, and seventh aspects of the present invention, it is possible to overlook the relationship between partial images extracted from an image representing an outline and an image representing details.

  According to the second aspect of the present invention, it is possible to overlook the magnification relationship and the positional relationship between the partial images.

  According to invention of Claim 3, it can arrange | position so that an image may be expanded according to the distance from the vertex of a cone.

  According to the fourth aspect of the present invention, the positional relationship between the partial images can be confirmed from an arbitrary viewpoint.

  According to the fifth aspect of the present invention, it is possible to display a designated area at an arbitrary magnification among the surfaces on which the partial images are arranged.

1 is a system configuration diagram of an image processing system. It is a functional block diagram of an image processing apparatus. It is a figure which shows an example of the position coordinate according to the magnification of an image. It is a figure which shows an example of a display image. It is a figure which shows an example which arrange | positions the partial image of the same magnification on the same plane. It is a figure explaining the specific example of a three-dimensional object production | generation process. It is a figure which shows an example of a bird's-eye view image. It is a figure which shows the display image which extracted the designated area | region part of the cross section designated about the three-dimensional object. It is a flowchart of the production | generation process of a three-dimensional object. It is explanatory drawing of the process which specifies the positional relationship between partial images.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, preferred embodiments (hereinafter referred to as embodiments) for carrying out the invention will be described with reference to the drawings.

  FIG. 1 shows a system configuration diagram of an image processing system 1 according to the present embodiment. As shown in FIG. 1, the image processing system 1 includes an imaging device 5, a first image processing device 10, and a second image processing device 20, and the first image processing device in the second image processing device 20. 10 is a process for generating a view image having a pyramid structure based on the image data transferred from 10.

  The imaging device 5 is connected to the first image processing device 10 via a wired or wireless communication interface, and transfers captured image data to the first image processing device 10. Note that the imaging device 5 adjusts the imaging magnification, the imaging position, the imaging direction, and the like according to the control signal input from the first image processing device 10.

  The first image processing device 10 extracts at least a part of the image data acquired from the imaging device 5 to generate transfer image data, and transmits the generated transfer image data to the second image processing device 20. is there.

  FIG. 2 shows a functional block diagram of the first image processing apparatus 10 according to the present embodiment. As shown in FIG. 2, the first image processing apparatus 10 includes an operation information receiving unit 30, an imaging device control unit 32, an image acquisition unit 34, a display image generation unit 36, a display image output unit 38, and a transfer image generation unit. 40 and a communication unit 42. The functions of the above-described units are such that a computer having a control unit such as a CPU, a storage unit such as a memory, and an input / output unit that transmits and receives data to and from an external device reads a program stored in a computer-readable information storage medium. It may be realized by executing. The program may be supplied to the first image processing apparatus 10 that is a computer by an information storage medium, or may be supplied via data communication means such as the Internet.

  The operation information receiving unit 30 includes an input interface connected to an input device such as a mouse or a keyboard, and receives user operation information input from the input device.

  The imaging device control unit 32 outputs a control signal to the imaging device 5 to control the imaging timing of the imaging device 5 and also controls the imaging magnification, the imaging position, the imaging direction, and the like of the imaging device 5. The imaging device control unit 32 may control the imaging device 5 based on the operation information received by the operation information receiving unit 30.

  The image acquisition unit 34 acquires image data captured by the imaging device 5. In the present embodiment, when acquiring image data from the imaging device 5, the image acquisition unit 34 acquires information on the imaging magnification and the imaging position when the image data is captured. Information on the imaging magnification and the imaging position may be acquired from the imaging device 5 or may be acquired based on control information set by the imaging device control unit 32. The imaging position may be information including, for example, the center position of the image (or one vertex of a rectangle) and the vertical and horizontal lengths.

  FIG. 3 shows an example of position coordinates corresponding to the magnification of each image acquired by the image acquisition unit 34. FIG. 3A shows the coordinate position of an image captured when the magnification α = 1. Note that the image captured when the magnification α = 1 is an image captured by the imaging device 5 of the entire subject. As shown in FIG. 3A, the coordinate range of the captured image is − when the horizontal axis is the X axis and the vertical axis is the Y axis with the position of the imaging center of the image as the origin. W / 2 ≦ x ≦ W / 2 and −H / 2 ≦ y ≦ H / 2.

FIG. 3B shows an example of position coordinates of an image captured by increasing the imaging magnification and enlarging a part of the image (zoomed in). As shown in FIG. 3B, the position coordinates of an image captured when the magnification α (> 1) is −W / when the position of the imaging center of the image is (Cx, Cy). It is 2 <= x-Cx <= W / 2, -H / 2 <= y-Cy <= H / 2. The center positions Cx and Cy are Cx = α · x ₀ and Cy = α · y _0, where x ₀ and y ₀ are the corresponding positions in the entire image shown in FIG.

  The display image generation unit 36 generates a display image to be displayed on the display device based on the image acquired by the image acquisition unit 34. The display image generation unit 36 maintains a correspondence relationship between the image coordinate system and the display image coordinate system.

  The display image output unit 38 includes a graphic interface connected to a display device such as a liquid crystal display, and outputs the display image generated by the display image generation unit 36 to the display device via the graphic interface.

  The transfer image generation unit 40 generates an image of an area designated based on the operation information received by the operation information reception unit 30 as transfer image information for the image displayed on the display device. Hereinafter, a specific example of the transfer image generation process in the present embodiment will be described.

  FIG. 4A shows an example of a display image output by the display image output unit 38. Here, based on the operation information received by the operation information receiving unit 30, assuming that a dot-and-dash line portion in the figure is selected from the display image, the transfer image generating unit 40 performs the following processing. First, the transfer image generation unit 40 converts the coordinate information of the one-dot chain line region of the display coordinate system into coordinate information of the coordinate system corresponding to the original image being displayed, and based on the converted coordinate information, Partial images are extracted from the images. The transfer image generation unit 40 generates transfer image information by adding coordinate information (center position of the partial image), width, height, and magnification information to the extracted partial image.

  The communication unit 42 includes a communication interface, and performs data communication with the second image processing apparatus 20. In the present embodiment, the communication unit 42 transmits the transfer image information generated by the transfer image generation unit 40 to the second image processing apparatus 20.

  Next, the second image processing device 20 will be described. The second image processing apparatus 20 receives the image information transmitted from the first image processing apparatus 10 and performs a process of generating a display image that displays the respective relationships of the received image information in a bird's-eye view.

  FIG. 2 shows a functional block diagram of the second image processing apparatus 20. As shown in FIG. 2, the second image processing apparatus 20 includes a communication unit 50, an image holding unit 52, a three-dimensional object generation unit 54, an operation information reception unit 56, a display image generation unit 58, and a display image output unit. 60. The functions of the above-described units are such that a computer having a control unit such as a CPU, a storage unit such as a memory, and an input / output unit that transmits and receives data to and from an external device reads a program stored in a computer-readable information storage medium. It may be realized by executing. The program may be supplied to the second image processing apparatus 20 that is a computer by an information storage medium, or may be supplied via data communication means such as the Internet.

  The communication unit 50 includes a communication interface, and performs data communication with the first image processing apparatus 10. In the present embodiment, the communication unit 50 receives the transfer image information transmitted from the first image processing apparatus 10.

  The image holding unit 52 includes a storage device such as a semiconductor memory or a magnetic disk device, and holds transfer image information received from the first image processing device 10.

  Based on the transfer image information held in the image holding unit 52, the three-dimensional object generation unit 54 generates a three-dimensional object in which an image represented by each transfer image information is arranged inside the cone object. . Details of the three-dimensional object generation process in this embodiment will be described below.

  When the transfer image information is received by the communication unit 50, the three-dimensional object generation unit 54 generates or updates a three-dimensional object based on the received image information. First, when the transfer image information having the same magnification as the received transfer image information is held in the image holding unit 52, the three-dimensional object generation unit 54 receives the transfer image information on the same plane as the held transfer image information. A partial image indicated by the transferred image information is additionally arranged.

  FIG. 5 shows an example in which partial images having the same magnification are arranged on the same plane. As shown in FIG. 5A, an initial plane is first formed by centering the first image. Next, as shown in FIG. 5B, when a second image having the same magnification is acquired, each of the first image and the second image is determined based on the information on the center positions of the first image and the second image. The relative arrangement is determined, and the center position of the plane on which the first and second images are arranged is recalculated. Hereinafter, a surface on which the first and second images are arranged is referred to as a surface A.

  Next, when the three-dimensional object generation unit 54 acquires an image captured at a magnification different from that of the already arranged image, the three-dimensional object generation unit 54 similarly generates the surface on which the image captured at the different magnification is arranged by the method described above. Then, a cone (three-dimensional object) is generated based on the surface of each magnification. Hereinafter, a specific example of the three-dimensional object generation process will be described with reference to FIG.

  As illustrated in FIG. 6A, for example, if a third image captured at a higher magnification than the first and second images is acquired, the three-dimensional object generation unit 54 includes the first and second images. A surface obtained by enlarging the surface A on which the image is arranged in accordance with the relative magnification with the third image is generated, and the third image is arranged on the enlarged surface. The arrangement of the third image may be determined based on the information on the center position of the third image. The three-dimensional object generation unit 54 calculates the center coordinates of the surface on which the third image is arranged, and arranges the first and second images according to the size of the surface on which the third image is arranged. When the size of the surface is modified and the centers of the two are aligned, the surfaces are arranged so that they are similar. Hereinafter, a surface on which the third image is arranged is referred to as a surface B.

  Next, as shown in FIG. 6B, the surface A and the surface B are spaced apart in parallel to the XY plane in a three-dimensional space, and the center coordinates of both coincide when viewed from the Z-axis direction. Arrange to do. Then, the cone object is configured with the intersection of the straight lines connecting the vertices of the surface A from the surface B as vertices. As a result, a three-dimensional object is generated in which the image of the maximum magnification acquired at the present time is mapped to the bottom surface of the cone inside the cone, and the image of a different magnification is mapped to a cross section parallel to the bottom surface. Note that the magnification of each image is proportional to the distance from the cone apex. The three-dimensional object generation unit 54 performs the above processing every time image information is received from the first image processing apparatus 10, and generates a three-dimensional object in which each partial image is mapped inside the cone model.

  The operation information receiving unit 56 includes an input interface connected to an input device such as a mouse or a keyboard, and receives user operation information input from the input device.

  The display image generation unit 58 generates a display image based on the three-dimensional object generated by the three-dimensional object generation unit 54. In the present embodiment, the display image generation unit 58 generates a view obtained by capturing a three-dimensional object placed in the virtual space from a virtual camera that is also placed in the virtual space as a bird's-eye view image that simultaneously overlooks each partial image. To do.

  FIG. 7 shows an example of an overhead image generated by the display image generation unit 58. As illustrated in FIG. 7, the display image generation unit 58 generates a bird's-eye view image obtained by viewing a three-dimensional object placed in the virtual space from a virtual camera placed in the virtual space. Note that the imaging position, imaging direction, imaging magnification, and the like of the virtual camera may be changed based on the operation information received by the operation information receiving unit 56. In this way, the display image generation unit 58 looks at the three-dimensional object in which the relative positional relationship and the magnification relationship of each partial image are three-dimensionally viewed from the imaging position designated by the user, the imaging direction, and the imaging magnification. An image is to be generated. Also, when displaying a bird's-eye view image, by specifying an arbitrary cross section (magnification plane) of the three-dimensional object, a display image that displays a plane in which each partial image is arranged at a magnification according to the specified cross section is generated. It is good to do. In this case, all partial images may be arranged on the designated cross section, or only a part of partial images that meet the designated conditions may be arranged.

  In addition, when an area is further specified for the cross section specified in the overhead image, the specified area portion of the cross section specified for the three-dimensional object is extracted and displayed as shown in FIG. An image may be generated. Thus, in navigation using a bird's-eye view of a three-dimensional object viewed from a virtual camera whose imaging position, imaging magnification, etc. are variable, the magnification and area specified by the user are specified by specifying the cross section and area of the three-dimensional object. Are generated as display images.

  The display image output unit 60 includes a graphic interface connected to a display device such as a liquid crystal display, and outputs the display image generated by the display image generation unit 58 to the display device via the graphic interface.

  Next, a flow of a three-dimensional object generation process based on the partial image transferred from the first image processing apparatus 10 performed by the second image processing apparatus 20 will be described.

  FIG. 9 shows a flowchart of a process for generating a three-dimensional object. As shown in FIG. 9, when the second image processing device 20 acquires an image from the first image processing device 10 (S101), if the acquired image is the first image (S102: Y), An initialization process is performed (S103). If the image is not the first image (S102: N), it is determined whether or not there is a surface on which an image having the same magnification as the acquired image is arranged (S104), and “Yes” is determined. (S104: Y), the acquired image is additionally arranged on the surface (S105). Then, the second image processing apparatus 20 determines whether or not there are a plurality of surfaces on which images having different magnifications are arranged (S106). When determining that “there is” (S106: Y), the vertex of each surface is determined. A connected three-dimensional object is generated (S108).

  If the second image processing apparatus 20 determines “No” in S104 (S104: N), it adds a plane on which the image received in S101 is placed (S107), and connects the vertices of each plane. The generated three-dimensional object is generated (S108). The second image processing apparatus 20 determines whether or not to continue the process (S109). If it is determined that the process is to be continued (S109: Y), the process returns to S101 and the subsequent processes are repeated. If it is determined (S109: N), the process is terminated.

  The present invention is not limited to the above embodiment. In the above-described embodiment, the case where a partial image extracted from images captured at different magnifications by the imaging device 5 is taken as an example. However, the present invention processes the image captured by the imaging device 5. It is not limited to. For example, an image may be displayed at a different magnification on a display connected to a computer, and the present invention may be applied to a partial image extracted from an image displayed at each magnification to generate a three-dimensional object. In this case as well, the partial image transfer side identifies the relative position between each partial image and each partial image by adding the display magnification to the partial image and the position information in the original image from which the partial image was extracted. Thereafter, the same processing as in the above embodiment may be performed.

  In the above embodiment, the imaging magnification and the position information are added to the partial images. However, when these pieces of information are not added to the acquired partial images, the positions between the partial images are as follows. The relationship may be analyzed.

  FIG. 10 is an explanatory diagram of processing for specifying the positional relationship between the acquired partial images. As illustrated in FIG. 10, the image processing apparatus extracts feature points from the acquired partial images, and analyzes the correspondence between the extracted feature points. Here, when each feature point between the partial images has a correspondence, the magnification between the partial images and the relative position between the partial images are specified based on the correspondence. After the magnification and relative position between the partial images are specified, the same processing as in the above embodiment may be performed.

  In the above embodiment, when generating the three-dimensional object, the center coordinates of the surface corresponding to each magnification are calculated based on the transferred partial image. It is also possible to generate a three-dimensional object by fixing each of the images to the center of each image and setting the size of each surface to a magnification α · S (S is the area of the image when the magnification α = 1).

  In addition, the present invention may be applied not only to the transferred image information but also when generating a display image overlooking the image indicated by each image information based on the image information acquired from the storage device or the like. Of course it is good.

  DESCRIPTION OF SYMBOLS 1 Image processing system, 5 Imaging device, 10 1st image processing device, 20 2nd image processing device, 30 Operation information reception part, 32 Imaging device control part, 34 Image acquisition part, 36 Display image generation part, 38 Display Image output unit, 40 Transfer image generation unit, 42 Communication unit, 50 Communication unit, 52 Image holding unit, 54 Three-dimensional object generation unit, 56 Operation information reception unit, 58 Display image generation unit, 60 Display image output unit

Claims (6)

Acquisition means for acquiring a partial image obtained by extracting at least a part of each of a plurality of images having different magnifications;
Generating means for generating a three-dimensional object by arranging each partial image acquired by the acquiring means inside a cone;
Image generating means for generating an image of the three-dimensional object based on a viewpoint set in a three-dimensional space in which the three-dimensional object generated by the generating means is arranged ,
The image processing apparatus generates the three-dimensional object by arranging the partial images on a plane having a distance corresponding to a magnification of each partial image from a vertex of the cone. The image processing apparatus according to claim 1, wherein the generation unit arranges the partial images having the same magnification based on the positional relationship in the image from which the partial images are extracted. The generation unit generates the three-dimensional object such that a straight line connecting corresponding image elements for each partial image acquired by the acquisition unit overlaps a vertex of the cone. 2. The image processing apparatus according to 2. The image of the cross section of the three-dimensional object specified in the image generated by the image generating means, the magnification based on the distance between the cross section and the apex of the cone, and each portion arranged inside the three-dimensional object the image processing apparatus according to any one of claims 1 to 3, further comprising means for generating on the basis of the image. Including a first image processing device and a second image processing device;
The first image processing apparatus includes:
Transfer means for transferring a partial image obtained by extracting at least a part of a plurality of images having different magnifications to the second image processing apparatus;
The second image processing apparatus includes:
Generating means for generating a three-dimensional object by arranging each partial image transferred by the transferring means inside a cone;
Image generating means for generating an image of the three-dimensional object based on a viewpoint set in a three-dimensional space in which the three-dimensional object generated by the generating means is arranged ,
The image processing system, wherein the generation unit generates the three-dimensional object by arranging the partial images on a plane having a distance corresponding to a magnification of each partial image from a vertex of the cone. Acquisition means for acquiring a partial image obtained by extracting at least a part of each of a plurality of images having different magnifications;
A generating unit configured to generate a three-dimensional object by arranging each partial image acquired by the acquiring unit inside a cone, wherein each partial image is determined from a vertex of the cone according to a magnification of each partial image; Generating means for generating the three-dimensional object arranged on a distance plane ;
A program for causing a computer to function as an image generation unit that generates an image of the three-dimensional object based on a viewpoint set in a three-dimensional space in which the three-dimensional object generated by the generation unit is arranged .

Images