JP2006025250A - Image processing apparatus, photographing system, image processing method, image processing program, and record medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a photographing method for displaying an object and allowing a user to photograph only an object that is allowed to be photographed. <P>SOLUTION: An image processing apparatus 40 comprises an origination point calculating means 42, a replacement region determining means 44, a replacement image generating means 46, an image processing means 48, and an output means. The origination point calculating means 42 computes whether an object 10 is included in a photographed image or not on the basis of a position of a photographing means 22, a position of the object 10, and camera parameters; and employs the position of the object 10 in the photographed image as an origination point if the object 10 is included in the photographed image. The replacement region determining means 44 determines a replacement region replaced in the photographed image based on the origination point. The replacement image generating means 46 generates a replacement image replaced at the replacement region determined by the replacement region determining means 44. The image processing means 48 synthesizes the replacement image at the replacement region in the photographed image. The output means outputs the photographed image synthesized by the image processing means 48. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image processing apparatus, an imaging system, an image processing method, an image processing program, and a recording medium.

  In recent years, small digital cameras and mobile phones with digital cameras have become widespread, and digital photographs have been easily taken. In addition, digital cameras that can be used to shoot moving images and digital video cameras that mainly shoot moving images have been obtained by general consumers, and it has become easy to shoot images.

Digital photos of still images and digital movies of movies are imported to a computer and sent to another computer via the Internet or other network connected to the computer, or sent to a communication partner using a mobile phone. Or In this way, a photographed image such as a digital photograph or a digital movie has been made public, beyond public use, by being disclosed to various people through communication (Non-patent Document 1, etc.) ).
Ueda, Nakanishi, Motoe, Matsukawa, “Spatiotemporal Poemer: WebGIS Operation Experiments and Evaluation Using GPS Camera Mobile Phones”, Information Processing Society of Japan, Interaction 2004 Proceedings pp.145-152, 2004

  However, in the conventional technique, since all the subjects in the photographing target range to which the camera is directed are photographed, there is a problem that an inconvenient subject appears in the photographed image. Here, an “inconvenient subject” means that the subject is inconvenient (when a subject that is not desired to be photographed is photographed), or inconvenient for the photographer (a subject that is not desired by the photographer is photographed). Can be classified).

  First, the inconvenient case for the subject is infringement of portrait rights, infringement of copyright, or infringement of privacy. For example, you may shoot a specific subject that should not be taken, such as taking a picture without the subject's consent, taking a voyeur, or taking something that should not be taken unintentionally. The problem has arisen.

  Regarding copyrights, for example, apart from the concert organizers taking pictures for the purpose of selling concert videos, non-concert organizers etc. prohibit photography by the organizers. Despite this, it is possible to photograph the singer without permission.

  In order to solve this problem, it is possible to work on the morals of the photographer, and no essential solution has been proposed. For example, even if a warning sign is placed around the subject so that the photographer does not shoot a subject that should not be photographed, or even if a subject that should not be photographed is photographed, there is a privacy problem. In order not to distribute photos and videos, a warning message is displayed when the shot image is distributed.

  For events such as concerts, the mainstream method is to manually check in cameras when entering the event venue. This takes time and effort on the event organizer side to separate photographers who have received permission, such as magazine reporters, and photographers who have not received permission, such as general spectators. In addition to prohibited acts intended by the organizers (for example, do not shoot the performer only during the event), it also relates to organizers, such as commemorative shoots of the audience outside the performance time of the event. Even the act of not taking pictures is prohibited. Therefore, the method of manually checking camera carry-in is extremely inconvenient for both the event organizer and the event audience.

  On the other hand, if it is inconvenient for the photographer, for example, when shooting properly, even if an object that should not be accidentally captured is framed in, even if the photographer does not know or understand that This is a case where it cannot be prevented from being copied because it has already been recorded. On the other hand, there is no means for erasing a subject that has been photographed as shooting noise, and the shooting has to be repeated.

  In addition, it is difficult to capture only a specific subject when shooting only a specific subject and other backgrounds cannot be captured (for example, a request to avoid miscellaneous backgrounds on a videophone). is there.

  In addition, there is a method to hide the things that should not be copied using a frame or cover in order to prevent unexpected and unwanted subjects and backgrounds from being shot. There was a sense of incongruity in the images, and it was difficult to shoot according to the subject and the situation where the subject was placed.

  In order to solve these problems, it is necessary to conditionally prohibit or permit shooting for a specific subject. However, it is not easy for the photographer to know which subjects should not be photographed or distributed.

  SUMMARY OF THE INVENTION Accordingly, it is a primary object of the present invention to provide a photographing method that solves the above-described problems and that can photograph only what may be photographed.

In order to solve the above problems, the present invention is an image processing apparatus that processes a captured image in which a subject is captured,
The photographed image is created by the photographing means photographing the subject according to camera parameters including a camera position, orientation, and angle of view,
Based on the position of the photographing means, the position of the subject, and the camera parameters, it is calculated whether or not the subject is included in the captured image. Determined by a transmission point calculation means that uses the position of the subject as a transmission point, a replacement area determination means that determines a replacement area to be replaced in an image taken based on the transmission point, and a replacement area determination means A replacement image generating means for generating a replacement image for replacement in the replaced area, an image processing means for synthesizing the replacement image in the replacement area in the captured image, and a photographic image synthesized by the image processing means. And means for outputting an image. Thus, by setting an inconvenient image in the replacement area, only what can be copied can be taken.

The present invention is an image processing apparatus for processing a captured image in which a subject with a built-in or attached transmission device is captured,
The photographed image is created by the photographing means photographing the subject according to camera parameters including a camera position, orientation, and angle of view,
A receiving unit that receives a position specifying signal transmitted from the transmitting unit by the transmitting unit, a position of the photographing unit, a position of the transmitting unit calculated based on the position specifying signal, and the camera parameter And calculating whether or not the transmission device is included in the captured image, and if included, a transmission point calculation means that uses the position of the transmission device in the captured image as a transmission point; and A replacement area determining means for determining a replacement area for replacement in an image photographed based on the transmission point, and a replacement image for generating a replacement image for replacement in the replacement area determined by the replacement area determination means Generating means, image processing means for combining the replacement image in the replacement area in the captured image, and outputting the captured image combined by the image processing means Means that, characterized in that it is configured to include. Thus, by setting an inconvenient image in the replacement area, only what can be copied can be taken.

  In the present invention, the replacement area determination means arranges an image area having a shape defined by a predetermined shape and its size in the captured image so as to include the transmission point, and The replacement area is determined. As a result, the shooting area of the subject is replaced with another image, so that it is possible to prevent infringement of portrait rights, infringement of copyright, or infringement of privacy, etc. for a subject that does not wish to be shot.

  In the present invention, the replacement area determination means arranges an image area having a shape defined by a predetermined shape and its size in the captured image so as to include the transmission point, and the shape is determined from the captured image. An area excluding the image area is determined as the replacement area. Thereby, since the area other than the shooting area of the subject is replaced with another image, it is possible to remove the subject that the photographer does not want from the shot image.

  The present invention is characterized in that the replacement area determining means determines an area transmitted from the transmitting device as a replacement area. As a result, the transmission device can explicitly designate the replacement area, and the intention of photographing the subject can be clearly reflected in the photographed image.

  The present invention is characterized in that the replacement image generating means determines an image transmitted from the transmitting device as a replacement image. As a result, the transmission device can explicitly designate the replacement image, and the intention of photographing the subject can be clearly reflected in the photographed image.

According to the present invention, the image processing device includes at least one of a replacement region or a replacement image associated with profile information based on an ID transmitted from the transmission device and profile information stored in a storage unit. Decide
The profile information includes at least one information of an ID, an attribute, a contract status, a billing status, a preset setting, or a time or a location related to shooting. As a result, since the compositing process to the photographed image differs for each profile information set before photographing for the same subject, a differentiated photographing service can be provided for each user.

  The present invention is characterized in that the means for outputting the photographed image stores the photographed image in a storage means. Thus, by setting an inconvenient image in the replacement area, only what can be copied can be photographed and stored.

  The present invention is characterized in that the means for outputting the captured image delivers the captured image to a network. Thus, by setting an inconvenient image as a replacement area, only what can be copied can be photographed and distributed.

  The present invention is a photographing system in which a photographing device having the photographing unit, a receiving device having the receiving unit, and the image processing device are connected to each other via a network. Thereby, since each apparatus can be purchased separately, the existing apparatus can be diverted.

The present invention is an image processing method for processing a captured image in which a subject with a built-in or attached transmission device is captured,
The photographed image is created by photographing the subject according to camera parameters including a camera position, an orientation, and an angle of view.
Based on the procedure for receiving the position specifying signal transmitted from the transmitting apparatus, the position of the photographing means, the position of the transmitting apparatus calculated based on the position specifying signal, and the camera parameter It is calculated whether or not the transmission device is included in the captured image, and if included, the transmission point calculation procedure using the position of the transmission device in the captured image as the transmission point, and the transmission point A replacement area determination procedure for determining a replacement area to be replaced in the captured image, a replacement image generation procedure for generating a replacement image for replacement in the replacement area, and the replacement area in the captured image In addition, an image processing procedure for synthesizing the replacement image and a procedure for outputting a photographed image synthesized by the image processing means are executed. Thus, by setting an inconvenient image in the replacement area, only what can be copied can be taken.

  The present invention is an image processing program for causing a computer to execute the image processing method. Thus, by setting an inconvenient image in the replacement area, only what can be copied can be taken.

  The present invention is a computer-readable recording medium on which the program is recorded. Thus, by setting an inconvenient image in the replacement area, only what can be copied can be taken.

  The present invention is characterized in that an inconvenient image is set as a replacement area. Thereby, only what can be copied can be taken.

  Hereinafter, an embodiment of a photographing system to which the present invention is applied will be described in detail with reference to the drawings. FIG. 1 is a configuration diagram of a photographing system according to an embodiment of the present invention. The subject 10 includes or is provided with a transmission device 12. The subject 10 may be a moving object such as a person or a stationary object such as a work of art. And there is no specific restriction about the shape. The subject 10 is not limited to a single body within a predetermined shooting range, and a plurality of the subjects 10 may exist. In the case where a plurality of bodies are present in the same shooting range, the replacement processing of the subject 10 described later by the image processing device 40 may be repeated for each subject 10.

  Then, the transmission unit 14 of the transmission device 12 transmits information specifying the position of the transmission device 12 in order to inform the reception unit 32 of the position of the subject 10. The transmitter 14 may be a light emitter such as a beacon or marker using infrared light or visible light. In this case, the receiving unit 32 can use an infrared light receiving unit that can receive infrared rays or the same camera as the camera of the photographing unit 22. On the other hand, when the transmission means 14 uses visible light, the same camera as that of the photographing means 22 may be used as the camera as the reception means 32. That is, the transmission means 14 is paired with the reception means 32, and means for communicating signals of the same format is mounted. The reason why the transmitting device 12 and the transmitting means 14 are different is that the transmitting device 12 may have a means (for example, a camera) different from the means for transmitting a signal.

  Further, the image processing device 40 performs image processing on the image including the subject 10 that reflects the shooting intention of the subject 10. Therefore, the image processing apparatus 40 is configured as a computer including at least a memory serving as a storage unit used when performing arithmetic processing and an arithmetic processing device that performs the arithmetic processing. The memory is constituted by a RAM (Random Access Memory) or the like. Arithmetic processing is realized by an arithmetic processing unit configured by a CPU (Central Processing Unit) executing a program on a memory.

  Further, the image processing device 40 captures an image including the subject 10, an imaging unit 22 that is a camera that captures an image including the subject 10, and a camera parameter acquisition unit that acquires parameters at the time of imaging of the imaging unit 22. 24. The camera parameter is a parameter relating to the situation of the camera at the time of shooting (from which position and from which size and which size of image is taken), for example, the position and orientation of the camera, the angle of view, and the pixel. Information about numbers. These camera parameters can be acquired from, for example, setting information at the time of photographing by the photographing means 22.

  Then, in order to detect the position of the transmission device 12, the image processing device 40 receives the information specifying the position of the transmission device 12, and the transmission device 12 based on the signal from the transmission device 12. And position detecting means 34 for detecting the position. The position detection unit 34 identifies the place where the transmission device 12 is placed as viewed from the reception unit 32. For example, when a beacon is used for the transmission unit 14, the relative position of the beacon received by the reception unit 32 with respect to the reception unit 32 is detected.

  Further, the image processing device 40 is based on the transmission point calculation means 42 for calculating the position of the transmission device 12 in the captured image and the position of the transmission device 12 in order to perform the replacement process on the subject 10 in the captured image. A replacement area determination means 44 for determining a replacement area in the image captured in the image; a replacement image generation means 46 for generating a replacement image to be combined with the replacement area determined by the replacement area determination means 44; The image processing means 48 for combining the image photographed by the above and the replacement image is further provided.

  Here, the replacement image generation means 46 accesses the image DB 46a that provides the replacement image and the profile DB 46b that determines the conditions for searching for the replacement image in order to acquire the image that is the basis of the replacement image to be generated. .

  Then, in order to output the captured image after the replacement processing, the image processing device 40 is configured to record the image combined by the image processing unit 48 and the image combined by the image processing unit 48 on the network. And image transmitting means 52 for transmitting to

  FIG. 2 is a flowchart showing a photographing process and an image processing process by the photographing system. Hereinafter, it demonstrates along this flowchart.

  First, the transmitting device 12 continues to transmit a signal (position specifying signal) for specifying the position of the transmitting device 12 by the transmitting means 14 (S1).

  Next, the photographing unit 22 photographs the subject 10 when the user presses the photographing button (S2). Then, an image taken and captured by the photographing apparatus 20 is as shown in FIG. FIG. 3 shows an example in which the imaging region a1 includes the subject 10 with the transmitter 12 attached thereto. The shape of the subject 10 is assumed to be a human in FIG. 1, but in FIG. 3 and the like, for the sake of explanation of calculation processing, it will be abbreviated as a cylindrical object. In the present embodiment, the shape of the subject 10 is not necessarily limited to the person shown in FIG. 1 or the cylindrical object shown in FIG. 3 and the like, and the subject 10 having an arbitrary shape can be handled. . The replacement area b1 is an area calculated as described later in order to perform processing for replacing the image captured in the replacement area b1 with another image.

  And the camera parameter acquisition means 24 acquires the camera parameter when the imaging | photography means 22 image | photographed (S3).

  Furthermore, the receiving means 32 receives the position specifying signal transmitted in S1 (S4). For simplicity, FIG. 1 shows only one receiving means 32, but there may be a plurality of receiving means 32.

  Here, the position detecting means 34 calculates the position of the subject 10 (strictly speaking, the position of the transmitting device 12) when photographed in S2 based on the position specifying signal in S4 (S5). The position detection unit 34 identifies the place where the transmission device 12 is placed as viewed from the reception unit 32. That is, the position detection unit 34 calculates the relative position of the transmission device 12 with respect to the specific reception unit 32. The calculated position information of the transmission device 12 is sent to the transmission point calculation means 42. This position detection means 34 can be realized in various forms.

  For example, the transmission device 12 may transmit information including information indicating the absolute position of the transmission device 12 in the signal. At this time, the relative position of the transmitting device 12 with respect to the receiving means 32 is calculated from the absolute position information of the transmitting device 12 input from the receiving means 32 to the position detecting means 34 and the positional information of the receiving means 32. Note that the absolute position information set by the user in the transmission device 12 can be used as the absolute position information of the transmission device 12 when the position information is known in advance, such as when the transmission device 12 is stationary. In addition, the transmission device 12 may acquire preset absolute position information from a database or a server by communication. On the other hand, when the transmitting device 12 moves and it is difficult to predict the moving trajectory, the transmitting device 12 detects the position of the own device by GPS (Global Positioning System), and uses it as absolute position information.

  In addition to this, when the transmitting device 12 transmits using visible light, the position detecting unit 34 extracts a color region composed of color pixels peculiar to the illuminant from a photographed image input from the camera that is the receiving unit 32, It is possible to regard the position of the area in the captured image as the position of the transmission device 12.

  In the above description, when the distance between the photographing unit 22 and the receiving unit 32 is small, the photographing unit 22 and the receiving unit 32 are assumed to be at the same position. On the other hand, when the distance between the photographing unit 22 and the receiving unit 32 is large, the positional relationship between the photographing unit 22 and the transmission device 12 is calculated by the following method. First, the positional relationship between the photographing unit 22 and the transmission device 12 is obtained. For example, when the transmitting device 12, the photographing unit 22, and the receiving unit 32 are in the positional relationship as shown in FIG. 4, the relative position between the transmitting device 12 and the receiving unit 32 installed by the user is determined by the position specifying signal. Obtained as a vector p. Then, the relative position can be obtained as a vector q from the position of the photographing means 22 installed by the user or the like and the position of the receiving means 32 installed by the user or the like, and finally the relative of the photographing means 22 and the transmission device 12 can be obtained. The position can be calculated as vector p + vector q. For the sake of simplicity of explanation, the focal position of the camera of the photographing unit 22 is regarded as the position of the photographing unit 22, and the position of the transmission point of the transmission device 12 is regarded as the position of the transmission device 12. This is just an example.

  In addition, when the positional relationship between the photographing unit 22 and the receiving unit 32 is fixed in the image processing device 40, the information regarding the camera position of the camera parameter and the position of the receiving unit 32 is determined with respect to these predetermined relative positions. It can be replaced with information about. In that case, processing for detecting the positions of the photographing means 22 and the receiving means 32 is not necessary.

  The position detection unit 34 notifies the transmission point calculation unit 42 of information regarding the calculated position of the transmission device 12. At this time, if the positional relationship between the photographing unit 22 and the receiving unit 32 is not fixed, information on the absolute position of the receiving unit 32 or information on the relative position with respect to the photographing unit 22 is sent together with information on the position of the transmitting device 12. May be.

  Then, the transmission point calculation means 42 calculates the position (transmission point) of the transmission device 12 in the photographed image based on the position of the transmission device 12 detected in S5 and the camera parameter acquired in S3. (S6). The transmission point calculation means 42 uses the camera parameters acquired by the camera parameter acquisition means 24 and the position information of the transmission apparatus 12 input from the position detection means 34 to include the transmission apparatus 12 in the photographed image. (I.e., whether there is a transmission point) (S61).

  Hereinafter, the determination process (S61) of whether the transmission apparatus 12 is included is demonstrated using FIG. FIG. 5 is a diagram in which the image processing device 40 and the subject 10 are projected onto a three-dimensional space. FIG. 5 is created by projecting the camera parameters of S3 and the position of the transmitter 12 of S5 onto a three-dimensional space. Then, in this three-dimensional space, it is determined whether or not the transmission device 12 is included in the shooting area a1 in the image shot by the shooting unit 22. This determination can be performed by determining whether or not the relative position (vector p + vector q) of the transmission device 12 with respect to the imaging unit 22 is included in the imaging area a1. . At the same time, since the relative position of the transmission device 12 can be projected onto the coordinates of the imaging region a1, the coordinates of the transmission device 12 on the imaging region a1 can be obtained.

  If no transmitting device 12 is included (S61, N), the image replacement process does not need to be performed, so the image processing apparatus 40 outputs the captured image as it is (S10). Finish the process. On the other hand, when the transmission device 12 is included (S61, Y), the process proceeds to the following process S7.

  The replacement area determination unit 44 determines a replacement area (hereinafter referred to as a replacement area) in the captured image area based on the transmission point in the captured image calculated in S6 (S7). For this purpose, the coordinates of the transmission device 12 on the imaging area a1 are sent to the replacement area determination means 44. Then, the replacement area determination unit 44 determines the replacement area b1. Hereinafter, a method for preventing the specific subject 10 from being captured will be described.

  Note that the replacement area b1 can be determined as a replacement area b1 with an image area having a shape defined by a shape, size, and number of pixels, centered on the coordinates of the transmission device 12. The shape may be determined freely such as a rectangle, a circle, and a diamond.

  At this time, the size of the replacement area b <b> 1 can be changed according to the distance from the imaging unit 22 of the transmission device 12 that is obtained from the relative position of the transmission device 12 from the imaging unit 22. By this method, it is possible to determine a replacement area b1 having a size sufficient to include all of the subject 10 in which the transmission device 12 is incorporated or attached in the replacement area b1. The entire captured image may be used as the replacement area.

  As another method, the transmitting device 12 transmits in advance a shape indicating the size and shape of the replacement area according to the subject 10, and the shape information is input from the receiving means 32, and the replacement area b1 here There is also a way to do it.

  Further, when the transmitting device 12 transmits ID information, there is a method of determining the replacement area b1 using the ID information input from the receiving means 32. In this case, it is possible to adopt a method of obtaining the shape of the replacement area by using a search server prepared separately and using the ID information as a key via a network. Further, when a replacement image is selected using an ID described as the next step, the replacement region b1 can be determined as the shape of the image.

  In the above description, the method for preventing the specific subject 10 from being captured has been described. However, it is also possible to perform the same procedure so that only the specific subject 10 is captured and the background is not captured. In this case, the replacement area determination unit 44 may replace the area determined as the replacement area b1 with an area other than b1 of the captured image.

  Then, the replacement image generating unit 46 determines a replacement image to be combined with the replacement area in S7 (S8). For example, a replacement image defined in advance can be used. In addition, the replacement image may be generated by calculation using an image captured in the replacement area b1. For example, the image may be an average color of the replacement area b1, or an image obtained by calculation such as blurring or mosaic.

  There is also a method in which the transmission device 12 transmits a replacement image, and the replacement image is input from the receiving unit 32 and used as the replacement image here. In this case, it is possible to determine the shape of the replacement area b1 at the same time as determining the replacement image. In addition, the replacement image can be dynamically acquired as an image opposite to the image processing device 40 viewed from the transmission device 12 by the transmission device 12, and in this case, the replacement region b1 is as if the subject 10 is placed. It is possible to replace it with an image as if it were not.

  Furthermore, when ID information is transmitted from the transmitting device 12, there is a method of determining a replacement image using ID information input from the receiving means 32. The image processing apparatus 40 searches for a replacement image using the ID information as a key. At this time, a replacement image is selected from the image DB 46a built in the image processing apparatus 40, or a replacement image is obtained using a search server prepared separately via a network and using the ID information as a key. Can do.

  By adopting this method, it is possible to select a replacement image in accordance with the subject 10 and the surrounding situation, and it is possible to select a replacement image without a natural discomfort. Depending on the photographer's attributes and presetting, it is possible to replace the image reflecting the photographer's preference or replace the image distinguishing the photographer.

  When searching for these replacement images, correspondence information between the profiles recorded in the profile DB 46b and the replacement images is used. The profile includes, for example, time, place, photographer attribute, contract status with the photographer, billing status of the photographer, and settings related to replacement predetermined by the photographer. It is possible to search for a replacement image using these profile information and obtain a search result. It is also possible to determine whether or not the photographer has permission to shoot according to the contract status or billing status of the photographer, determine the size of the replacement area of the subject 10 by the photographer, or eliminate the replacement area by the photographer. It becomes possible. In the replacement image generated as described above, the replacement area b1 is sent to the image processing means 48.

  The image processing means 48 processes the image so as to replace the range in the replacement area in S7 with the replacement image in S8 with respect to the captured image in S2 (S9). That is, the image determined by the replacement image generation unit 46 is sent to the image processing unit 48, and the image processing unit 48 replaces the replacement area b1 of the captured image with the replacement image.

  The image processing device 40 outputs the image generated in S9 (S10). The output format is, for example, recorded as a captured image in the image recording unit 50, transmitted from the image transmission unit 52 to another device or server via a network, or disclosed via the Internet.

  The present invention described above can be widely modified without departing from the spirit thereof as follows.

  For example, the image processing apparatus 40 according to the present embodiment is not particularly limited in hardware format as long as it is used for capturing an image captured by a camera. For example, it may be realized as a camera device connected to a network, a recording device or a distribution device, or may be realized by a personal computer connected with a telephone or a camera. Further, the photographing means 22 may be configured to be usable regardless of a still image or a moving image as long as it is an image.

  In FIG. 1, the image processing apparatus 40 is configured to include the photographing unit 22, the receiving unit 32, and the unit related to image processing. It is good also as such a structure. FIG. 6 is an example in which the functions of the image processing device 40 in FIG. 1 are divided into three devices, that is, the image processing device 40, the imaging device 20, and the receiving device 30. These three devices are connected to each other via a network and exchange information.

  In this manner, by using the image processing device 40 and the receiving device 30 as separate housings, a configuration in which a plurality of receiving devices 30 are used as shown in FIG. 7 is possible. The network 9 connects each receiving device 30 and the image processing device 40, and the photographing device 20 and the image processing device 40, and may be wired or wireless. For example, the Internet can be used. In addition, a signal emitted from the transmission device 12 is received by the reception device 30. The route of this signal may be wired or wireless, but for example, a radio wave or a light wave can be used. Through the network 9, these pieces of information (images and position information of the transmission device 12) are sent to the image processing device 40.

  Furthermore, as an example of utilizing a plurality of receiving devices 30, an example in which the receiving devices 30 are arranged in a grid is shown in FIG. Here, the transmitter 12 can be mounted with a wireless tag called an RFID (Radio Frequency Identification) tag. An RFID tag is a device that transmits weak radio waves. For example, when there are a plurality of receiving devices 30, the position of the transmitting device 12 can be determined from the position of the receiving device 30 that can receive the radio wave transmitted from the RFID tag that is the transmitting device 12. That is, since the RFID tag has a narrow communication range, the operation position of the subject 10 can be covered by arranging the RFID tag in a grid pattern as shown in FIG.

  FIG. 9 is a layout diagram in which a plurality of receiving devices 30 are arranged in a lattice pattern in order to determine the position of the transmitting device 12. In FIG. 9, only the receiving device 30 a and the receiving device 30 b that are within the reach range c <b> 2 of the radio wave transmitted from the transmitting device 12 can receive the signal from the transmitting device 12. Therefore, as shown in FIG. 9, it can be seen that the position of the transmitting device 12 is in the vicinity of the receiving device 30 a and the receiving device 30 b, so that the center of gravity of the positions of the receiving device 30 a and the receiving device 30 b is roughly determined. Can be determined as the position of. To determine the position of the transmitting device 12 in more detail, measure the received radio wave intensity, determine the position according to it, specify the receiving direction using a highly directional antenna, and accurately transmit by triangulation The position of the device 12 can be determined.

  In addition, regarding the replacement area determination processing in S7, the transmission device 12 may specify a flag that specifies whether the replacement area includes the subject 10 or not. Compared to the method in which the transmitting device 12 transmits the replacement area itself, the amount of communication can be reduced, and important information such as whether or not photographing of the subject 10 is permitted can be transmitted reliably. . On the other hand, in the method in which the transmitting device 12 transmits the replacement area itself, the transmitting device 12 that grasps the shape of the subject 10 corresponding to the transmitting device 12 designates the replacement area. Specification is possible.

  Further, the communication method of the position specifying signal of S1 and S4 may be specified depending on whether or not the photographing of the subject 10 is permitted. For example, when the photographing of the subject 10 is permitted, the communication signal is reflected as noise in the photographed image by adopting a communication method such as an RFID tag so that the communication signal is not visible to human eyes. Can be prevented.

  In addition, when a communication method in which a communication signal is blocked by an obstruction such as infrared communication is employed, the communication signal is interrupted when the singer (subject 10) is hidden behind the stage. Impossibility can be regarded as movement of the position of the subject 10. As a result, the amount of calculation for position calculation can be reduced as compared with a method of always receiving communication signals.

  On the other hand, when the photographing of the subject 10 is not permitted, the transmitting means 14 that is visible to the human eye such as a light is effective. This is because the light may be reflected as noise in the captured image in order to erase the area of the subject 10, and since the mechanism is simpler than a method that implements a specific communication protocol, it is inexpensive. Can be realized. Further, when there is a sun or a display device that emits light in the area to be imaged, those can be used as a replacement area in the same manner as the subject 10.

  Further, regarding the communication processing of the position specifying signal in S1 and S4, instead of the image processing apparatus 40 directly receiving the position notification from the transmission apparatus 12, a means for the image processing apparatus 40 to grasp the position of the subject 10 is provided. It is good also as a structure. This eliminates the need for the subject 10 to have the transmitter 12.

It is a block diagram of the imaging | photography system regarding one Embodiment of this invention. It is a flowchart which shows the imaging | photography process and image processing process regarding one Embodiment of this invention. It is a figure which shows the picked-up image of the to-be-photographed object regarding one Embodiment of this invention. It is a figure which shows the positional relationship of the transmission device, imaging | photography means, and receiving means regarding one Embodiment of this invention. It is the figure which projected the image processing apparatus and the to-be-photographed object in the three-dimensional space regarding one Embodiment of this invention. 1 is a configuration diagram of an imaging system including three apparatuses, an image processing apparatus, an imaging apparatus, and a receiving apparatus according to an embodiment of the present invention. It is a block diagram which utilizes the some receiver regarding one Embodiment of this invention. It is a figure which shows an example which arrange | positions the receiver concerning one Embodiment of this invention in a grid | lattice form. It is a figure which shows an example which arrange | positions the receiver concerning one Embodiment of this invention in a grid | lattice form.

Explanation of symbols

DESCRIPTION OF SYMBOLS 9 Network 10 Subject 12 Transmission apparatus 14 Transmission means 20 Imaging apparatus 22 Imaging means 24 Camera parameter acquisition means 30 Reception apparatus 32 Reception means 34 Position detection means 40 Image processing apparatus 42 Transmission point calculation means 44 Replacement area determination means 46 Replacement image generation means 46 46a Image DB
46b Profile DB
48 Image processing means 50 Image recording means (storage means)
52 Image transmission means

Claims (13)

An image processing apparatus for processing a photographed image obtained by photographing a subject,
The photographed image is created by the photographing means photographing the subject according to camera parameters including a camera position, orientation, and angle of view,
Based on the position of the photographing means, the position of the subject, and the camera parameters, it is calculated whether or not the subject is included in the captured image. Determined by a transmission point calculation means that uses the position of the subject as a transmission point, a replacement area determination means that determines a replacement area to be replaced in an image taken based on the transmission point, and a replacement area determination means A replacement image generating means for generating a replacement image for replacement in the replaced area, an image processing means for synthesizing the replacement image in the replacement area in the captured image, and a photographic image synthesized by the image processing means. An image processing apparatus comprising: means for outputting an image. An image processing device that processes a captured image in which a subject with a built-in or attached transmission device is captured,
The photographed image is created by the photographing means photographing the subject according to camera parameters including a camera position, orientation, and angle of view,
A receiving unit that receives a position specifying signal transmitted from the transmitting unit by the transmitting unit, a position of the photographing unit, a position of the transmitting unit calculated based on the position specifying signal, and the camera parameter And calculating whether or not the transmission device is included in the captured image, and if included, a transmission point calculation means that uses the position of the transmission device in the captured image as a transmission point; and A replacement area determining means for determining a replacement area for replacement in an image photographed based on the transmission point, and a replacement image for generating a replacement image for replacement in the replacement area determined by the replacement area determination means Generating means, image processing means for combining the replacement image in the replacement area in the captured image, and outputting the captured image combined by the image processing means The image processing apparatus characterized by being configured to include means that the.   The replacement area determining means arranges an image area of a shape defined by a predetermined shape and its size in the captured image so as to include the transmission point, and uses the image area of the shape as the replacement area. The image processing apparatus according to claim 2, wherein the determination is performed.   The replacement area determining means arranges an image area of a shape defined by a predetermined shape and its size in the captured image so as to include the transmission point, and determines the image area of the shape from the captured image. The image processing apparatus according to claim 2, wherein a region to be excluded is determined as the replacement region.   The image processing apparatus according to claim 2, wherein the replacement area determination unit determines an area transmitted from the transmission apparatus as a replacement area.   The image processing apparatus according to claim 2, wherein the replacement image generation unit determines an image transmitted from the transmission apparatus as a replacement image. The image processing device determines at least one of a replacement region or a replacement image associated with the profile information based on the ID transmitted from the transmission device and the profile information stored in the storage unit. ,
The profile information includes at least one information of an ID, an attribute, a contract status, a billing status, a preset, or a time or a location related to a photographer related to a photographer. The image processing apparatus according to claim 2.   The image processing apparatus according to claim 2, wherein the unit that outputs the captured image stores the captured image in a storage unit.   The image processing apparatus according to claim 2, wherein the means for outputting the captured image distributes the captured image via a network.   An imaging system in which an imaging apparatus having the imaging means, a receiving apparatus having the receiving means, and the image processing apparatus according to any one of claims 2 to 9 are connected to each other via a network. An image processing method for processing a captured image in which a subject with a built-in or attached transmission device is captured,
The photographed image is created by the photographing means photographing the subject according to camera parameters including a camera position, orientation, and angle of view,
The computer receives the position specifying signal transmitted from the transmitting device, the position of the photographing means, the position of the transmitting device calculated based on the position specifying signal, and the camera parameter. Whether or not the transmission device is included in the captured image, and if included, the transmission point calculation procedure using the position of the transmission device in the captured image as the transmission point, and the transmission A replacement area determination procedure for determining a replacement area for replacement in an image captured based on a point; a replacement image generation procedure for generating a replacement image for replacement in the replacement area; and An image processing procedure for synthesizing the replacement image in the replacement area and a procedure for outputting the captured image synthesized by the image processing means to the storage means are executed. Image processing method according to claim.   An image processing program for causing a computer to execute the image processing method according to claim 11. A computer-readable recording medium on which the program according to claim 12 is recorded.

Images