JP2011097238A - Terminal device with privacy protection image processing function, and method and program therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a terminal device with a privacy protection image processing function to which blurring surely applies to the terminal device which requires blurring beforehand, and editing after photographing by hand is not necessary. <P>SOLUTION: A portable terminal which photographs an image, processes the photographed image data, and takes it out in frame units. The presence of a marker in the taken-out image data is recognized. When the marker recognition means recognizes the marker, it calculates a position of the marker, and carries out image processing by predetermined processing for the image data based on the calculated position. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to protection of privacy and the like when shooting moving images and still images in a terminal such as a mobile phone.

  In recent years, a mobile device such as a mobile phone has a camera, and it has become possible to easily take still images and moving images. On the other hand, portrait rights and privacy in still images and moving images are becoming problems. Therefore, it is necessary to edit the photographed still image or moving image later and insert blur and mosaic.

  In addition, there is a technique in which editing is performed while shooting is performed instead of editing later, and an image including blur and mosaic is acquired.

  An example of such a technique is described in Patent Documents 1 and 2, for example.

  In the technique described in Patent Document 1, a region of a subject and a region other than the subject are separated from the captured image, and at least a region other than the subject is subjected to blur processing.

  In the technique described in Patent Document 2, mosaic processing is performed when a moving object is recognized from a captured image.

JP 2009-206585 A JP 2008-042595 A

  As described above, by using techniques such as Patent Documents 1 and 2, it is possible to acquire an image including blur and mosaic while performing shooting. However, in these techniques, it is determined whether or not to perform blurring or the like on the basis of whether or not the subject is a moving object or not. For this reason, processing such as blurring is not necessarily performed only for a portion that meets the user's intention. For example, processing such as blurring is not always performed when a photograph is prohibited at a sightseeing spot or when a movie theater screen is shot.

  In addition, there are cases where a person who does not want to be photographed wants to protect himself / herself from being photographed. However, the above-described technique has not always fulfilled such a desire.

  Furthermore, editing later is time-consuming and requires editing software and specialized knowledge, which is very heavy for a general user who does not have specialized knowledge.

  Therefore, the present invention is a terminal device having a privacy protection image processing function, which is always subjected to blurring and the like, and it is not necessary to perform human editing after shooting, for those which are determined to be blurry in advance. It is an object to provide a method and a program thereof.

  According to the first aspect of the present invention, in a portable terminal capable of capturing an image, an image processing means that processes the captured image data and extracts the data in units of frames, and a marker is included in the extracted image data. A marker recognizing unit for recognizing whether or not the marker is recognized, and when the marker recognizing unit recognizes the marker, the position of the marker is calculated, and processing determined based on the calculated position is performed on the image data. There is provided a portable terminal comprising image processing means for performing image processing.

  According to a second aspect of the present invention, in an image processing system comprising a marker used for image processing and a mobile terminal that performs image processing using the marker, the mobile terminal is configured as described above. An image processing system characterized by being a portable terminal is provided.

  According to a third aspect of the present invention, in an image processing method in a mobile terminal capable of capturing an image, an image processing step of processing the captured image data and extracting it in frame units, and the extracted image A marker recognition step for recognizing whether a marker is present in the data, and when the marker recognition step recognizes the marker, the position of the marker is calculated, and a process determined based on the calculated position is performed on the image data And an image processing step for performing image processing.

  According to a fourth aspect of the present invention, in an image processing program incorporated in a portable terminal capable of taking an image, image processing means for processing the data of the taken image and taking out the frame unit, and the taking out A marker recognizing unit for recognizing whether or not a marker is present in the image data, and when the marker recognizing unit recognizes the marker, the position of the marker is calculated, and the process determined based on the calculated position is An image processing program is provided that causes a computer to function as a portable terminal including image processing means that performs image processing by performing the processing on image data.

  According to the present invention, when shooting a still image or a moving image, a blur or a mosaic can be added without editing in real time at a location and size intended by the user. For this reason, blur and mosaic can be applied to objects other than the target object that the user wants to shoot, and the privacy of other people, cars, etc. can be protected.

It is a figure showing the basic composition of the embodiment of the present invention. It is a figure showing the fundamental composition of the treating part in the embodiment of the present invention. It is a flowchart showing the basic operation | movement of embodiment of this invention. It is an image figure (1/2) for demonstrating the basic operation | movement of embodiment of this invention. It is an image figure (2/2) for demonstrating the basic operation | movement of embodiment of this invention.

  First, an outline of an embodiment of the present invention will be described. The embodiment of the present invention generally outlines a blur or mosaic at a marker position when a specific marker covers a still image or a moving image when shooting a still image or a moving image on a terminal such as a mobile phone. It is added to a moving image to protect privacy and the like. Specifically, the video shot by the camera is always analyzed, and it is confirmed whether there is a marker in the video. When a marker is present in the camera image, the position and orientation of the marker are specified. The blur and mosaic set by the user are added according to the position and orientation of the marker. By this blur and mosaic, the blur and mosaic are added in real time, and privacy protection is realized.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

  A block diagram of FIG. 1 shows a portable terminal 100 according to an embodiment of the present invention. In addition, although the portable terminal 100 which is this embodiment is a mobile phone, for example, this does not limit the application object of this embodiment. This embodiment can be applied to devices such as PHS (Personal Handy-phone System) and PDA (Personal Digital Assistant) in addition to mobile phones. In addition, it can be applied to, for example, a digital camera.

  Referring to FIG. 1, the mobile terminal 100 includes a memory 111, a central processing unit 112, an operation reception unit 121, a photographing unit 122, and a display unit 123. The present embodiment further includes a marker 200.

  Here, the processing unit 110 is realized by the cooperation of the memory 111 and the central processing unit 112. Specifically, the processing unit 110 is realized by the central processing unit 112 reading and executing programs and data stored in the memory 111.

  The processing unit 110 will be described with reference to the block diagram of FIG. Referring to FIG. 2, the processing unit 110 implements a moving image processing unit 113, a marker recognition unit 114, and a video processing unit 115.

  The moving image processing unit 113 has a function of processing the moving image data of the video imaged by the imaging unit 114 and extracting each frame. Furthermore, the moving image processing unit 113 has a function of extracting moving objects.

  The marker recognizing unit 114 has a function of checking whether the marker 200 is present in the image for each frame extracted by the moving image processing unit 113.

  The video processing unit 115 has a function of adding blur and mosaic to the video according to the type and size of blur and mosaic determined based on the calculated marker information. The image processing unit 115 corresponds to the “image processing unit” of the present invention. The processing for adding the blur and the mosaic corresponds to the operation of “image processing” of the present invention.

  The operation reception unit 121 is, for example, an input key, and is used by the user to operate the mobile terminal 100. The operation receiving unit 121 may be realized by any mechanism. It may be realized as a key, a touch panel, or another device.

  The imaging unit 122 is a camera that can capture still images and moving images.

  The display unit 123 is a display that processes and displays a video image captured by the imaging unit 122.

  The memory 111 is a portion that is connected to input / output devices such as the operation receiving unit 121, the photographing unit 122, and the display unit 123, and exchanges data.

  The marker 200 is a target to which blur and mosaic are added in the captured image of the imaging unit 122. In the present description, the marker 200 is illustrated as a rectangle, but the marker 200 is not a specific image, and the design and the like can be freely changed.

  Next, the operation of this embodiment will be described using the flowchart of FIG. 3 and the image diagrams of FIGS. 4 and 5 exemplify the case where the person 300 is moving and the person 300 is moving, the person 300 is merely an example. It is possible to target any object such as an animal other than a person, a car, or a stationary object such as a building.

  As a premise of the operation, the marker 200 is arranged in advance at an actual position where blur and mosaic are to be generated. The size and type of blur and mosaic displayed on the marker 200 can be set for the mobile terminal 100 such as a mobile phone, and the setting information is stored in the memory 111. Note that the information to be displayed may be only blurry, only mosaic, or a combination thereof. Furthermore, you may change the color and brightness of the image of an applicable location. For example, if the distance from the marker 200 is short, the blur and the mosaic may be enlarged.

  The mobile terminal 100 such as a mobile phone that has received the user's operation captures the surrounding scenery by the image capturing unit 114 (step S401).

  The video imaged by the imaging unit 114 is processed by the moving image processing unit 113 of the central processing unit 112, and the moving image data is processed to extract one frame at a time (step S402).

  The image for each frame is checked by the marker recognition unit 114 to see if the marker 200 is present in the image (step S403).

  If the marker 200 is not found in the image (NO in step S403), no processing is performed and the image is displayed on the display unit 115 (step S404).

  If the marker 200 is in the image (YES in step S403), the position, size, and orientation of the marker in the image are calculated (step S405).

  Based on the calculated marker information, display information such as the type, size, and orientation of the blur and mosaic to be displayed on the video is determined (step S406).

  In the image processing unit 115, blur and mosaic are added to the image (step S407).

  The added video is displayed on the display unit 115 (step S408).

  Through the above operation, the present embodiment can edit a blur or a mosaic in real time at a location and size intended by the user (or manufacturer of the mobile terminal 100, a store, etc.) when shooting a still image or a moving image. Will be able to add. As a result, it is possible to protect the privacy of other people, cars, etc. by applying blur and mosaic other than the target object to be photographed.

  Next, it can be assumed that a person, a vehicle, or the like passes in front of the marker 200 displayed during the above-described operation and the marker is interrupted. In such a case, the following three settings are possible.

  Setting 1. Since no marker is displayed, no blur or mosaic is displayed for the blocked marker.

  Setting 2. Since the position of the marker can be estimated even if it is blocked, the blockage is ignored and a blur or a mosaic is displayed for the estimated position of the marker. As a result, even objects that have blocked the marker are subject to blur and mosaic.

  Setting 3. Since the marker position can be estimated even if it is blocked, it is ignored and the blur or mosaic is displayed for the estimated position of the marker. Unlike “Setting 2.”, the marker is displayed. Only objects that have been obstructed are removed from the object of blur or mosaic.

  Next, FIG. 5 shows an operation when a moving object (such as a person) is blocked in front of the marker and is set as the above-mentioned “setting 3.”. In FIG. 5, the plurality of markers 200 are collectively referred to as a marker group 201.

  The video imaged by the imaging unit 114 is processed by the moving image processing function 113 of the central processing unit 112, and the moving image data is processed and taken out frame by frame. The image for each frame is checked by the marker recognizing unit 114 to see if the marker 200 is in the image.

  When the marker 200 suddenly disappears, it is compared with the previous image to determine whether the marker 200 has been removed due to the movement of the photographing unit 114 or whether the marker 200 has been blocked by the moving object.

  When the marker 200 is blocked by the moving object, the moving image processing unit 113 extracts the moving object.

Based on the above information, the video processing unit 115 does not blur or mosaic only the moving object,
A blur or a mosaic is added to a portion other than the moving object and displayed on the display unit 115.

  Furthermore, if a function for detecting a face image is further added and the moving object is determined to be a face image, the moving object that seems to be the face image may be set to be blurred or mosaiced. Good.

  As specific effects when the embodiment of the present invention described above is applied, for example, the following effects can be considered.

  As an example of the first effect, if you happened to walk on the road, even if you were shooting a still image or a moving image somewhere, wearing a marker, There are things that can prevent blurs and mosaics from entering the captured image.

  As an example of the second effect, it is possible to prevent voyeurism, such as sightseeing spots, etc., where photography is prohibited, and by placing a marker on the place, etc., even if it is taken, the photograph does not move cleanly The thing is mentioned.

  As an example of the third effect, it is possible to prevent voyeurism by displaying it in the vicinity of a movie theater screen.

  The terminal device having the privacy protection image processing function according to the embodiment of the present invention can be realized by hardware, but a program for causing a computer to function as the terminal device having the privacy protection image processing function Can also be realized by reading from a computer-readable recording medium and executing it.

  Also, the privacy-protected image processing method according to the embodiment of the present invention can be realized by hardware, but the computer reads a program for causing the computer to execute the method from a computer-readable recording medium and executes the program. Can also be realized.

  Moreover, although the above-described embodiment is a preferred embodiment of the present invention, the scope of the present invention is not limited only to the above-described embodiment, and various modifications are made without departing from the gist of the present invention. Implementation in the form is possible.

DESCRIPTION OF SYMBOLS 100 Mobile terminal 110 Processing part 111 Memory 112 Central processing unit 113 Moving image processing part 114 Marker recognition part 115 Image processing part 121 Operation reception part 122 Image | photographing part 123 Display part 200 Marker 201 Marker group 300 Person

Claims (16)

On mobile devices that can take images,
Image processing means for processing the data of the captured image and extracting it in units of frames;
Marker recognition means for recognizing whether a marker is present in the extracted image data;
When the marker recognition unit recognizes the marker, the image processing unit calculates the position of the marker and performs image processing by performing a process determined based on the calculated position on the image data;
A portable terminal comprising: The mobile terminal according to claim 1,
The image processing means calculates the size and direction of the marker in addition to the position of the marker, and performs processing determined on the image data based on the calculated position, size and direction. A portable terminal characterized by processing. The mobile terminal according to claim 1 or 2,
If the marker suddenly disappears between successive frames during the processing, the moving image processing means moves because the marker has been removed from the imaging target due to the movement of the mobile terminal. It is determined whether or not the marker is blocked by the presence of an object, and if the marker is blocked by the presence of a moving object as a result of the determination, A portable terminal that performs a predetermined process on the image data. The mobile terminal according to claim 1 or 2,
If the marker suddenly disappears between successive frames during the processing, the moving image processing means moves because the marker has been removed from the imaging target due to the movement of the mobile terminal. It is determined whether or not the marker is blocked by the presence of an object, and if the marker is blocked by the presence of a moving object as a result of the determination, the marker is based on the position where the marker is estimated to exist. A portable terminal, wherein a predetermined process is performed on the image data, but the process is not performed on the image data of the moving object. The mobile terminal according to any one of claims 1 to 4,
The portable terminal characterized in that the image processing is adding any one or a combination of mosaic and blur to the image data. In an image processing system comprising a marker used for image processing and a mobile terminal that performs image processing using the marker,
The image processing system, wherein the portable terminal is the portable terminal according to any one of claims 1 to 5. In an image processing method in a mobile terminal capable of taking an image,
An image processing step of processing the data of the captured image and extracting it in units of frames;
A marker recognition step for recognizing whether a marker is present in the extracted image data;
When the marker recognition step recognizes the marker, an image processing step of calculating the position of the marker and performing image processing by performing a process determined based on the calculated position on the image data;
An image processing method comprising: The image processing method according to claim 7.
In the image processing step, in addition to the position of the marker, the size and direction of the marker are calculated, and the image data is subjected to processing determined based on the calculated position, size, and direction. An image processing method characterized by performing processing. The image processing method according to claim 7 or 8,
In the moving image processing step, if the marker suddenly disappears between successive frames during the processing, the reason is that the marker has been removed from the imaging target due to the movement of the mobile terminal. It is determined whether or not the marker is blocked by the presence of an object, and if the marker is blocked by the presence of a moving object as a result of the determination, the marker is based on the position where the marker is estimated to exist. An image processing method, wherein a predetermined process is performed on the image data. The image processing method according to claim 7 or 8,
In the moving image processing step, if the marker suddenly disappears between successive frames during the processing, the reason is that the marker has been removed from the imaging target due to the movement of the mobile terminal. It is determined whether or not the marker is blocked by the presence of an object, and if the marker is blocked by the presence of a moving object as a result of the determination, the marker is based on the position where the marker is estimated to exist. An image processing method, wherein a predetermined process is performed on the image data, but the process is not performed on the image data of the moving object. The image processing method according to any one of claims 7 to 10,
The image processing method is characterized in that any one or a combination of mosaic and blur is added to the image data. In an image processing program incorporated in a mobile terminal capable of taking an image,
Image processing means for processing the data of the captured image and extracting it in units of frames;
Marker recognition means for recognizing whether a marker is present in the extracted image data;
When the marker recognition unit recognizes the marker, the image processing unit calculates the position of the marker and performs image processing by performing a process determined based on the calculated position on the image data;
An image processing program for causing a computer to function as a portable terminal comprising: In the image processing program according to claim 12,
The image processing means calculates the size and direction of the marker in addition to the position of the marker, and performs processing determined on the image data based on the calculated position, size and direction. An image processing program characterized by performing processing. In the image processing program according to claim 12 or 13,
The moving image processing means, when the marker suddenly disappears between successive frames when performing the processing, the reason is because the marker has been removed from the shooting target by moving the image processing program, It is determined whether the marker has been blocked by the presence of a moving object. An image processing program for performing the predetermined processing on the image data. In the image processing program according to claim 12 or 13,
The moving image processing means, when the marker suddenly disappears between successive frames when performing the processing, the reason is because the marker has been removed from the shooting target by moving the image processing program, It is determined whether the marker has been blocked by the presence of a moving object, and if the marker is blocked by the presence of a moving object as a result of the determination, it is based on the position where the marker is presumed to exist. An image processing program that performs the predetermined processing on the image data, but does not perform the processing on the image data of the moving object. The image processing program according to any one of claims 12 to 15,
The image processing program is characterized in that any one or a combination of mosaic and blur is added to the image data.

Images