JP2005210324A - Image abstraction processing program and recording medium - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image abstraction processing program or the like, capable of transmitting a live video image to a display apparatus, such as a monitor, without violating the individual's right of portrait or the privacy of the individual in the live video image, modifying the live video image as if to be a video image photographed remotely or degrading the resolution of the live video image lower than a prescribed resolution. <P>SOLUTION: A computer applies characteristics extraction and area division to a photographed image with an imaging apparatus to distinguish a portrait image and a background image, applies a prescribed person abstraction processing to the distinguished portrait image to acquire an abstracted portrait image, applies a prescribed background abstraction processing to the background image to acquire the abstracted background image, and combines the acquired abstracted portrait image with the acquired abstracted background image. The image of a desired animation character can be used for the abstracted portrait image, and an image painted in the same color as the color of the background image can be used for the abstracted background image. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to an image abstraction processing program for performing abstraction processing on an image captured by an imaging apparatus.

Conventionally, various live images are captured by television (TV) broadcasting or the like, sent to various display devices such as monitors, and used as life information. Here, the life information is, for example, traffic jam information or weather information on a road passing by commuting. In particular, it is anticipated that the live video distribution service for living information will be expanded in the future in car navigation systems, and it is assumed that the live information live video will be projected on display devices such as monitors for car navigation devices in the near future. Yes. In TV broadcasts and the like, as described in Non-Patent Document 1, an image captured at a predetermined resolution is transmitted.
Shin Hasegawa, “Revised Image Engineering”, pages 114-116, 4th edition, revised on December 20, 1994, Corona Corporation.

  In the live video mentioned above, there are cases where humans or cars in traffic are photographed, and if this live video is directly transmitted to a display device such as a monitor, personal portrait rights or privacy may be infringed. was there. For this reason, an image obtained by photographing a passing person or a car from a distance or a blurred image reduced to a resolution lower than the predetermined resolution is transmitted to a display device such as a monitor. However, when these images are viewed on a display device such as a monitor, there is a problem in that they are hindered from being used as life information because they are images from a distant location or unclear images.

  Accordingly, an object of the present invention is to solve the above-described problems, and does not infringe on the individual's portrait rights or privacy in the live video, and the live video is taken from a distance. Another object of the present invention is to provide an image abstraction processing program or the like that can be transmitted to a display device such as a monitor without dropping to a resolution lower than a predetermined resolution.

  An image abstraction processing program of the present invention is an image abstraction processing program for performing abstraction processing on an image captured by an imaging device, and extracts a feature from an image captured by the imaging device on a computer And an image recognition step for performing region division to identify a person image and a background image, and performing a predetermined person abstraction process on the person image identified in the image recognition step to obtain an abstracted person image , An abstraction processing step for obtaining a background image abstracted by performing a predetermined background abstraction processing on the background image, an abstracted person image and an abstracted background image obtained in the abstraction processing step Is an image abstraction processing program for executing a synthesis processing step for synthesizing.

  Here, in the image abstraction processing program according to the present invention, the person image processing in the abstraction processing step includes a predetermined feature amount of the predetermined feature point extracted in the image recognition step and a person image feature amount recording in advance. A person type identifying step for identifying a type of the person image identified in the image recognition step by comparing with a predetermined feature amount at a predetermined feature point of the person image recorded in the section, and identifying in the image recognition step And a person replacement processing step of replacing the person image thus identified with an abstracted person image corresponding to the type specified in the person type specifying step.

  Here, in the image abstraction processing program of the present invention, the background abstraction processing in the abstraction processing step includes a predetermined feature amount of a predetermined feature point extracted in the image recognition step and a background image feature amount in advance. In a background type specifying step for comparing a predetermined feature amount at a predetermined feature point of a background image recorded in the recording unit and specifying a type of the background image identified in the image recognition step, and in the image recognition step A background replacement processing step of replacing the identified background image with an abstracted background image corresponding to the type specified in the background type specifying step.

  Here, in the image abstraction processing program according to the present invention, the abstracted person image may be an animation character image.

  Here, in the image abstraction processing program of the present invention, the abstracted background image can be an image painted in the same color as the background image.

  The recording medium of the present invention is a computer-readable recording medium on which any of the image abstraction processing programs of the present invention is recorded.

  The recording medium of the present invention has a data structure that holds combined data obtained by performing an abstraction process on an image captured by an imaging device and predetermined tag information, and the data structure is An image recognition step for identifying a person image and a background image by performing feature extraction and area division on the image captured by the imaging device, and a predetermined person abstraction for the person image identified in the image recognition step An abstraction processing step for obtaining an abstracted person image by performing processing, and applying a predetermined background abstraction processing to the background image to obtain an abstracted background image, and obtained by the abstraction processing step. The synthesized data obtained by executing the synthesis processing step for synthesizing the abstracted person image and the abstracted background image with predetermined tag information A computer readable recording medium recording a data structure, characterized in that the holding in association with tag information associated with the personal image or background image I.

  According to the image abstraction processing program of the present invention, a computer performs feature extraction and area division on an image captured by an imaging device to identify a person image and a background image. Perform a specified person abstraction process to obtain an abstracted person image, and apply a predetermined background abstraction process to the background image to obtain an abstracted background image. The image and the abstract background image are combined. An image of a desired animation / character is used as the abstracted person image, and an image obtained by painting the background image with the same color as the color is used as the abstracted background image. As a result, it is possible to display on a monitor or the like without infringing on the individual's portrait rights or privacy in the live video, and without making the live video shot from a distance or dropping it to a resolution lower than the predetermined resolution. There is an effect that it can be transmitted to the apparatus.

  Hereinafter, each embodiment will be described in detail with reference to the drawings.

  FIG. 1 is a flowchart showing the processing flow of an image abstraction processing program according to the first embodiment of the present invention. As shown in FIG. 1, feature extraction and region segmentation are performed on an image captured by an imaging device (not shown) to identify a person image and a background image (image recognition step, step S10). Here, it is assumed that the imaging device captures live images of roads and passersby in a city, for example. For feature extraction, edge detection and pixel density gradient in the prior art can be used, and for area segmentation, the area expansion method and the statistical hypothesis test method in the prior art can be used.

  Next, the person image identified in the image recognition step (step S10) is subjected to a predetermined person abstraction process (steps S18 and S20) to obtain an abstracted person image, and the background image has a predetermined background. Abstraction processing (steps S14 and S16) is performed to obtain an abstracted background image (abstraction processing step, step S12).

  In the person abstraction process described above, a predetermined feature amount of the predetermined feature point extracted in the image recognition step (step S10) and a predetermined image of the person image recorded in advance in the person image feature amount recording unit 10 (described later). Is compared with a predetermined feature quantity at the feature point of the person, and the type of the person image identified in the image recognition step (step S10) is specified (person type specifying step; step S18). FIG. 2 illustrates the contents of the person image feature amount recording unit 10. As shown in FIG. 2, the person image feature amount recording unit 10 includes various feature points P1 (12), P2 (14),. . . , Pn (16), etc., feature amounts F11, F12,. . . , F1n and the like. As the feature point P1 or the like, for example, the top of the person, the left and right eyebrows, the left and right eyes, the top of the nose, the bottom of the nose, the top of the ear, the bottom of the ear, the contour of the face, and the like can be used. As the feature amount F11 or the like, for example, the two-dimensional position coordinates of each feature point or the area occupied by each feature point may be used. The feature quantity such as the feature point P1 extracted in the image recognition step (step S10) is compared with the feature quantity F11 etc. in the feature point P1 etc. of the person image recorded in the person image feature quantity recording unit 10 in advance. The person type 18 corresponding to the row in the person image feature amount recording unit 10 having the largest number of matching feature values within a predetermined allowable range can be specified as the person image type. For example, if the feature values of the feature points P1 to Pn of a certain person image coincide with F11 to F1n within a predetermined allowable range, the person type 18 of the person image is “A1 (adult, male, age Y1, body type S1”. Etc.) ”. The person type 18 may include a face type or the like. Here, A1 is a type code for identifying the person type 18.

  Subsequently, the person image identified in the image recognition step (step S10) is replaced with an abstracted person image corresponding to the person type 18 specified in the person type specifying step (step S18) (person replacement processing step). Step S20). The abstracted person image is preferably an animation character image, for example. Specifically, the face type in the type code A1 may be replaced with an image of a desired animation character according to a round face, a square face, an egg face, or the like. It is also possible to code a desired animation character and record it in the person image feature amount recording unit 10 in association with the type code.

  On the other hand, in the background abstraction processing described above, the predetermined feature amount of the predetermined feature point extracted in the image recognition step (step S10) and the background image recorded in advance in the background image feature amount recording unit 20 (described later). Are compared with a predetermined feature amount at the predetermined feature point to specify the type of the background image identified in the image recognition step (step S10) (background type specifying step; step S14). FIG. 3 illustrates the contents of the background image feature amount recording unit 20. As illustrated in FIG. 3, the background image feature amount recording unit 20 includes various feature points Q1 (22), Q2 (24),. . . , Qn (26), etc., for each feature quantity G11, G12,. . . , G1n and the like. As the feature point Q1, etc., for example, the edge or center of the background can be used. As the feature amount G11 and the like, for example, the two-dimensional position coordinates of each feature point, the area or color occupied by each feature point, and the like can be used. The feature amount such as the feature point Q1 extracted in the image recognition step (step S10) is compared with the feature amount G11 or the like in the feature point Q1 or the like of the background image recorded in the background image feature amount recording unit 20 in advance. The background type 28 corresponding to the row in the background image feature value recording unit 20 having the largest number of matching feature values within a predetermined allowable range can be specified as the background image type. For example, when the feature amounts of the feature points Q1 to Qn of a certain background image coincide with Gk1 to Gkn within a predetermined allowable range, the background type 28 of the background image is specified as “Bk (car)”. The background type 28 may include other types. Here, Bk is a type code for identifying the type.

  Subsequently, the background image identified in the image recognition step (step S10) is replaced with an abstracted background image corresponding to the background type 28 specified in the background type specifying step (step S14) (background replacement processing step. Step S16). The abstracted background image is preferably an image filled with the same color as the background image, for example. The color of the background image may be a color that matches the feature amount G11 such as the feature point Q1 in the background image feature amount recording unit 20 within a predetermined allowable range. Specifically, when a certain background image matches the type code Bk (car), it is filled with a color (white etc.) such as the feature amount G11 such as the feature point Q1 indicating the color in the row of the type code Bk (car). Replace with an image.

  Finally, the abstracted person image acquired in the abstraction processing step (steps S14 and S16 or steps S18 and S20) and the abstracted background image can be combined to obtain composite data (composite data). Processing step, step S22).

  As described above, according to the first embodiment of the present invention, the computer performs feature extraction and region segmentation on the image captured by the imaging device to identify the person image and the background image, and the identified person image Performs a predetermined person abstraction process to obtain an abstracted person image, and applies a predetermined background abstraction process to the background image to obtain an abstracted background image. The person image and the abstracted background image can be synthesized. An image of a desired animation / character can be used as the abstracted person image, and an image obtained by painting the background image with the same color as the color can be used as the abstracted background image. For this reason, it does not infringe individual portrait rights or privacy in live video, and displays on a monitor etc. without making the live video shot from a distance or dropping it to a resolution lower than the predetermined resolution An image abstraction processing program or the like that can be transmitted to the apparatus can be provided.

  A data structure in which the synthesized data obtained in the synthesizing process step (step S22) in the first embodiment is associated with predetermined tag information can be held in a computer-readable recording medium. That is, the data structure allows a computer to perform feature extraction and region segmentation on an image captured by an imaging device to identify a person image and a background image, and the identified person image has a predetermined person abstraction. Process to obtain an abstracted person image, apply a predetermined background abstraction process to the background image to obtain an abstracted background image, and obtain the abstracted person image and abstraction It is possible to have a data structure in which synthesized data obtained by synthesizing the background image and the predetermined tag information related to the person image or the background image are stored in association with each other. As the predetermined tag information, for example, an XML (Extensible Markup Language) tag can be used.

  As described above, according to the second embodiment of the present invention, a data structure in which synthesized data is associated with, for example, an XML tag can be recorded on a computer-readable recording medium. As a result, data mining and filtering operations are facilitated by using the tag on the display device side such as a monitor transmitted from the imaging device side to the data structure recorded on the recording medium. Can be easily selected. Further, by using the tag in the same manner on the above-described imaging apparatus side, it becomes easy to select the composite data to be transmitted according to the delivery time, place and case.

  FIG. 4 is a block diagram showing an internal circuit 30 of a computer on the imaging device side, for example, which executes the computer program of the present invention for realizing the above-described embodiments. As shown in FIG. 4, the CPU 31, ROM 32, RAM 33, image control unit 36, controller 37, input control unit 40 and external interface (I / F) unit 44 are connected to a bus 46. 4, the above-described computer program of the present invention includes a readable / writable recording medium such as a hard disk (HDD) 38a, or a readable recording medium such as a ROM 32 or a CD-ROM 38n (all of which are removable recording media). ) Is recorded. A person image feature amount recording unit 10 and a background image feature amount recording unit 20 according to the first embodiment are recorded on a hard disk (HDD) 38a. Furthermore, a data structure in which the composite data in the second embodiment is associated with, for example, an XML tag can be recorded. The computer program of the present invention is loaded into the RAM 33 from the ROM 32 via the bus 46 or from a recording medium such as a hard disk (HDD) 38a or CD-ROM 38n via the controller 37 via the bus 46. The input control unit 40 is connected to an input operation unit 42 such as a mouse and a keyboard to perform input control and the like. The VRAM 35 which is an image memory has a capacity corresponding to the data capacity of at least one screen of the display unit 34 such as an imaging device, and the image control unit 36 converts the data in the VRAM 35 into image data and sends it to the display unit 34. It has a function to send out. The external I / F unit 44 has an input / output interface function when the computer communicates with the outside.

  The object of the present invention can be achieved by the CPU 31 executing the above-described computer program of the present invention. As described above, the computer program is a computer in the form of a readable / writable recording medium such as a hard disk (HDD) 38a or a readable recording medium such as a ROM 32 or a CD-ROM 38n (both including a removable recording medium). A readable / writable recording medium such as a hard disk (HDD) 38a that records the computer program, or a readable recording medium such as a ROM 32 or CD-ROM 38n (both are removable recording media). Including) also constitutes the present invention. As the recording medium on which the computer program is recorded, for example, a memory card, a memory stick, a DVD, an optical disk, an FD, or the like can be used in addition to the recording medium described above.

  As an application example of the present invention, the present invention can be applied to a case where an image in a live video in TV broadcasting is transmitted from an image pickup device to a display device such as a monitor of a car navigation device.

It is a flowchart which shows the flow of a process of the image abstraction processing program in Example 1 of this invention. 3 is a diagram illustrating the contents of a person image feature amount recording unit 10. FIG. FIG. 4 is a diagram illustrating the contents of a background image feature amount recording unit 20. It is a block diagram which shows the internal circuit 30 of the computer which performs the computer program of this invention.

Explanation of symbols

  10 person image feature amount recording unit 12, 14, 16, 22, 24, 26 feature point, 18 person type, 20 background image feature amount recording unit, 28 background type, 30 internal circuit, 31 CPU, 32 ROM, 33 RAM 34 display unit, 35 VRAM, 36 image control unit, 37 controller, 38a hard disk (HDD), 38n CD-ROM, 40 input control unit, 42 input operation unit, 44 external I / F unit, 46 bus.

Claims (7)

An image abstraction processing program for performing abstraction processing on an image captured by an imaging device,
An image recognition step of performing feature extraction and region segmentation on the image captured by the imaging device to identify a person image and a background image;
The person image identified in the image recognition step is subjected to a predetermined person abstraction process to obtain an abstracted person image, and the background image is subjected to a predetermined background abstraction process to be abstracted Abstraction processing step to get,
An image abstraction processing program for executing a synthesis processing step for synthesizing the abstracted person image acquired in the abstraction processing step and the abstracted background image. The image abstraction processing program according to claim 1, wherein the person image processing in the abstraction processing step includes:
Comparing the predetermined feature amount of the predetermined feature point extracted in the image recognition step with the predetermined feature amount of the predetermined feature point of the person image recorded in the person image feature amount recording unit in advance, A person type specifying step for specifying the type of the person image identified in the image recognition step;
An image abstraction process comprising: a person replacement process step of replacing the person image identified in the image recognition step with an abstracted person image corresponding to the type specified in the person type specification step program. The image abstraction processing program according to claim 1, wherein the background abstraction processing in the abstraction processing step includes:
Comparing a predetermined feature amount of the predetermined feature point extracted in the image recognition step with a predetermined feature amount of the predetermined feature point of the background image recorded in the background image feature amount recording unit in advance, A background type specifying step for specifying the type of the background image identified in the image recognition step;
An image abstraction process comprising: a background replacement process step of replacing the background image identified in the image recognition step with an abstracted background image corresponding to the type specified in the background type specifying step program.   4. The image abstraction processing program according to claim 1, wherein the abstracted person image is an image of an animation character.   5. The image abstraction processing program according to claim 1, wherein the abstracted background image is an image painted in the same color as the background image.   A computer-readable recording medium on which the image abstraction processing program according to any one of claims 1 to 6 is recorded. A data structure in which composite data obtained by performing an abstraction process on an image captured by an imaging device and predetermined tag information are associated with each other and held in a computer.
An image recognition step for performing feature extraction and area division on the image captured by the imaging device to identify a person image and a background image;
The person image identified in the image recognition step is subjected to a predetermined person abstraction process to obtain an abstracted person image, and the background image is subjected to a predetermined background abstraction process to be abstracted An abstraction processing step for obtaining
The synthesized image obtained by executing the synthesizing process step of synthesizing the abstracted person image acquired in the abstracting process step and the abstracted background image, and predetermined tag information, and the person image Alternatively, a computer-readable recording medium on which a data structure is recorded, wherein the tag information associated with the background image is held in association with each other.

Images