JP2013141199A - Image processing apparatus, program, image processing method, and imaging device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To compound a text into an image for a browser to easily read the text.SOLUTION: An image processing apparatus includes: an image input unit that inputs image data; an edge detection unit that detects an edge in the image data input by the image input unit; a text input unit that inputs text data; an area determination unit that determines a composite area of the text data in the image data on the basis of the edge detected by the edge detection unit; and a compounding unit that compounds the text data into the composite area determined by the area determination unit.

Description

  The present invention relates to an image processing device, a program, an image processing method, and an imaging device.

  Currently, a technique for superimposing text related to a captured image on the captured image is disclosed (see, for example, Patent Document 1). In the technique described in Patent Literature 1, a composite image is generated by superimposing text on a non-important area other than an important area in which a relatively important subject is captured in a captured image. Specifically, an area in which a person is shown is classified as an important area, and text is superimposed on a non-important area that does not include the center of the image.

JP 2007-96816 A

  However, the technology described in Patent Document 1 does not consider readability when text is superimposed on an image. For this reason, for example, when text is superimposed on an area where a complex texture exists, the outline of the font used for text display and the edge of the texture may overlap and the readability of the text may deteriorate. That is, the text may be difficult to read.

  The present invention has been made in view of the above points, and an object of the present invention is to provide an image processing apparatus, a program, an image processing method, and an imaging apparatus capable of synthesizing text in an image so that a viewer can easily read the text. Is to provide.

  SUMMARY An advantage of some aspects of the invention is that an acquisition unit that acquires image data and text data, and an edge of the image data acquired by the acquisition unit are provided. A detection unit for detecting; an area determination unit for determining an area in which the text data is arranged in the image data based on the edge detected by the detection unit; and the text in the region determined by the area determination unit And an image generation unit that generates an image in which data is arranged.

  According to another aspect of the present invention, an image input unit that inputs image data, an edge detection unit that detects an edge in the image data input by the image input unit, a text input unit that inputs text data, A region determining unit that determines a combined region of the text data in the image data based on an edge detected by an edge detecting unit; and a combining unit that combines the text data with the combined region determined by the region determining unit; An image processing apparatus comprising:

  According to another aspect of the present invention, the step of inputting image data, the step of inputting text data, the step of detecting edges in the input image data, and the image data based on the detected edges A program for causing a computer to execute a step of determining a synthesis region of the text data and a step of synthesizing the text data with the determined synthesis region.

  According to one embodiment of the present invention, an image processing device inputs image data, the image processing device inputs text data, and the image processing device uses an edge in the input image data. The image processing apparatus determines a text data synthesis area in the image data based on the detected edge; and the image processing apparatus determines the text in the determined synthesis area. And a step of synthesizing data.

  Another embodiment of the present invention is an imaging device including the above-described image processing device.

  An image processing apparatus according to an aspect of the present invention includes a detection unit that detects an edge of image data, and an arrangement region in which characters in the image data are arranged based on the position of the edge detected by the detection unit And an image generation unit that generates an image in which the characters are arranged in the arrangement region determined by the region determination unit.

  According to the present invention, text can be synthesized in an image so that a viewer can easily read the text.

It is an example of the functional block diagram of the imaging device by the 1st Embodiment of this invention. It is a block diagram which shows the function structure of the image process part by 1st Embodiment. It is an image figure which shows an example of the input image by 1st Embodiment, a cost image, and a synthesized image. 4 is a flowchart illustrating a procedure of still image composition processing according to the first embodiment. It is a flowchart which shows the procedure of the synthetic | combination process of the moving image by 1st Embodiment. It is a block diagram which shows the function structure of the image process part by this embodiment by 2nd Embodiment. It is a flowchart which shows the procedure of the synthetic | combination process by 2nd Embodiment. It is a block diagram which shows the function structure of the image process part by 3rd Embodiment. It is a flowchart which shows the procedure of the synthetic | combination process by 3rd Embodiment. It is an image figure which shows the calculation method of the sum total of the cost in a text rectangular area. It is a figure which shows typically an example of the process which extracts the feature-value of a captured image. It is a figure which shows typically another example of the process which extracts the feature-value of a captured image. It is a flowchart which shows typically the determination method of a smile level. It is a figure which shows an example of the output image from an image processing apparatus. It is a figure which shows another example of the output image from an image processing apparatus. It is a schematic block diagram showing the internal structure of the image process part of an imaging device. It is a flowchart which shows the flow of determination of a representative color. It is a conceptual diagram which shows an example of the process in an image process part. It is a conceptual diagram which shows an example of the process in an image process part. It is a conceptual diagram which shows the result of the clustering implemented with respect to the main area | region shown in FIG. It is an example of the image which the text was added by the text addition part. It is another example of the image which the text was added by the text addition part. It is a figure which shows an example of the correspondence table of a color and a word. It is a figure which shows an example of the correspondence table for a distant view image (2nd scene image). It is a figure which shows an example of the corresponding | compatible table for other images (3rd scene image).

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[First Embodiment]
FIG. 1 is an example of a functional block diagram of the imaging apparatus 100 according to the present embodiment.
As shown in FIG. 1, the imaging apparatus 100 includes an imaging unit 110, a buffer memory unit 130, an image processing unit 140, a display unit 150, a storage unit 160, a communication unit 170, an operation unit 180, and a CPU (Central processing unit) 190. Prepare.

  The imaging unit 110 includes an optical system 111, an imaging element 119, and an A / D conversion unit 120. The optical system 111 includes one or more lenses.

  For example, the image sensor 119 converts an optical image formed on the light receiving surface into an electrical signal and outputs the electrical signal to the A / D converter 120.

  Further, the image sensor 119 uses the image data obtained when a still image capturing instruction is received via the operation unit 180 as captured image data of the captured still image, and the A / D conversion unit 120 and the image processing unit 140. To be stored in the storage medium 200. In addition, the image sensor 119 obtains moving image image data continuously captured at a predetermined interval obtained when a moving image capturing instruction is received via the operation unit 180 as captured image data of the captured moving image as A The data is stored in the storage medium 200 via the / D conversion unit 120 and the image processing unit 140. In addition, for example, the imaging element 119 uses the A / D conversion unit 120 and the image processing as the through image data (captured image) as image data obtained continuously in a state where an imaging instruction is not received via the operation unit 180. The data is continuously output to the display unit 150 via the unit 140.

  Note that the optical system 111 may be attached to and integrated with the imaging apparatus 100, or may be detachably attached to the imaging apparatus 100.

  The A / D converter 120 performs analog / digital conversion on the electronic signal converted by the image sensor 119, and outputs captured image data (captured image) that is the converted digital signal.

  That is, the imaging unit 110 is controlled by the CPU 190 based on the set imaging conditions (for example, aperture value, exposure value, etc.), and forms an optical image via the optical system 111 on the imaging device 119 for A / D conversion. A captured image based on the optical image converted into a digital signal by the unit 120 is generated.

  The operation unit 180 includes, for example, a power switch, a shutter button, a cross key, a confirmation button, and other operation keys. The operation unit 180 is operated by the user, receives a user operation input, and outputs it to the CPU 190.

  The image processing unit 140 performs image processing on the image data stored in the buffer memory unit 130 based on the image processing conditions stored in the storage unit 160. Details of the image processing unit 140 will be described later. The image data stored in the buffer memory unit 130 is image data input to the image processing unit 140. For example, the image data read from the captured image data, the through image data, or the storage medium 200 described above. This is taken image data.

  The display unit 150 is a liquid crystal display, for example, and displays image data, an operation screen, and the like. For example, the display unit 150 displays a captured image to which text is added by the image processing unit 140.

The storage unit 160 stores various information.
The buffer memory unit 130 temporarily stores image data captured by the imaging unit 110. The communication unit 170 is connected to a removable storage medium 200 such as a card memory, and writes, reads, or deletes photographed image data in the storage medium 200.

  The storage medium 200 is a storage unit that is detachably connected to the imaging device 100, and stores, for example, photographed image data generated by the imaging unit 110. The CPU 190 controls each component included in the imaging device 100. The bus 300 is connected to the imaging unit 110, the CPU 190, the operation unit 180, the image processing unit 140, the display unit 150, the storage unit 160, the buffer memory unit 130, and the communication unit 170, and outputs from each unit. The transferred image data and control signals are transferred.

FIG. 2 is a block diagram illustrating a functional configuration of the image processing unit 140 according to the present embodiment.
The image processing unit (image processing apparatus) 140 includes an image input unit 11, a text input unit 12, a first position input unit 13, an edge detection unit 14, a face detection unit 15, and a character size determination unit 16. The cost calculation unit 17, the region determination unit 18, and the synthesis unit 19 are included.

  The image input unit 11 inputs still image data or moving image data. The image input unit 11 outputs the input image data to the edge detection unit 14 and the character size determination unit 16. Note that the image input unit 11 may input image data via a network or a storage medium, for example. Hereinafter, an image indicated by the image data input to the image input unit 11 is referred to as an input image. Also, an XY coordinate system is defined with the width direction of the rectangular image format in the input image as the X-axis direction and the direction (height direction) orthogonal to the X-axis direction as the Y-axis direction.

  The text input unit 12 inputs text data corresponding to the input image. The text data corresponding to the input image is data relating to the text to be superimposed on the input image, and includes text, initial character size, line feed position, number of rows, number of columns, and the like. The initial character size is an initial value of the character size of the text, and is the character size designated by the user. The text input unit 12 outputs the input text data to the character size determination unit 16.

The first position input unit 13 receives an input of an important position in the input image (hereinafter referred to as an important position (first position)). For example, the first position input unit 13 displays an input image on the display unit 150 and sets a position specified by the user on the touch panel installed on the display unit 150 as an important position. Alternatively, a first position input unit 13 may receive an input of the direct coordinate values of key position (x _{0, y 0).} The first position input unit 13 outputs the coordinate value (x ₀ , y ₀ ) of the important position to the cost calculation unit 17. The first position input unit 13 sets a predetermined position (for example, the center of the input image) as an important position when no important position is input from the user.

The edge detection unit 14 detects an edge in the image data input from the image input unit 11 using, for example, the Canny algorithm. Then, the edge detection unit 14 outputs the image data and data indicating the position of the edge detected from the image data to the cost calculation unit 17. In this embodiment, the edge is detected using the Canny algorithm. For example, the edge is detected based on an edge detection method using a differential filter or a high-frequency component in the result of two-dimensional Fourier transform. A method or the like may be used.
The face detection unit 15 detects a human face in the image data input from the image input unit 11 by pattern matching or the like. Then, the face detection unit 15 outputs the image data and data indicating the position of the human face detected from the image data to the cost calculation unit 17.

  Based on the image size (width and height) of the image data input from the image input unit 11 and the number of lines and columns of the text data input from the text input unit 12, the character size determination unit 16 Determine the character size of the data. Specifically, the character size determination unit 16 sets f satisfying the following expression (1) as the character size so that all texts in the text data can be combined with the image data.

  Here, m is the number of text data columns, and l is the number of text data rows. L (≧ 0) is a parameter indicating the ratio of line spacing to character size. Further, w is the width of the image area in the image data, and h is the height of the image area in the image data. Expression (1) represents that the width of the text is smaller than the width of the image area in the image data, and the height of the text is smaller than the height of the image area in the image data.

  For example, if the initial character size included in the text data does not satisfy Expression (1), the character size determination unit 16 gradually decreases the character size until Expression (1) is satisfied. On the other hand, when the initial character size included in the text data satisfies Expression (1), the character size determination unit 16 sets the initial character size included in the text data as the character size of the text data. Then, the character size determination unit 16 outputs the text data and the character size of the text data to the region determination unit 18.

  The cost calculation unit 17 calculates the cost of each coordinate position (x, y) in the image data based on the position of the edge in the image data, the position of the person's face, and the important position. The cost represents importance in the image data. For example, the cost calculation unit 17 calculates the cost of each position so that the cost of the position where the edge is detected by the edge detection unit 14 is high. The cost calculation unit 17 increases the cost as it is closer to the important position, and lowers the cost as it is farther from the important position. In addition, the cost calculation unit 17 increases the cost of the region with the human face.

Specifically, first, the cost calculation unit 17 uses, for example, a global cost image c _g (x, y) indicating a cost based on the important position (x ₀ , y ₀ ) using a Gaussian function expressed by the following equation (2). ) Is generated.

However, x ₀ is the X coordinate value of the critical position, y ₀ is the Y-coordinate values of the critical position. S ₁ (> 0) is a parameter that determines how the cost spreads in the width direction (X-axis direction), and S ₂ (> 0) is a parameter that determines how the cost spreads in the height direction (Y-axis direction). It is. Parameter S ₁ and parameter S ₂ can be set by the user, for example, by setting screen or the like. Than to change the parameters S ₁ and parameter S _2, it is possible to adjust the shape of the distribution in the global cost image. In this embodiment, the global cost image is generated by a Gaussian function. For example, the cosine function ((cos (πx) +1) / 2, where −1 ≦ x ≦ 1) or the origin x = 0. A function of a distribution whose value increases as it is closer to the center, such as a function represented by a triangular (mountain) straight line that takes the maximum value or a Lorentz function (1 / (ax ² +1), a is a constant). It may be used to generate a global cost image.

Next, the cost calculation unit 17 generates a face cost image c _f (x, y) indicating the cost based on the position of the person's face by the following equations (3) and (4).

However, (x ⁽ⁱ⁾ , y ⁽ⁱ⁾ ) is the center position of the i (1 ≦ i ≦ n) -th face among the detected n faces, and s ⁽ⁱ⁾ is the i-th face It is a size. That is, the cost calculation unit 17 generates a face cost image in which the pixel value in the human face region is “1” and the pixel value in the unexpected region is “0”.

Next, the cost calculation unit 17 generates an edge cost image c _e (x, y) indicating the cost based on the edge by the following equation (5).

  That is, the cost calculation unit 17 generates an edge cost image in which the pixel value of the edge portion is “1” and the pixel value of the region other than the edge is “0”. Note that the edge portion may be a position where the edge is present, or may be a region including the position where the edge is present and its periphery.

  Then, the cost calculation unit 17 generates a final cost image c (x, y) based on the global cost image, the face cost image, and the edge cost image by the following equation (6).

However, C _g (≧ 0) is a parameter of the weighting coefficient of the global cost image, C _f (≧ 0) is a parameter of the weighting coefficient of the face cost image, and C _e (≧ 0) is the weighting of the edge cost image. This is a coefficient parameter. The ratio of the parameter C _g , the parameter C _e and the parameter C _f can be changed by the user on the setting screen or the like. Further, the final cost image c (x, y) shown in Expression (6) is normalized so that 0 ≦ c (x, y) ≦ 1. The cost calculation unit 17 outputs the image data and the final cost image of the image data to the region determination unit 18. The parameter C _g , the parameter C _e and the parameter C _f may be 1 or less.

Note that the image processing unit 140 may automatically change the ratio of the parameter C _g , the parameter C _e, and the parameter C _f according to the input image. For example, if the input image is a landscape image, the larger the parameter C _g than other parameters. Further, when the input image is a portrait (person image) increases the parameter C _f than other parameters. Further, when the input image is a large building images building or the like increases the parameter C _e than other parameters. Specifically, the cost calculation unit 17, when the face of a person is detected by the face detection unit 15 determines the input image portrait and, to increase the parameter C _f than other parameters. On the other hand, when the face detection unit 15 does not detect a human face, the cost calculation unit 17 determines that the input image is a landscape image, and makes the parameter _Cg larger than the other parameters. Also, the cost calculating unit 17, if the detected edge by the edge detecting section 14 is greater than the predetermined value, determines the input image as a building image, to increase the parameter C _e than other parameters.
Alternatively, the image processing unit 140 has a landscape image mode, a portrait mode, and a building image mode, and the parameter C _g , the parameter C _e, and the parameter according to the mode currently set in the image processing unit 140. The ratio of C _f may be changed.

Further, when the image data is a moving image, the cost calculation unit 17 calculates an average value of costs of a plurality of frame images included in the moving image data for each coordinate position. Specifically, the cost calculation unit 17 acquires a frame image of a moving image at a predetermined time (for example, 3 seconds) interval, and generates a final cost image for each acquired frame image. And the cost calculation part 17 produces | generates the average final cost image which averaged the final cost image of each frame image. The pixel value at each position in the average final cost image is the average value of the pixel values at each position in each final cost image.
In the present embodiment, the average value of the costs of a plurality of frame images is calculated. However, for example, a total value may be calculated.

The area determination unit 18 determines a synthesis area for synthesizing text in the image data based on the final cost image input by the cost calculation unit 17 and the character size of the text data input by the character size determination unit 16. . Specifically, first, the area determination unit 18 determines the width w _text and the height h _text of a text rectangular area that is a rectangular area for displaying text, based on the number of rows and columns of text data and the character size. Is calculated. The text rectangular area is an area corresponding to the synthesis area. Subsequently, the area determination unit 18 calculates the total cost c ^* _text (x, y) in the text rectangular area at each coordinate position (x, y) by the following equation (7).

Then, the area determination unit 18 sets the coordinate position (x, y) at which the total cost c ^* _text (x, y) in the text rectangular area is minimum as the text synthesis position. That is, the region determination unit 18 sets a text rectangular region having a coordinate position (x, y) at which the total cost c ^* _text (x, y) in the text rectangular region is minimum as a top left vertex as a text synthesis region. To do. The area determination unit 18 outputs image data, text data, and data indicating a text synthesis area to the synthesis unit 19. In the present embodiment, the area determination unit 18 determines the synthesis area based on the total cost (total value) in the text rectangular area. For example, the average cost value in the text rectangular area is the highest. A small area may be used as a synthesis area. Alternatively, the region determination unit 18 may set a region having the smallest cost weighted average value obtained by increasing the weight of the center of the text rectangular region as the composite region.

  The synthesis unit 19 receives image data, text data, and data indicating a text synthesis area. The synthesizing unit 19 generates and outputs image data of a synthesized image obtained by superimposing the text of the text data on the synthesis area of the image data.

FIG. 3 is an image diagram illustrating an example of an input image, a cost image, and a composite image according to the present embodiment.
FIG. 3A shows an input image. FIG. 3B shows a global cost image. In the example shown in FIG. 3B, the center of the input image is the important position. As shown in FIG. 3B, the pixel value of the global cost image is closer to “1” as it is closer to the center, and closer to “0” as it is farther from the center. FIG. 3C shows a face cost image. As shown in FIG. 3C, the pixel value of the face cost image is “1” in the area of the human face and “0” in the area other than the human face. FIG. 3D shows an edge cost image. As shown in FIG. 3D, the edge value of the pixel value of the edge cost image is “1”, and the area other than the edge portion is “0”.

  FIG. 3E shows a final cost image obtained by combining a global cost image, a face cost image, and an edge cost image. FIG. 3F shows a synthesized image obtained by superimposing text on the input image. As shown in FIG. 3F, the text of the text data is superimposed on an area where the total cost in the final cost image is small.

Next, still image synthesis processing by the image processing unit 140 will be described with reference to FIG.
FIG. 4 is a flowchart illustrating a procedure of still image composition processing according to the present embodiment.
First, in step S101, the image input unit 11 accepts input of still image data (hereinafter referred to as still image data).
Next, in step S102, the text input unit 12 receives input of text data corresponding to the input still image data.
Next, in step S103, the first position input unit 13 receives an input of an important position in the input still image data.

Subsequently, in step S104, the character size determination unit 16 determines the character size of the text data based on the size of the input still image data and the number of rows and columns of the input text data.
Next, in step S105, the face detection unit 15 detects the position of the person's face in the input still image data.
Next, in step S106, the edge detection unit 14 detects the position of the edge in the input still image data.

Subsequently, in step S107, the cost calculation unit 17 generates a global cost image based on the designated important position. That is, the cost calculation unit 17 generates a global cost image that has a higher cost as it is closer to the important position and a lower cost as it is far from the important position.
Next, in step S108, the cost calculation unit 17 generates a face cost image based on the detected face position of the person. That is, the cost calculation unit 17 generates a face cost image in which the cost of the human face region is high and the cost of the region other than the human face is low.
Next, in step S109, the cost calculation unit 17 generates an edge cost image based on the detected edge position. That is, the cost calculation unit 17 generates an edge cost image in which the cost of the edge portion is high and the cost of the region other than the edge is low.

Subsequently, in step S110, the cost calculation unit 17 generates a final cost image by combining the generated global cost image, face cost image, and edge cost image.
In step S111, the area determination unit 18 determines a text synthesis area in the still image data based on the generated final cost image and the determined character size of the text data.
Finally, in step S112, the synthesizer 19 synthesizes the still image data and the text data by superimposing the text of the text data on the determined synthesis area.

Next, a moving image composition process performed by the image processing unit 140 will be described with reference to FIG. FIG. 5 is a flowchart showing the procedure of the moving image composition process according to the present embodiment.
First, in step S201, the image input unit 11 receives input of moving image data (hereinafter referred to as moving image data).
Next, in step S202, the text input unit 12 accepts input of text data corresponding to the input moving image data.
Next, in step S203, the first position input unit 13 receives designation of an important position in the input moving image data.

Subsequently, in step S204, the character size determination unit 16 determines the character size of the text data based on the size of the moving image data and the number of rows and columns of the text data.
Next, in step S205, the cost calculation unit 17 acquires the first frame image from the moving image data.

Subsequently, in step S206, the face detection unit 15 detects the position of the person's face in the acquired frame image.
Next, in step S207, the edge detection unit 14 detects the position of the edge in the acquired frame image.

Subsequently, in step S208, the cost calculation unit 17 generates a global cost image based on the input important position.
Next, in step S209, the cost calculation unit 17 generates a face cost image based on the detected face position of the person.
Next, in step S210, the cost calculation unit 17 generates an edge cost image based on the detected edge position.

Subsequently, in step S211, the cost calculation unit 17 generates a final cost image by combining the generated global cost image, face cost image, and edge cost image.
Next, in step S212, the cost calculation unit 17 determines whether or not the current frame image is the last frame image in the moving image data.
When the current frame image is not the last frame image (step S212: No), in step S213, the cost calculation unit 17 converts the frame image after a predetermined time t seconds (for example, 3 seconds) from the current frame image to moving image data. And return to step S206.

  On the other hand, when the current frame image is the last frame in the moving image data (step S212: Yes), in step S214, the cost calculation unit 17 generates an average final cost image by averaging the final cost images of the respective frame images. To do. The pixel value at each coordinate position in the average final cost image is the average value of the pixel values at each coordinate position in the final cost image of each frame image.

Next, in step S215, the area determination unit 18 determines a text synthesis area in the moving image data based on the generated average final cost image and the determined character size of the text data.
Finally, in step S216, the synthesizer 19 synthesizes the moving image data and the text data by superimposing the text data on the determined synthesis area.

In the present embodiment, the composite area in the entire moving image data is determined based on the average final cost image, but the composite area may be determined every predetermined time of the moving image data. For example, the image processing unit 140 sets the synthesis region r ₁ based on the first frame image as the synthesis region of the frame image from 0 seconds to t−1 seconds, and the synthesis region r ₂ based on the t-second frame image from t seconds. The composite area of frame images up to 2t-1 seconds is determined, and the composite area of each frame image is determined in the same manner. As a result, the text can be synthesized at the optimum position in accordance with the movement of the subject in the moving image data.

  As described above, according to the present embodiment, the image processing unit 140 determines a synthesis region in which text is to be synthesized based on an edge cost image indicating a cost related to an edge in image data. Therefore, it is possible to synthesize text in a region with few edges (that is, a region where no complex texture exists). Thereby, since it is possible to prevent the outline of the font used for text display and the texture edge from overlapping, the text can be synthesized in the input image so that the viewer can easily read the text.

  In addition, when the text display position is fixed, depending on the content of the input image and the amount of text, the text may overlap the subject, the person of interest, the object, the background, etc., and the original impression of the input image may be deteriorated. . Since the image processing unit 140 according to the present embodiment determines a synthesis area for text synthesis based on a face cost image indicating a cost related to a human face in the image data, the text is synthesized in an area other than the human face. can do. Further, since the image processing unit 140 determines a synthesis area for synthesizing text based on a global cost image indicating a cost related to an important position in the image data, the image processing unit 140 can synthesize text in an area away from the important position. it can. For example, in many images, a subject is present in the central portion, and text can be synthesized in an area other than the subject by setting the central portion as an important position. In the image processing unit 140 according to the present embodiment, since the user can specify the important position, for example, in the input image A, the central portion is set as the important position, and in the input image B, the end portion is set as the important position. The important position can be changed.

  In addition, according to the present embodiment, the image processing unit 140 determines a synthesis area in which text is synthesized based on a final cost image obtained by combining a global cost image, a face cost image, and an edge cost image. Therefore, it is possible to synthesize text at the optimal position comprehensively.

  By the way, when the character size is fixed, the relative size of the text with respect to the image data may change drastically depending on the image size of the input image, resulting in a text display unsuitable for the viewer. For example, when the character size of text data is relatively large with respect to the input image, all the text does not fit in the input image and the sentence cannot be read. According to the present embodiment, the image processing unit 140 changes the character size of the text data in accordance with the image size of the input image, so that the entire text can be stored in the input image.

  Further, according to the present embodiment, the image processing unit 140 can synthesize text with moving image data. Thereby, for example, it can be applied to a service or the like that dynamically displays a comment received from a user in an image while a moving image is distributed and reproduced by broadcasting or the Internet. In addition, since the image processing unit 140 determines the synthesis region using the average final cost image of a plurality of frame images, the image processing unit 140 synthesizes the text into a comprehensively optimal region considering the movement of the subject in the entire moving image. be able to.

[Second Embodiment]
Next, an image processing unit (image processing apparatus) 140a according to a second embodiment of the present invention will be described.
FIG. 6 is a block diagram illustrating a functional configuration of the image processing unit 140a according to the present embodiment. In this figure, the same parts as those of the image processing unit 140 shown in FIG. The image processing unit 140a includes a second position input unit 21 in addition to the configuration of the image processing unit 140 shown in FIG.
The second position input unit 21 receives an input of a position (hereinafter referred to as a text position (second position)) where text is combined in the image data. For example, the second position input unit 21 displays the image data input to the image input unit 11 on the display unit 150, and sets the position specified by the user on the touch panel installed on the display unit 150 as the text position. Alternatively, the second position input unit 21 may directly accept input of coordinate values (x ₁ , y ₁ ) of the text position. The second position input unit 21 outputs the coordinate value (x ₁ , y ₁ ) of the text position to the cost calculation unit 17a.

Based on the text position (x ₁ , y ₁ ) input by the second position input unit 21, the position of the edge in the image data, the position of the person's face, and the important position, the cost calculation unit 17 a The cost of each coordinate position (x, y) in the image data is calculated. Specifically, the cost calculation unit 17a combines the text position cost image indicating the cost based on the text position (x ₁ , y ₁ ), the global cost image, the face cost image, and the edge cost image. Generate a cost image. The generation method of the global cost image, the face cost image, and the edge cost image is the same as that in the first embodiment.

The cost calculation unit 17a generates a text position cost image c _t (x, y) by the following equation (8).

However, S ₃ (> 0) is a parameter that determines how the cost spreads in the width direction (X-axis direction), and S ₄ (> 0) is a parameter that determines how the cost spreads in the height direction (Y-axis direction). It is. The text position cost image is an image that is lower in cost as it is closer to the text position (x ₁ , y ₁ ) and higher in cost as it is farther from the text position.

  And the cost calculation part 17a produces | generates the final cost image c (x, y) by following Formula (9).

However, C _t (≧ 0) is a parameter of the weighting coefficient of the text position cost image.
Expression (9) is an expression in which C _t is added to the denominator of Expression (6) and C _{t ct} (x, y) is added to the numerator. Note that if the text position is not designated by the second position input unit 21, the cost calculation unit 17a generates the final cost image by the above-described equation (6) without generating the text position cost image. Alternatively, the cost calculation unit 17a sets the parameter C _t = 0 when the text position is not designated by the second position input unit 21.

  In addition, when the image data is a moving image, the cost calculation unit 17a calculates an average value of costs of a plurality of frame images included in the moving image data for each coordinate position. Specifically, the cost calculation unit 17a acquires a frame image of a moving image at a predetermined time (for example, 3 seconds) interval, and generates a final cost image for each acquired frame image. Then, the cost calculation unit 17a generates an average final cost image obtained by averaging the final cost images of the respective frame images.

Next, with reference to FIG. 7, the composition processing by the image processing unit 140a will be described. FIG. 7 is a flowchart showing the procedure of the synthesis process according to the present embodiment.
The processes shown in steps S301 to S303 are the same as the processes shown in steps S101 to S103 described above.
Subsequent to step S303, in step S304, the second position input unit 21 accepts designation of a text position in the input image data.
The processing shown in steps S305 to S307 is the same as the processing shown in steps S104 to S106 described above.

Following step S307, in step S308, the cost calculation unit 17a generates a text position cost image based on the designated text position.
The processing shown in steps S309 to S311 is the same as the processing shown in steps S107 to S109 described above.

Subsequent to step S311, in step S312, the cost calculation unit 17a generates a final cost image by combining the text position cost image, the global cost image, the face cost image, and the edge cost image.
Next, in step S313, the area determination unit 18 determines a text synthesis area in the image data based on the generated final cost image and the determined character size of the text data.
Finally, in step S314, the synthesis unit 19 superimposes the text data on the determined synthesis area to synthesize the image data and the text data.

  In the present embodiment, the text position is specified in the second position input unit 21. However, for example, an area where text is to be synthesized may be specified. In this case, the cost calculation unit 17a generates a text position cost image in which the pixel value of the designated area is “0” and the pixel value of the other area is “1”. That is, the cost calculation unit 17a reduces the cost of the designated area.

  As described above, according to the present embodiment, the user can designate the position where the text is to be synthesized, and the image processing unit 140a determines the synthesis area by reducing the cost of the designated text position. Thereby, not only the effect similar to 1st Embodiment but the position designated by the user can be preferentially selected as a text data synthesis region.

[Third Embodiment]
Next, an image processing unit (image processing apparatus) 140b according to a third embodiment of the present invention will be described.
FIG. 8 is a block diagram illustrating a functional configuration of the image processing unit 140b according to the present embodiment. In this figure, the same parts as those of the image processing unit 140 shown in FIG. The image processing unit 140b includes a second position input unit 31 in addition to the configuration of the image processing unit 140 illustrated in FIG.
The second position input unit 31 receives an input of a text position (second position) in either the X-axis direction (width direction) or the Y-axis direction (height direction). The text position is a position where the text is synthesized in the image data. For example, the second position input unit 31 displays the image data input to the image input unit 11 on the display unit 150, and sets the position specified by the user on the touch panel installed on the display unit 150 as the text position. Alternatively, the second position input unit 31 may directly accept the input of the X coordinate value x ₂ or the Y coordinate value y ₂ of the text position. The second position input unit 31 outputs the X-coordinate value _{x 2} or Y-coordinate value _{y 2} of the text located in the region determination unit 18b.

When the position x _{2 in the} width direction is designated by the second position input unit 31, the region determination unit 18 b fixes the X coordinate value to x ₂ in the above-described formula (7) and sets c ^* _text (x ₂ , A Y coordinate value y _min that minimizes y) is obtained. Then, the region determination unit 18b sets the position (x ₂ , y _min ) as a composite position.

In addition, when the position y _{2 in the} height direction is designated by the second position input unit 31, the region determination unit 18 b fixes the Y coordinate value to y ₂ in the above-described formula (7) and sets c ^* _text ( x _min where x, y ₂ ) is minimized is obtained. Then, the region determination unit 18b sets the position (x _min , y ₂ ) as the synthesis position.

Next, with reference to FIG. 9, the composition processing by the image processing unit 140b will be described. FIG. 9 is a flowchart showing the procedure of the synthesis process according to this embodiment.
The processing from step S401 to S403 is the same as the processing from step S101 to S103 described above.
Following step S403, at step S404, a second position input section 31 accepts an input of the X-coordinate of the text position value _{x 2} or Y-coordinate value _{y 2.}
The processing from step S405 to S411 is the same as the processing from step S104 to S110 described above.

Following step S411, at step S412, the area determining unit 18b is, the X coordinate value _{x 2} or Y-coordinate value _{y 2} in the specified text position, a character size of the text data, based on the final cost image, A text synthesis area in image data is determined.
Finally, in step S413, the synthesizer 19 synthesizes the image data and the text data by superimposing the text data on the determined synthesis area.

  Thus, according to the present embodiment, it is possible to specify the coordinate in the width direction or the height direction of the position where the text is to be synthesized. The image processing unit 140b sets an optimum region based on the final cost image among the designated positions in the width direction or the height direction as a synthesis region. As a result, the text can be superimposed on an optimum area (for example, an area with high text readability, an area without a human face, or an area other than the important position) which is an area desired by the user.

Further, a program for realizing each step shown in FIG. 4, FIG. 5, FIG. 7 or FIG. 9 is recorded on a computer-readable recording medium, and the program recorded on this recording medium is read by a computer system, By executing the processing, the image data and the text data may be combined. Here, the “computer system” may include an OS and hardware such as peripheral devices.
Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
The “computer-readable recording medium” means a flexible disk, a magneto-optical disk, a ROM, a writable nonvolatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, etc. This is a storage device.

Further, the “computer-readable recording medium” means a volatile memory (for example, DRAM (Dynamic DRAM) in a computer system that becomes a server or a client when a program is transmitted through a network such as the Internet or a communication line such as a telephone line. Random Access Memory)), etc., which hold programs for a certain period of time.
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.
The program may be for realizing a part of the functions described above.
Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to
For example, in the above-described embodiment, the imaging apparatus 100 includes the image processing units 140, 140a, and 140b. For example, a terminal device such as a personal computer, a tablet PC (Personal Computer), a digital camera, or a mobile phone The image processing units 140, 140a, and 140b, which are processing devices, may be provided.

  In the above-described embodiment, all the regions in the image data are set as the synthesis region candidates. However, in consideration of the margin of the image data, a region other than the margin may be set as the synthesis region candidate. In this case, the character size determination unit 16 sets f satisfying the following expression (10) as the character size.

However, M ₁ is a parameter indicating the size of the width direction of the margin, M ₂ is a parameter indicating the size of the height direction of the margin. The parameter M ₁ and the parameter M ₂ may be the same value (M ₁ = M ₂ = M). The cost calculation units 17 and 17a generate a final cost image of an area excluding margins in the image data. In addition, the region determination units 18 and 18b select a composite region from regions excluding margins (M ₁ <x <w−M ₁ , M ₂ <y <h−M ₂ ).

  In the present embodiment, the important position is input by the first position input unit 13, but a global cost image may be generated with a predetermined position (for example, the center of the image data) as an important position. Good. For example, when the center of the image data is an important position, the cost calculation units 17 and 17a generate a global cost image by the following equation (11).

  However, S (> 0) is a parameter that determines how the cost spreads.

  Further, when the important position is determined in advance, the global cost image is determined by the image size. Therefore, a global cost image may be prepared in advance for each image size and stored in the storage unit 160. The cost calculation units 17 and 17a read a global cost image corresponding to the image size of the input image from the storage unit 160 and generate a final cost image. This eliminates the need to generate a global cost image for each process of combining text data with image data, thereby reducing the overall processing time.

  In the above-described embodiment, a face cost image based on a human face region is generated. However, a cost image based on an arbitrary feature amount (for example, an object or an animal) may be generated. In this case, the cost calculation units 17 and 17a generate a feature cost image having a high cost in the feature value region. For example, the cost calculation units 17 and 17 a generate a feature amount cost image in which the pixel value of the feature amount area detected by object recognition or the like is set to “1” and the pixel values of other regions are set to “0”. Then, the cost calculation unit 17 generates a final cost image based on the feature amount cost image.

Further, the area determination units 18 and 18b calculate all the coordinate positions (x, y) in advance by the following equation (12) before calculating the total cost c ^* _text (x, y) in the text rectangular area. Alternatively, a differential image may be generated.

In this case, the area determination units 18 and 18b calculate the total cost c ^* _text (x, y) in the text rectangular area by the following equation (13).

FIG. 10 is an image diagram showing a method of calculating the total cost in the text rectangular area.
As shown in this figure, when Expression (13) is used, the total cost c ^* _text (x, y) in the text rectangular area can be calculated by four operations. As a result, the processing time can be shortened compared to the case where the total cost c ^* _text (x, y) in the text rectangular area is calculated by the above-described equation (7).

  FIG. 11 is a diagram schematically illustrating an example of a process for extracting a feature amount of a captured image used for determining a sentence arranged on an image. In the example of FIG. 11, the determination unit of the image processing apparatus classifies the scene of the captured image into a person image or a landscape image. Next, the image processing apparatus extracts a feature amount of the captured image according to the scene. The feature amount can be the number of faces (number of subjects) and the average color (color arrangement pattern) in the case of a person image, and can be the average color (color arrangement pattern) in the case of a landscape image. . Based on these feature quantities, words (adjectives and the like) to be inserted into the person image template or the landscape image template are determined.

  Here, in the example of FIG. 11, the color arrangement pattern is configured by a combination of a plurality of representative colors constituting the captured image. Therefore, the color arrangement pattern can represent the average color (average color) of the captured image. In one example, “first color”, “second color”, and “third color” are defined as a color arrangement pattern, and based on a combination of these three colors, that is, based on three average colors, a person image Or a word (adjective) to be inserted into a text template for a landscape image.

  In the example of FIG. 11, the scene of the captured image is classified into two types (a person image and a landscape image). In another example, the scene of the captured image can be classified into three or more types (3, 4, 5, 6, 7, 8, 9, or 10 types or more).

  FIG. 12 is a diagram schematically illustrating another example of a process for extracting a feature amount of a captured image used for determining a sentence arranged on an image. In the example of FIG. 12, the scene of the captured image can be classified into three or more types.

  In the example of FIG. 12, the determination unit of the image processing apparatus determines whether the captured image is a person image (first mode image), a distant view image (second mode image), or another image (third mode image). judge. First, as in the example of FIG. 11, the determination unit determines whether the captured image is a person image or an image different from the person image.

  Next, when the captured image is an image different from the human image, the determination unit determines whether the captured image is a distant view image (second mode image) or another image (third mode image). To do. This determination can be performed using, for example, a part of the image identification information given to the captured image.

  Specifically, in order to determine whether the captured image is a distant view image, a focal length that is a part of the image identification information can be used. The determination unit determines that the captured image is a distant view image when the focal distance is greater than or equal to a preset reference distance, and determines the captured image as another image when the focal distance is less than the reference distance. As described above, the captured image is classified into three types of scenes: a person image (first mode image), a distant view image (second mode image), or another image (third mode image). Note that examples of distant view images (second mode images) include landscape images such as the sea and mountains, and examples of other images (third mode images) include flowers and pets.

  Also in the example of FIG. 12, after the scene of the captured image is classified, the image processing apparatus extracts the feature amount of the captured image according to the scene.

  In the example of FIG. 12, when the captured image is a person image (first scene image), the number of faces (number of subjects) and the feature amount of the captured image used to determine the text arranged on the image A smile level can be used. That is, when the captured image is a human image, a word to be inserted into the human image template may be determined based on the determination result of the smile level in addition to or instead of the determination result of the number of faces (number of subjects). it can. Hereinafter, an example of a smile level determination method will be described with reference to FIG.

  In the example of FIG. 13, the determination unit of the image processing apparatus detects a face area from a person image by a method such as face recognition (step S5001). In one example, the degree of smile of a person image is calculated by digitizing the degree of ascending of the mouth corner. For example, various known techniques for face recognition can be used for calculating the smile level.

  Next, the determination unit compares the smile level with the first smile threshold α set in advance (step S5002). When it is determined that the smile level is greater than or equal to α, the determination unit determines that the smile level of the person image is “smile: large”.

  On the other hand, when it is determined that the smile level is less than α, the determination unit compares the smile level with a second smile threshold value β set in advance (step S5003). When it is determined that the smile level is β or more, the determination unit determines that the smile level of this person image is “smile: medium”. Furthermore, when it is determined that the smile level is less than β, the determination unit determines that the smile level of the person image is “smile: small”.

  A word to be inserted into the person image template is determined based on the determination result of the smile level of the person image. Here, examples of the word corresponding to the smile level of “smile: large” include “full of joy” and “very good”. Examples of words that correspond to the smile level of “smile: medium” include “joyful” and “good calm”. Examples of words corresponding to the smile level of “smile: small” include “seriously seems” and “cool”.

  In the above description, the case where the word inserted into the person image template is a continuous form has been described. However, the present invention is not limited to this, and may be an end form, for example. In this case, examples of words corresponding to the smile level of “smile: large” include “smile is nice”, “it is a very good smile”, and the like. Examples of words corresponding to the smile level of “smile: medium” include “smiley” and “good expression”. Examples of words corresponding to a smile level of “smile: small” include “looks serious” and “looks serious”.

  FIG. 14A is an example of an output image showing the operation result of the image processing apparatus, and this output image has a sentence determined based on the example of FIG. In the example of FIG. 14A, the captured image is determined to be a person image, and the number of subjects and the color arrangement pattern (average color) are extracted as the feature amount. Further, the word inserted into the person image template is determined as “heavy” according to the color arrangement pattern. As a result, the output result shown in FIG. 14A is obtained. That is, in the example of FIG. 14A, the word “heavy” (adjective, combined form) is determined based on the average color of the captured image.

  FIG. 14B is another example of an output image showing the operation result of the image processing apparatus, and this output image has a sentence determined based on the example of FIG. In the example of FIG. 14B, it is determined that the captured image is a person image, and the number of subjects and the smile level are extracted as feature amounts. Further, according to the smile level, the word inserted into the person image template is determined as “good expression”. As a result, the output result shown in FIG. 14B is obtained. That is, in the example of FIG. 14B, the word (end form) of “good expression” is determined based on the smile level of the person in the captured image. As shown in the output result of FIG. 14B, by using the word output using the smile level for the person image, it is possible to attach character information that is relatively close to the impression received from the image.

  Returning to FIG. 12, when the captured image is a landscape image (second scene image) or another image (third scene image), as the feature amount of the captured image used to determine the text arranged on the image, A representative color can be used instead of the average color. As the representative color, the “first color” in the color arrangement pattern, that is, the most frequently used color in the captured image can be used. Alternatively, the representative color can be determined using clustering as described below.

  FIG. 15 is a schematic block diagram illustrating an internal configuration of an image processing unit included in the imaging apparatus. In the example of FIG. 15, the image processing unit 5040 of the image processing apparatus includes an image data input unit 5042, an analysis unit 5044, a text creation unit 5052, and a text addition unit 5054. The image processing unit 5040 performs various types of analysis processing on the image data generated by the imaging unit or the like, thereby acquiring various types of information regarding the content of the image data, and creating text that is highly consistent with the content of the image data. Then, text can be added to the image data.

  The analysis unit 5044 includes a color information extraction unit 5046, a region extraction unit 5048, and a clustering unit 5050, and performs analysis processing on the image data. The color information extraction unit 5046 extracts first information regarding color information of each pixel included in the image data from the image data. Typically, the first information is a total of the HSV values of all the pixels included in the image data. However, the first information is the frequency at which the predetermined color appears in the image (frequency in pixel units, area ratio, etc.) for a predetermined color associated with similarity (for example, associated with a predetermined color space). The color resolution and the type of color space are not limited.

  For example, the first information may be information indicating how many pixels of each color are included in the image data for each color represented by an HSV space vector (HSV value) or RGB value. . However, the color resolution in the first information may be changed as appropriate in consideration of the burden of calculation processing, and the type of color space is not limited to HSV or RGB, and may be CMY, CMYK, or the like.

  FIG. 16 is a flowchart showing the flow of representative color determination performed in the analysis unit 5044. In step S5101, the image processing apparatus starts calculating the representative color of specific image data 5060 (captured image, see FIG. 17).

  In step S5102, the image data input unit 5042 of the image processing apparatus outputs the image data to the analysis unit 5044. Next, the color information extraction unit 5046 of the analysis unit 5044 calculates first information 5062 regarding the color information of each pixel included in the image data (see FIG. 17).

  FIG. 17 is a conceptual diagram illustrating a calculation process of the first information 5062 performed by the color information extraction unit 5046 in step S5102. The color information extraction unit 5046 aggregates the color information included in the image data 5060 for each color (for example, for each gradation of 256 gradations) to obtain first information 5062. The histogram shown in the lower part of FIG. 17 represents an image of the first information 5062 calculated by the color information extraction unit 5046. The horizontal axis of the histogram in FIG. 17 is color, and the vertical axis represents how many pixels of a predetermined color are included in the image data 5060.

  In step S5103 of FIG. 16, the region extraction unit 5048 of the analysis unit 5044 extracts the main region in the image data 5060. For example, the region extraction unit 5048 extracts a focused region from the image data 5060 shown in FIG. 17, and recognizes the central portion of the image data 5060 as the main region (see the main region 5064 in FIG. 18). ).

  In step S5104 in FIG. 16, the region extraction unit 5048 of the analysis unit 5044 determines a clustering target region to be implemented in step S5105. For example, as shown in the upper part of FIG. 18, the region extraction unit 5048 recognizes that part of the image data 5060 is the main region 5064 in step S5103 and extracts the main region 5064, the clustering target The first information 5062 (main first information 5066) corresponding to the area 5064 is used. The histogram shown in the lower part of FIG. 18 represents an image of the main first information 5066.

  On the other hand, when the region extraction unit 5048 has not extracted the main region 5064 in the image data 5060 in step S5103, the region extraction unit 5048 displays the first information corresponding to the entire region of the image data 5060 as shown in FIG. 5062 is determined as a clustering target. Note that there is no difference in the subsequent processing between the case where the main region 5064 is extracted and the case where it is not extracted, except that the target region for clustering is different. I will explain.

  In step S5105 of FIG. 16, the clustering unit 5050 of the analysis unit 5044 performs clustering on the main first information 5066 that is the first information 5062 of the area determined in step S5104. FIG. 19 is a conceptual diagram showing the result of clustering performed by the clustering unit 5050 on the main first information 5066 of the main region 5064 shown in FIG.

  For example, the clustering unit 5050 classifies the main first information 5066 (see FIG. 18) having 256 gradations into a plurality of clusters by the k-means method. Note that the clustering is not limited to the k-means method (k average method). In other examples, other methods such as the shortest distance method can be used.

  The upper part of FIG. 19 shows to which cluster each pixel is classified, and the histogram shown in the lower part of FIG. 19 shows the number of pixels belonging to each cluster. By the clustering by the clustering unit 5050, the 256 first main information 5066 (FIG. 18) is classified into less than 256 (three in the example shown in FIG. 19) clusters. The result of clustering can include information about the size of each cluster and information about the color of each cluster (the position of the cluster in the color space).

  In step S5106, the clustering unit 5050 of the analysis unit 5044 determines a representative color of the image data 5060 based on the clustering result. In one example, when the clustering unit 5050 obtains a clustering result as illustrated in FIG. 17, the color belonging to the maximum cluster 5074 including the most pixels among the plurality of calculated clusters is set as the representative color of the image data 5060. To do.

  When the calculation of the representative color is completed, the sentence creation unit 5052 creates a text using information on the representative color and assigns the text to the image data 5060.

  The text creation unit 5052 reads a text template for a landscape image, for example, and applies a word (for example, “2012/03/10”) corresponding to the generation date and time of the image data 5060 to {date and time} of the text template. In this case, the analysis unit 5044 can retrieve information related to the generation date and time of the image data 5060 from the storage medium and output the information to the text creation unit 5052.

  In addition, the sentence creation unit 5052 applies a word corresponding to the representative color of the image data 5060 to the {adjective} of the sentence template. The sentence creation unit 5052 reads the correspondence information from the storage unit 5028 and applies it to the sentence template. In one example, the storage unit 5028 stores a table in which colors and words are associated with each scene. The sentence creation unit 5052 can create a sentence (for example, “I found a very beautiful thing”) using words read from the table.

  FIG. 20 shows image data 5080 to which text is given by the series of processes described above.

  FIG. 21 shows an example of image data to which text is given by a series of processes similar to the above when the scene is a distant view image. In this case, the scene is classified as a distant view image, and the representative color is determined to be blue. For example, in a table in which colors and words are associated with each scene, the word “fresh” is associated with the representative color “blue”.

  FIG. 22 is a diagram illustrating an example of a table having correspondence information between colors and words. In the table of FIG. 22, a color and a word are associated with each scene of a person image (first scene image), a distant view image (second scene image), and another image (third scene image). In one example, when the representative color of the image data is “blue” and the scene is another image (third scene image), the sentence creation unit 5052 uses a word corresponding to the representative color (for example, from the correspondence information in the table). “Classy”) and select {adjective} in the sentence template.

  The correspondence table between colors and words can be set based on a color chart such as a PCCS color system, CICC color system, or NCS color system.

  FIG. 23 shows an example of a correspondence table for a distant view image (second scene image) using a color chart of the CCIC display system. FIG. 24 shows an example of a correspondence table for other images (third scene images) using a CCIC display color chart.

  In FIG. 23, the horizontal axis corresponds to the hue of the representative color, and the vertical axis corresponds to the tone of the representative color. By using the table of FIG. 23 to determine the word, not only the information on the hue of the representative color but also the information on the tone of the representative color is used to determine the word, and a text that is relatively close to the sensibility generated by humans is given. Is possible. Hereinafter, a specific text setting example in the case of a distant view image (second scene image) using the table of FIG. 23 will be described. In the case of other images (third scene images), the same setting can be made using the table of FIG.

  In FIG. 23, when the representative color is determined to be the area A5001, the representative color designation (red, orange, yellow, blue, etc.) is applied to the word in the text as it is. For example, if the hue of the representative color is “red (R)” and the tone is “Vivid Tone (V)”, the adjective “crimson” representing the color is selected.

  When the representative color is determined to be the color of the region A5002, A5003, A5004, or A5005, an adjective associated with the color is applied to the word in the text. For example, when the representative color is determined to be the color (green) of the area A5003, the adjectives associated with green, such as “comfortable” and “fresh”, are applied.

  When the representative color is determined to be the color of the area A5001 to A5005 and the tone is a vivid tone (V), a strong tone (S), a bright tone (B), or a pale tone (LT). Applies adverbs that indicate the degree before the adjectives (eg, very, pretty, etc.).

  When the representative color is determined to be the area A5006, that is, “white tone (white)”, “clean”, “clear”, and the like, which are words associated with white, are selected. Further, if the representative color is determined to be the area A5007, that is, a gray color (light gray tone: ltGY, medium gray tone: mGY, or dark gray tone: dkGY), it is a safe adjective. “Clean”, “nice”, etc. are selected. In an image in which a white or gray color, that is, an achromatic color is a representative color, various colors are often included in the entire image. Therefore, by using a word that is less related to color, it is possible to prevent a text having an inappropriate meaning from being added, and it is possible to provide a text that is relatively close to an image received from an image.

  Further, when the representative color does not belong to any of the areas A5001 to A5007, that is, when the representative color is a low tone (dark grayish tone) or black (black tone), characters having a predetermined meaning (Word or sentence) can be selected as text. Characters having a predetermined meaning include, for example, “where is here”, “a”, and the like. These words and sentences can be stored in the storage unit of the image processing apparatus as a “tweet dictionary”.

  In other words, when the representative color is determined to be low tone or black, it may be difficult to determine the hue of the entire image. Even in such a case, characters having less relation to the color are used as described above. Thus, it is possible to prevent a text having an inappropriate meaning from being added, and to add a text close to an image received from an image.

  In the above example, the case where the sentence and the word are uniquely determined according to the scene and the representative color has been described. However, the present invention is not limited to this, and exception processing is sometimes performed in the selection of the sentence and the word. it can. For example, the text may be extracted from the “tweet dictionary” once every plural times (for example, once every 10 times). As a result, the display content of the text is not necessarily patterned, so that the user can be prevented from getting bored with the display content.

  In the above example, the case where the sentence adding unit arranges the text generated by the sentence creating unit at the upper part or the lower part of the image has been described. However, the present invention is not limited to this. It can also be arranged.

  In the above example, the case where the position of the text is fixed in the image has been described. However, the present invention is not limited to this. For example, the text can be displayed so as to flow on the display unit of the image processing apparatus. Thereby, the input image is not easily affected by the text, or the text visibility is improved.

  In the above example, the case where the text is always pasted on the image has been described. However, the present invention is not limited to this. For example, in the case of a person image, the text is not pasted, and in the case of a distant view image or other images. You may make it paste a text.

  In the above example, the sentence adding unit has described the case where the display method (font, color, display position, etc.) of the text generated by the sentence creating unit is determined by a predetermined method. A variety of text display methods can be determined. Hereinafter, some examples of these methods will be described.

  In one example, the user can correct the text display method (font, color, display position) via the operation unit of the image processing apparatus. Alternatively, the user can change or delete the contents (words) of the text. In addition, the user can select not to display the entire text, that is, display / non-display of the text.

  In one example, the size of the text can be changed according to the scene of the input image. For example, when the scene of the input image is a person image, the text can be reduced, and when the scene of the input image is a distant view image or other images, the text can be increased.

  In one example, text can be highlighted and combined with image data. For example, when the input image is a person image, a balloon can be given to the person and text can be placed in the balloon.

  In one example, the display color of the text can be set with reference to the representative color of the input image. Specifically, a color having the same hue as the representative color of the input image and a different tone can be used as a text display color. As a result, it is possible to give a text that is in harmony with the input image without excessively claiming the text.

  In particular, when the representative color of the input image is white, exception processing may be performed in determining the text display color. Here, in the exception processing, for example, the text color can be set to white and the peripheral portion of the text can be set to black.

  DESCRIPTION OF SYMBOLS 11 ... Image input part 12 ... Text input part 13 ... 1st position input part 14 ... Edge detection part 15 ... Face detection part 16 ... Character size determination part 17, 17a ... Cost calculation part 18, 18b ... Area determination part 19 ... Composition unit 21, 31 ... second position input unit 100 ... imaging device 110 ... imaging unit 140, 140a, 140b ... image processing unit 160 ... storage unit

Claims (14)

An acquisition unit for acquiring image data and text data;
A detection unit for detecting an edge of the image data acquired by the acquisition unit;
An area determination unit that determines an area in which the text data is arranged in the image data based on the edge detected by the detection unit;
An image generation unit that generates an image in which the text data is arranged in an area determined by the area determination unit;
An image processing apparatus comprising: An image processing apparatus according to claim 1,
The area determination unit determines an area with few edges in the image data as an area where the text data is arranged. An image input unit for inputting image data;
An edge detection unit for detecting edges in the image data input by the image input unit;
A text input section for inputting text data;
An area determination unit that determines a synthesis area of the text data in the image data based on the edge detected by the edge detection unit;
A synthesizing unit that synthesizes the text data with the synthesis region determined by the region determining unit;
An image processing apparatus comprising: The image processing apparatus according to claim 3.
The area determination unit determines an area with few edges in the image data as the synthesis area. The image processing apparatus according to claim 3 or 4,
A cost calculation unit that calculates the cost representing the importance at each position of the image data so that the cost of the position where the edge is detected by the edge detection unit is high;
The image processing apparatus according to claim 1, wherein the region determination unit determines a region having a low cost corresponding to the synthesis region as the synthesis region based on the cost calculated by the cost calculation unit. The image processing apparatus according to claim 5.
A first position input unit for inputting a first position in the image data;
The cost calculating unit increases the cost as the position is closer to the first position input by the first position input unit, and lowers the cost as the position is farther from the first position. Processing equipment. The image processing apparatus according to claim 5 or 6,
A face detection unit for detecting a human face from the image data;
The cost calculation unit increases the cost of a region with a face detected by the face detection unit. The image processing apparatus according to any one of claims 5 to 7,
A second position input unit for inputting a second position for combining the text data;
The cost calculation unit reduces the cost of the second position input by the second position input unit. The image processing apparatus according to any one of claims 3 to 8,
An image processing apparatus, comprising: a character size determining unit that determines a character size of the text data so that all of the text of the text data can be combined in an image area of the image data. The image processing apparatus according to any one of claims 3 to 9,
The image input unit inputs image data of a moving image,
The area determination unit determines the synthesis area of the text data based on a plurality of frame images included in the image data of the moving image. Inputting image data;
Entering text data; and
Detecting an edge in the input image data;
Determining a synthesis area of the text data in the image data based on the detected edge;
Combining the text data with the determined combining region;
A program that causes a computer to execute. An image processing apparatus for inputting image data;
The image processing apparatus inputting text data;
The image processing device detecting an edge in the input image data;
The image processing device determining a synthesis area of the text data in the image data based on the detected edge;
The image processing device combining the text data with the determined combining region;
An image processing method comprising: An imaging apparatus comprising: the image processing apparatus according to claim 3. A detection unit for detecting an edge of image data;
An area determination unit that determines an arrangement area in which characters in the image data are arranged based on the position of the edge detected by the detection unit;
An image generation unit that generates an image in which the characters are arranged in the arrangement region determined by the region determination unit;
An image processing apparatus comprising:

Images