JP2009010452A - Image processing system and image processing program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing system that corrects the direction of an image to an erected direction by using an arrangement of areas in the image even when the image is input in directions other than the erected direction. <P>SOLUTION: An image decomposition means of the image processing system decomposes the inside of the image into areas based upon features of pixels, a discrimination means discriminates the areas decomposed by the image decomposition means, a calculation means calculates the angle of rotation of the image according the discrimination results of the discrimination means and the arrangement of the areas, and an image rotation means rotates the image based upon the angle of rotation calculated by the calculation means. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing system and an image processing program.

  An increasing number of image processing apparatuses are capable of processing images as digital information. In this image processing apparatus, in order to obtain a well-formed document, a target image is analyzed, and various processes are performed according to the analysis result.

  As a related technology, for example, Patent Document 1 relates to an electronic camera that captures a subject and generates a moving image or a still image, and an electronic camera that includes a display screen for monitoring. Capture the subject for the purpose of ensuring that the top and bottom of the image to be recorded with proper adaptation is always constant and that the top and bottom of the image to be displayed with constant adaptation to the change in the display screen posture is always constant. An imaging means for generating an image, an imaging attitude detection means for detecting a direction and an angle of rotation of the imaging means with respect to a predetermined attitude around a rotation axis along the imaging optical axis of the imaging means, and imaging A recording image converting means for generating a recorded image by rotating an image in the direction and angle detected by the posture detecting means, and a recording means for recording the recorded image generated by the recorded image converting means. It has been disclosed.

  For example, Patent Document 2 provides a digital camera that can automatically record the vertical direction of a person's face as a predetermined direction when a still image is recorded by shooting with a camera. The face image presence / absence determination unit determines the presence / absence of a human face image from the captured image data, the face direction determination processing unit determines the vertical direction of the captured face, and then the rotation angle. The determination unit determines how many times the recording data itself is to be rotated from the determination result of the face in the vertical direction, sends the result to the image rotation processing unit, and rotates the recording data based on the data before recording Is disclosed.

  For example, Patent Document 3 has an object to provide a print image selection device and a print image selection method capable of efficiently organizing images according to shooting scenes and selecting images to be printed. When shooting is finished at this point and a digital image is input from the digital camera to the print image selection device, an image processing filter is applied to the input image, image processing is performed, and then an image that has failed to be shot is displayed. Digital images that are automatically excluded and not excluded are organized by shooting scene, and at the time of sorting, the top and bottom of the image is detected simultaneously with the identification of the shooting scene, and belong to the same shooting scene based on this top and bottom The image is automatically rotated, and then a digital image is selected for each shooting scene. Can be performed the print order of Tsu door is disclosed.

For example, Patent Document 4 has an object of more reliably performing object detection and image content determination. When determining image content, it is determined whether a skin color region including skin color pixels is a human face. In order to determine whether the skin color area is in contact with the upper left and right sides of the image or whether there is a pixel area corresponding to both eyes or mouth, the rotation angle in the right direction of the image is 270 degrees. The image is rotated repeatedly by 90 degrees until the image is rotated, and the process of determining whether the sky blue area is in contact with the lower side of the image in order to determine whether the sky blue area consisting of sky blue pixels is empty Rotate the image 90 degrees to the left until it reaches 0 degrees. As a result, the image content is determined by rotating the image at a plurality of rotation angles. Disclosure that you can To have.
Japanese Patent Laid-Open No. 10-164426 JP 2005-260870 A JP 2006-25006 A JP 2006-107037 A

By the way, when an image is input in a direction other than erect, the direction of the image cannot be erect unless it is a limited target image.
The present invention provides an image processing system and an image processing program for erecting the direction of an image by using the arrangement of areas in the image even when the image is input in a direction other than erect. It is aimed.

The gist of the present invention for achieving the object lies in the inventions of the following items.
[1] Image decomposition means for decomposing the inside of an image into regions based on image characteristics;
Discriminating means for discriminating an area decomposed by the image decomposition means;
A calculation unit that calculates a rotation angle of the image according to a determination result by the determination unit and an arrangement of the region;
An image processing system comprising image rotation means for rotating the image based on the rotation angle calculated by the calculation means.

[2] The calculation means includes:
The image processing system according to [1], wherein a combination of regions whose arrangement relationship is uniquely determined among the regions is detected, and a rotation angle of the image is calculated according to the arrangement relationship of the combination of the regions. .

[3] The calculation means includes:
The image processing system according to [1], wherein a predetermined area among the areas is detected, and a rotation angle of the image is calculated according to a characteristic related to a direction of the area.

[4] The calculation means includes:
The image processing system according to [3], wherein when a plurality of the predetermined areas are detected, the rotation angle of the image is calculated according to the arrangement of the areas.

[5] The calculation means includes:
The image processing system according to [4], wherein when there is a difference in rotation angles calculated from the plurality of detected regions, the rotation angle of the image is calculated using the plurality of rotation angles.

[6]
Image decomposing means for decomposing the image into regions based on pixel characteristics;
Discriminating means for discriminating an area decomposed by the image decomposition means;
A calculation unit that calculates a rotation angle of the image according to a determination result by the determination unit and an arrangement of the region;
An image processing program that causes an image rotation unit to rotate the image based on the rotation angle calculated by the calculation unit.

  According to the image processing system of claim 1, even when the image is input in a direction other than the upright direction, the arrangement of the regions in the image is used, as compared with the case where the present configuration is not provided. The image orientation can be upright.

  According to the image processing system of claim 2, in addition to the effect of the image processing system of claim 1, the rotation angle of the image can be calculated more accurately by using a combination of regions whose arrangement relationship is uniquely determined. Can do.

  According to the image processing system of claim 3, in addition to the effect of the image processing system of claim 1, the rotation angle of the image can be calculated more accurately by using the feature related to the direction of the region.

  According to the image processing system of claim 4, in addition to the effect of the image processing system of claim 3, the rotation angle of the image can be calculated even when a plurality of predetermined regions are detected.

  According to the image processing system of claim 5, in addition to the effect of the image processing system of claim 4, the rotation angle of the image can be calculated even when the calculated rotation angle is different.

  According to the image processing program of claim 6, even when the image is input in a direction other than erecting, the arrangement of the regions in the image is used, as compared with the case where the present configuration is not provided. The image orientation can be upright.

Hereinafter, an example of a preferred embodiment for realizing the present invention will be described with reference to the drawings.
FIG. 1 shows a conceptual module configuration diagram of a configuration example of the present embodiment.
The module generally refers to components such as software (computer program) and hardware that can be logically separated. Therefore, the module in the present embodiment indicates not only a module in a computer program but also a module in a hardware configuration. Therefore, the present embodiment also serves as an explanation of a computer program, a system, and a method. However, for the sake of explanation, the words “store”, “store”, and equivalents thereof are used, but these words are stored in a storage device when the embodiment is a computer program, or It means to control to be stored in the storage device. In addition, the modules correspond almost one-to-one with the functions. However, in mounting, one module may be composed of one program, or a plurality of modules may be composed of one program. A plurality of programs may be used. The plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers in a distributed or parallel environment. Note that one module may include other modules. In the following, “connection” includes not only physical connection but also logical connection (data exchange, instruction, reference relationship between data, etc.).
In addition, the system or device is configured by connecting a plurality of computers, hardware, devices, and the like by communication means such as a network (including one-to-one correspondence communication connection), etc., and one computer, hardware, device. The case where it implement | achieves by etc. is also included.

  As shown in FIG. 1, the present embodiment includes an image decomposition module 11, a determination module 12, a calculation module 13, and an image rotation module 14. The calculation module 13 includes a relationship detection module 131, a region detection module 132, and a rotation angle calculation module 133.

The image decomposition module 11 is connected to the determination module 12, receives an image, decomposes the inside of the image into regions based on the image characteristics, and passes the decomposed image to the determination module 12.
Here, an image is received by an image processing apparatus having any two or more functions of a digital camera (still camera), a fax machine, a scanner, a copying machine, and a multifunction machine (scanner, printer, copying machine, fax machine, etc.) ) Or the like, or by reading an image stored in the image storage device. Then, for example, by using information on the color of each pixel that is close in position, similar pixels are divided into regions by grouping. Further, in addition to the information regarding the color of each pixel, a boundary line may be extracted, and the region may be divided along the boundary line. The region may be divided using other methods.
Further, a partial image in the image (for example, a photograph or the like arranged in a business document) may be cut out, and the cut-out partial image may be set as a target image of the present embodiment.

The determination module 12 is connected to the image decomposition module 11 and the calculation module 13, determines an area decomposed by the image decomposition module 11, and passes the determination result to the calculation module 13.
The discrimination here is so-called image recognition in which features of each region are extracted and recognized. For example, a face may be recognized from features such as color and shape using face recognition technology. In this determination, in addition to what the area is, the inclination of the area may be detected.

The calculation module 13 is connected to the determination module 12 and the image rotation module 14, calculates the rotation angle of the image according to the determination result by the determination module 12 and the arrangement of the areas, and passes the rotation angle to the image rotation module 14. .
For example, the relationship detection module 131 detects a combination of regions whose arrangement relationship is uniquely determined among the regions. Then, the rotation angle calculation module 133 may calculate the rotation angle of the image according to the arrangement relationship of the combination of the regions.
The area detection module 132 detects a predetermined area among the areas. Then, the rotation angle calculation module 133 may calculate the rotation angle of the image according to the feature related to the direction of the region. Further, when the area detection module 132 detects a plurality of predetermined areas, the rotation angle calculation module 133 may calculate the rotation angle of the image according to the arrangement of the areas. Further, the rotation angle calculation module 133 may calculate the rotation angle of the image using the plurality of rotation angles when there is a difference in the rotation angles calculated from the plurality of detected regions.

  The image rotation module 14 is connected to the calculation module 13 and rotates the image based on the rotation angle calculated by the calculation module 13.

Next, the function and operation (operation) will be described.
FIG. 2 is an example of an image 20 targeted by the present embodiment. FIG. 3 is an example of a region where the target image of the image 20 is decomposed by the image decomposition module 11.
The image decomposition module 11 receives the image 20. This image 20 is a landscape image in which mountains (islands) on the sea and the sun are reflected. The image 20 is input upside down.
The image decomposition module 11 performs region division on the image 20. In this case, the values of adjacent pixels are divided into similar groups. The decomposition results are region 21 (sea), region 22 (mountain), region 23 (mountain), region 24 (sun), and region 25 (sky).

  Next, the discrimination module 12 discriminates each area. That is, by extracting features of the region (for example, pixel values, shapes, etc.) and comparing with a dictionary (typical pattern) in the feature space, the region 21 is a sea, the region 22 is a mountain, It is determined that 23 is a mountain, the region 24 is the sun, and the region 25 is empty. Other determination methods, for example, determination by rules may be performed.

Next, the rotation angle of the image 20 is calculated by the calculation module 13.
That is, the relationship detection module 131 searches for a combination of the determination results in the relationship table 80. Here, the relationship table 80 will be described with reference to FIG. The relationship table 80 stores a combination column 81 and an upward column 82 in association with each other, and is searched by the relationship detection module 131. That is, the combination column 81 stores a combination of regions whose arrangement relation is uniquely determined, and the upward column 82 stores the region located in the upward direction among the regions.

  If the combination of the discrimination results by the discrimination module 12 is in the combination column 81, the relationship detection module 131 takes out the one stored in the corresponding upward column 82. Then, the rotation angle calculation module 133 calculates the rotation angle so that the upper column 82 is in the upward direction. For example, in the case of FIG. 3, since the region 25 and the region 21 are a combination of “sky” and “sea”, the rotation angle (that is, 180 degrees) is calculated so that the “sky” is upward. Further, since the region 24 and the region 21 are a combination of “sun” and “sea”, the rotation angle (ie, 180 degrees) is calculated so that the “sun” is directed upward.

  Further, the area detection module 132 searches whether the determination result is in the directionality table 90. Here, the directionality table 90 will be described with reference to FIG. The directionality table 90 stores the target column 91 and the rule column 92 in association with each other, and is searched by the area detection module 132. That is, the target column 91 stores a region where the rotation angle can be calculated only by detecting the direction of one region, and the rule column 92 calculates the rotation angle using the result of the direction detection of that region. Remember the rules for.

  If the determination result by the determination module 12 is in the target column 91, the area detection module 132 takes out the rule stored in the rule column 92 corresponding thereto. Then, the rotation angle calculation module 133 calculates the rotation angle according to the rule. For example, in the case of FIG. 3, since the discrimination result of the region 22 is “mountain”, the rotation angle is calculated so that the “convex direction is up”. Here, the convex direction refers to the direction of a vertex on the diagonal of the longest side of the region. The convex direction of the region 22 is the downward direction. Further, since the determination result of the region 25 is “empty”, the rotation angle is calculated so that “the opening direction is up”. Here, the opening direction refers to the direction of the longest side facing the periphery of the image 20. The opening direction of the region 25 is a downward direction.

  In the above-described example, even when the region detection module 132 detects a plurality of predetermined regions, there is no contradiction in the rotation angle calculated from them. For example, in the case of an image taken through a fisheye lens, from a plurality of mountains The calculated rotation angles are different. Therefore, in this case, processing by the relationship detection module 131 is also performed, and the rotation angle is calculated according to the result (area arrangement relationship).

  Further, when there is a difference in rotation angles calculated from a plurality of detected areas, the rotation angle calculation module 133 may calculate the rotation angle of the image using the plurality of rotation angles. For example, the average value of the calculated plurality of rotation angles is set as the rotation angle of the image, and many of the calculated plurality of rotation angles are set as the rotation angle of the image.

A processing example according to the present embodiment will be described with reference to the flowchart shown in FIG.
In step S41, the image input by the image decomposition module 11 is divided into regions.
In step S42, the determination module 12 determines the area divided in step S41.
In step S43, the calculation module 13 calculates the rotation angle of the image using the determination result in step S42. The process of step S43 is demonstrated using the flowchart shown to FIGS.
In step S44, the image rotation module 14 rotates the image based on the rotation angle calculated in step S43.

An example of processing (step S43) by the calculation module 13 will be described using the flowchart shown in FIG.
In step S51, the relationship detection module 131 uses the determination result in step S42 to detect a combination of regions in which the arrangement relationship is uniquely determined.
In step S52, the rotation angle calculation module 133 calculates the rotation angle of the image using the combination of areas detected in step S51.

An example of processing (step S43) by the calculation module 13 will be described using the flowchart shown in FIG.
In step S61, the relationship detection module 131 uses the determination result in step S42 to detect a region in which the rotation angle of the image can be calculated by detecting the direction of the region.
In step S62, the rotation angle calculation module 133 calculates the rotation angle of the image using the direction of the area detected in step S61.

An example of processing (step S43) by the calculation module 13 will be described using the flowchart shown in FIG.
In step S71, the relationship detection module 131 detects a combination of regions in which the arrangement relationship is uniquely determined using the determination result in step S42.
In step S72, the relationship detection module 131 detects a region where the rotation angle of the image can be calculated by detecting the direction of the region using the determination result of step S42.
In step S73, the rotation angle calculation module 133 calculates the rotation angle of the image using the detection results in steps S71 and S72.
Note that the order of step S71 and step S72 may be reversed. Moreover, after performing any one process of step S71 or step S72, when some contradiction arises in step S73 (for example, when there are a plurality of detection results in step S71 or step S72 and they are different from each other), You may make it perform the process different from any one process of step S71 or step S72 performed previously. In that case, the rotation angle of the image may be calculated from the calculated plurality of rotation angles.

  With reference to FIG. 10, a hardware configuration example of the present embodiment will be described. The configuration illustrated in FIG. 10 is configured by, for example, a personal computer (PC), and illustrates a hardware configuration example including a data reading unit 1017 such as a scanner and a data output unit 1018 such as a printer.

  A CPU (Central Processing Unit) 1001 describes the execution sequence of each module such as the image decomposition module 11, the determination module 12, the calculation module 13, and the image rotation module 14 described in the above embodiment. It is a control part which performs the process according to a computer program.

  A ROM (Read Only Memory) 1002 stores programs used by the CPU 1001, calculation parameters, and the like. A RAM (Random Access Memory) 1003 stores programs used in the execution of the CPU 1001, parameters that change as appropriate during the execution, and the like. These are connected to each other by a host bus 1004 including a CPU bus.

  The host bus 1004 is connected to an external bus 1006 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 1005.

  A keyboard 1008 and a pointing device 1009 such as a mouse are input devices operated by an operator. The display 1010 includes a liquid crystal display device or a CRT (Cathode Ray Tube), and displays various types of information as text and image information.

  An HDD (Hard Disk Drive) 1011 includes a hard disk, drives the hard disk, and records or reproduces a program executed by the CPU 1001 and information. The hard disk stores an input image, a decomposed image by the image decomposition module 11, and the like. Further, various computer programs such as various other data processing programs are stored.

  The drive 1012 reads out data or a program recorded in a removable recording medium 1013 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, and the data or program is read out as an interface 1007 and an external bus 1006. , The bridge 1005, and the RAM 1003 connected via the host bus 1004. The removable recording medium 1013 can also be used as a data recording area similar to a hard disk.

  The connection port 1014 is a port for connecting the external connection device 1015 and has a connection unit such as USB and IEEE1394. The connection port 1014 is connected to the CPU 1001 and the like via the interface 1007, the external bus 1006, the bridge 1005, the host bus 1004, and the like. A communication unit 1016 is connected to a network and executes data communication processing with the outside. The data reading unit 1017 is a scanner, for example, and executes document reading processing. The data output unit 1018 is a printer, for example, and executes document data output processing.

  Note that the hardware configuration illustrated in FIG. 10 illustrates one configuration example, and the present embodiment is not limited to the configuration illustrated in FIG. 10, and is a configuration that can execute the modules described in the present embodiment. I just need it. For example, some modules may be configured with dedicated hardware (for example, Application Specific Integrated Circuit (ASIC), etc.), and some modules are in an external system and connected via a communication line Alternatively, a plurality of the systems shown in FIG. 10 may be connected to each other via communication lines so as to cooperate with each other. Further, it may be incorporated in a copying machine, a fax machine, a scanner, a printer, a multifunction machine, or the like.

In the above embodiment, the data structures shown in FIGS. 8 and 9 are not limited to these data structures, and may be other data structures. For example, a link structure or the like may be used instead of the table structure. Further, the data items are not limited to those shown in the drawings, and may have other data items.
Further, the result of rotation by the image rotation module 14 may be presented to the operator, and it may be asked whether or not to accept the rotation result. In response to the selection operation of the operator (selection of “OK” or “NG”), the rotated image is stored in the case of “OK”, and the rotation angle is designated by the operator in the case of “NG”. The rotation by the image rotation module 14 may be performed according to the rotation angle. In the case of “NG”, when the relationship detection module 131 and the region detection module 132 have obtained a plurality of detection results, the rotation angle calculated by the rotation angle calculation module 133 is calculated using the detection results that are not employed. If the calculated rotation angle is different from the previous rotation angle, the image rotation module 14 may be rotated.

The program described above may be provided by being stored in a recording medium, or the program may be provided by communication means. In that case, for example, the above-described program may be regarded as an invention of a “computer-readable recording medium recording the program”.
The “computer-readable recording medium on which a program is recorded” refers to a computer-readable recording medium on which a program is recorded, which is used for program installation, execution, program distribution, and the like.
The recording medium is, for example, a digital versatile disc (DVD), which is a standard established by the DVD Forum, such as “DVD-R, DVD-RW, DVD-RAM,” and DVD + RW. Standards such as “DVD + R, DVD + RW, etc.”, compact discs (CDs), read-only memory (CD-ROM), CD recordable (CD-R), CD rewritable (CD-RW), etc. MO), flexible disk (FD), magnetic tape, hard disk, read only memory (ROM), electrically erasable and rewritable read only memory (EEPROM), flash memory, random access memory (RAM), etc. It is.
The program or a part of the program may be recorded on the recording medium for storage or distribution. Also, by communication, for example, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a wired network used for the Internet, an intranet, an extranet, etc., or wireless communication It may be transmitted using a transmission medium such as a network or a combination of these, or may be carried on a carrier wave.
Furthermore, the program may be a part of another program, or may be recorded on a recording medium together with a separate program. Moreover, it may be divided and recorded on a plurality of recording media. Further, it may be recorded in any manner as long as it can be restored, such as compression or encryption.

It is a conceptual module block diagram about the structural example of this Embodiment. It is an example of the image which this Embodiment makes object. It is an example of the area | region decomposed | disassembled by the image decomposition | disassembly module. It is a flowchart which shows the process example by this Embodiment. It is a flowchart which shows the process example by a calculation module. It is a flowchart which shows the process example by a calculation module. It is a flowchart which shows the process example by a calculation module. It is explanatory drawing which shows the data structure example of a relationship table. It is explanatory drawing which shows the example of a data structure of a directionality table. It is a block diagram which shows the hardware structural example of the computer which implement | achieves this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 11 ... Image decomposition module 12 ... Discrimination module 13 ... Calculation module 14 ... Image rotation module 80 ... Relationship table 90 ... Directionality table 131 ... Relationship detection module 132 ... Area detection module 133 ... Rotation angle calculation module 1001 ... CPU
1002 ... ROM
1003 ... RAM
DESCRIPTION OF SYMBOLS 1004 ... Host bus 1005 ... Bridge 1006 ... External bus 1007 ... Interface 1008 ... Keyboard 1009 ... Pointing device 1010 ... Display 1011 ... HDD
DESCRIPTION OF SYMBOLS 1012 ... Drive 1013 ... Removable recording medium 1014 ... Connection port 1015 ... External connection apparatus 1016 ... Communication part 1017 ... Data reading part 1018 ... Data output part

Claims (6)

Image decomposing means for decomposing the inside of the image into regions based on the characteristics of the image;
Discriminating means for discriminating an area decomposed by the image decomposition means;
A calculation unit that calculates a rotation angle of the image according to a determination result by the determination unit and an arrangement of the region;
An image processing system comprising image rotation means for rotating the image based on the rotation angle calculated by the calculation means. The calculating means includes
The image processing system according to claim 1, wherein a combination of regions whose arrangement relation is uniquely determined among the areas is detected, and a rotation angle of the image is calculated according to the arrangement relation of the combination of the areas. . The calculating means includes
The image processing system according to claim 1, wherein a predetermined area is detected from the areas, and a rotation angle of the image is calculated in accordance with a characteristic related to a direction of the area. The calculating means includes
The image processing system according to claim 3, wherein when a plurality of the predetermined areas are detected, the rotation angle of the image is calculated according to the arrangement of the areas. The calculating means includes
The image processing system according to claim 4, wherein when there is a difference in rotation angles calculated from the plurality of detected areas, the rotation angle of the image is calculated using the plurality of rotation angles. Computer
Image decomposing means for decomposing the image into regions based on pixel characteristics;
Discriminating means for discriminating an area decomposed by the image decomposition means;
A calculation unit that calculates a rotation angle of the image according to a determination result by the determination unit and an arrangement of the region;
An image processing program that causes an image rotation unit to rotate the image based on the rotation angle calculated by the calculation unit.

Images