CN104346613A - Image processing apparatus and image processing method - Google Patents

Abstract

The image processing device of the present invention includes: obtaining means for obtaining a print image obtained by imaging a print with code information attached to a frame part; A frame image area corresponding to the print is determined as a reading area for reading the code information; a reading unit, which reads the code information from the reading area determined by the determining unit.

Description

Image processing device and image processing method

technical field

The present invention relates to an image processing device, an image processing method, and a program.

Background technique

Currently known is a device for generating coded information that encodes predetermined information as a regular arrangement of pixel sets (US8,186,594B2).

For example, the code information is captured by an imaging device such as a mobile phone or a smartphone while being formed on a recording medium such as paper. The imaging device acquires original predetermined information represented by the code information by performing predetermined decoding processing on the image information of the code information captured by the image.

Contents of the invention

According to the present invention, it is possible to appropriately detect a frame portion from an image obtained by imaging a print.

According to an embodiment of the present invention, an image processing device is characterized by comprising: an acquisition unit that acquires a print image obtained by imaging a print with code information added to a frame portion; A frame image area corresponding to the footprint of the frame portion in the footprint image obtained by the unit is determined as a reading area for reading the code information; a reading unit that reads the code from the reading area determined by the determination unit information.

According to an embodiment of the present invention, there is an image processing method using an image processing device, and the image processing method is characterized by including the step of acquiring a print image obtained by imaging a print with code information attached to a frame portion. ; Determine the frame image area corresponding to the imprint of the above-mentioned frame part in the obtained imprint image as a determination step for reading the above-mentioned code information; read the above-mentioned code information from the determined above-mentioned reading area Take steps.

Description of drawings

FIG. 1 is a block diagram showing a schematic configuration of a mobile terminal to which a first embodiment of the present invention is applied.

FIG. 2 is a flowchart showing an example of the operation of the code reading process of the mobile terminal shown in FIG. 1 .

FIG. 3 is a diagram for explaining code reading processing in FIG. 2 .

FIG. 4 is a diagram schematically showing an example of an image in the code reading process of FIG. 2 .

FIG. 5 is a diagram schematically showing an example of an image in the code reading process of FIG. 2 .

FIG. 6 is a diagram schematically showing an example of an image in the code reading process of FIG. 2 .

FIG. 7 is a diagram schematically showing an example of an image in the code reading process of FIG. 2 .

FIG. 8 is a block diagram showing a schematic configuration of a mobile terminal to which a second embodiment of the present invention is applied.

FIG. 9 is a flowchart showing an example of the operation of the code reading process of the mobile terminal in FIG. 8 .

FIG. 10 is a diagram schematically showing an example of an image in the code reading process of FIG. 9 .

FIG. 11 is a diagram schematically showing an example of an image in the code reading process of FIG. 9 .

FIG. 12 is a block diagram showing a schematic configuration of a mobile terminal to which Embodiment 3 of the present invention is applied.

FIG. 13 is a flowchart showing an example of the operation of the code reading process of the mobile terminal shown in FIG. 12 .

FIG. 14 is a diagram for explaining code reading processing in FIG. 13 .

FIG. 15 is a diagram schematically showing an example of an image in the code reading process of FIG. 13 .

FIG. 16 is a diagram schematically showing an example of an image in the code reading process of FIG. 13 .

Detailed ways

Hereinafter, specific aspects of the present invention will be described using the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 1 is a block diagram showing a schematic configuration of a mobile terminal 100 to which the first embodiment of the present invention is applied.

As shown in FIG. 1 , the mobile terminal 100 includes a central control unit 1, a memory 2, an imaging unit 3, an imaging control unit 4, an image data generation unit 5, a code processing unit 6, an action processing unit 7, a display unit 8, and a display control unit. 9. Voice communication unit 10, communication control unit 11, operation input unit 12, etc.

In addition, the central control unit 1, the memory 2, the imaging unit 3, the imaging control unit 4, the image data generation unit 5, the code processing unit 6, the motion processing unit 7, the display control unit 9, the voice communication unit 10, and the communication control unit 11 via Bus 13 connection.

In addition, the mobile terminal 100 is constituted by, for example, a camera, a mobile phone, a mobile station used in a mobile communication network such as a PHS (Personal Handyphone System), a PDA (Personal Digital Assistant), and the like.

The central control unit 1 controls each unit of the mobile terminal 100 . Specifically, the central control unit 1 includes a CPU (central processing unit, not shown) that controls each unit of the mobile terminal 100 , and performs various control operations in accordance with various processing programs (not shown) for the mobile terminal 100 .

The memory 2 is constituted by, for example, DRAM (Dynamic Random Access Memory) or the like. In addition, the memory 2 includes a buffer memory temporarily storing data processed by the central control unit 1 and the code processing unit 6, etc., a work memory of the central control unit 1, etc., and various programs related to the functions of the mobile terminal 100 are stored. and program memory for data, etc. (both are omitted from the illustration).

The imaging unit 3 images the imprint Si (see FIG. 3A ) of the seal S stamped on the recording medium P. As shown in FIG.

The stamp S forms a frame with a polygonal shape (such as a square) around a predetermined mark for leaving the imprint Si on the recording medium P, and is formed so that predetermined information can be arranged in a regular arrangement of pixel sets in the frame. The encoded code information Sc is attached to the frame portion Sw of the print Si.

The imprint Si remains on the recording medium P by stamping the seal S on the recording medium P, and a frame portion Sw corresponding to a polygonal frame remains around the stamp image Sp formed on the stamp surface.

The frame portion Sw is attached with a plurality of code information Sc obtained by encoding predetermined information in a regular array of pixel sets. That is, in the frame portion Sw, code information Sc is added to at least two sides Sa, Sa among a plurality of sides Sa, . . . of approximately equal length. Specifically, the same code information Sc is added to each of the four sides Sa, . That is, the code information Sc is embedded multiple times in the imprint Si. In addition, the four corners of the square frame portion Sw are respectively attached with marks Sm for detecting a predetermined shape of the vertex C (for example, a square, etc.).

For example, code information Sc is added in a predetermined direction from a position separated by a predetermined interval from the mark Sm at a predetermined position so that the width direction of each side Sa of the frame portion Sw is substantially along the center side of the side Sa. Extending direction (vertical direction substantially perpendicular to the width direction).

Here, code information Sc is obtained by encoding original predetermined information (eg, URL, etc.) according to a predetermined encoding format (eg, Reed-Solomon code, Golay code, etc.). For example, the code information Sc regularly arranges a set of white pixels with a pixel value of "1" and a set of black pixels with a pixel value of "0" in predetermined dimensions.

In addition, in this embodiment, it is assumed that when stamping the seal S on the recording medium P, stamping is performed in a state where force is not applied substantially uniformly to the entire surface of the stamp surface. Therefore, for example, part of the frame portion Sw of imprinted Si (for example, the lower part in FIG. 4A ) swells, and a part of imprinted Si (for example, the upper left part in FIG. 4A ) is voided.

In addition, the imaging unit 3 includes a lens unit 3 a and an electronic imaging unit 3 b.

The lens unit 3 a is composed of a plurality of lenses such as a zoom lens and a focus lens.

The electronic imaging unit 3 b is composed of an image sensor such as CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor), for example, and converts optical images passed through various lenses of the lens unit 3 a into two-dimensional image signals.

In addition, although illustration is omitted, the imaging unit 3 may include a diaphragm for adjusting the amount of light passing through the lens unit 3a.

The imaging control unit 4 controls imaging of a subject by the imaging unit 3 . That is, the imaging control unit 4 includes a timing generator, a driver, and the like, which are not shown in the drawings. In addition, the imaging control unit 4 scans and drives the electronic imaging unit 3b through a timing generator and a driver, and converts the optical image into a two-dimensional image signal through the electronic imaging unit 3b at each predetermined cycle. Frame images are read screen by screen in the area and output to the image data generation unit 5 .

In addition, the imaging control unit 4 performs adjustment control of imaging conditions of a subject such as AF (auto focus processing), AE (auto exposure processing), and AWB (auto white balance).

The image data generator 5 adjusts the gain appropriately for each color component of RGB with respect to the signal of the analog value of the frame image transmitted from the electronic imaging unit 3b, and then performs sample-holding by a sample-and-hold circuit (not shown). A /D converter (not shown) converts it into digital data, performs color processing including pixel interpolation processing and γ correction processing by a color processing circuit (not shown), and generates a digital value luminance signal Y and color difference signal Cb, Cr (YUV data).

Then, the image data generation unit 5 sequentially outputs the generated YUV data of each frame image to the memory 2 and stores them in the memory 2 .

The code processing unit 6 includes an image acquisition unit 6a, a binarization unit 6b, a straight line estimation unit 6c, a frame detection unit 6d, and an information reading unit 6e.

In addition, each part of the code processing part 6 is comprised by predetermined logic circuits, for example, but this structure is an example, and it is not limited to this.

The image acquisition unit 6 a sequentially acquires captured images Ia obtained by imaging imprints Si imprinted on the recording medium P (see FIG. 4 ).

That is, the image acquisition unit 6a acquires a captured image (imprint image) Ia of an imprint Si obtained by adding code information Sc to a frame portion Sw having a predetermined width around a predetermined print image Sp. Specifically, the image acquisition unit 6 a acquires, from the memory 2 , a copy of image data of a predetermined resolution of the captured image Ia generated by the image data generation unit 5 by imaging the print Si by the imaging unit 3 .

The binarization unit 6b generates a first binarized image Ib (see FIG. 4b ).

That is, the binarization unit 6b performs binarization processing (for example, an adaptive binarization process) that performs binarization at a predetermined threshold on the luminance component Y of the image data (YUV data) of the captured image Ia acquired by the image acquisition unit 6a. processing, etc.) to generate the image data of the first binarized image Ib.

In addition, the binarization unit 6b generates a second binarized image Id (see FIG. 6C ). That is, the binarization unit 6b performs binarization in accordance with a predetermined threshold value on the luminance component Y of the image data (YUV data) of the map-converted image Ic generated by the map conversion unit d2 of the frame detection unit 6d. processing (for example, adaptive binarization processing, etc.) to generate image data of the second binarized image Id.

In addition, the above-mentioned binarization processing is a well-known technique, so detailed description thereof will be omitted here.

The straight line estimating unit 6c configures the outer contour of the frame portion Sw corresponding to the polygonal frame of the stamp S in the captured image Ia acquired by the image acquiring unit 6a, and estimates a predetermined number of points corresponding to the number of corners of the polygonal frame. Straight L. Specifically, the straight line estimation unit 6c includes a contour specifying unit c1 and a straight line specifying unit c2.

The contour specifying unit c1 specifies a polygonal convex hull region A1 corresponding to the outer contour of the frame portion Sw of the footprint Si.

That is, the contour specifying unit c1 specifies a polygonal convex hull region A1 corresponding to the outer contour of the frame portion Sw of the captured image Ia acquired by the image acquiring unit 6a (see FIG. 4C ).

Specifically, for example, the contour specifying unit c1 obtains the image data of the first binarized image Ib corresponding to the captured image Ia generated by the binarizing unit 6b, and performs convex hull processing on the image data, whereby A set of black pixels with a pixel value "0" within the range calculates a plurality of line segments connecting pixels constituting the outermost outline. As a result, black pixels with a pixel value of "0" existing within a predetermined range are surrounded by a plurality of line segments, and the polygonal area formed by these line segments becomes a convex hull area A without concave portions. At this time, the contour specifying unit c1 forms a plurality of convex hull regions A by changing the range to be processed within the binarized image.

Then, the contour determination unit c1 determines the convex hull region A having the largest area among the formed plurality of convex hull regions A, ..., as a polygonal shape (for example, a hexagonal shape) corresponding to the outer contour of the frame portion Sw of the imprint Si. Convex hull area A1.

In addition, the content of the above-mentioned convex hull processing is an example, it is not limited to this, and it can change arbitrarily suitably. In addition, the convex hull region A is a well-known technology, so detailed description is omitted here.

The straight line specifying unit c2 specifies a predetermined number of straight lines L constituting the outer contour of the frame image Wa corresponding to the frame portion Sw of the imprint Si (see FIG. 6A ).

That is, the straight line specifying unit c2 specifies the outer sides of the frame image Wa based on the positions of a plurality of (for example, six) vertices B constituting the convex hull area A1 of the polygonal shape (for example, hexagonal shape) specified by the outline specifying unit c1. A predetermined number of straight lines (for example, four straight lines constituting the outer contour of a square) L of the outline. More specifically, the straight line specifying unit c2 is based on pixels overlapping each of a plurality of straight lines L passing through any two vertices B, B constituting the polygonal convex hull area A1 and each of the straight lines L and the polygonal convex hull area A1. At least one of the relative relationship between the number of adjacent straight lines L determines the predetermined number of straight lines L constituting the outer contour of the frame image Wa.

That is, the straight line specifying unit c2 specifies the convex hull region constituting the polygonal shape among the plurality of straight lines L, . A straight line L in which a plurality of pixels of A1 overlap more than a predetermined value is determined as the straight line L constituting the outer contour of the frame image Wa.

Specifically, the straight line specifying unit c2 performs line detection processing based on the RANSAC method on pixels constituting the polygonal convex hull region A1 specified by the outline specifying unit c1, thereby specifying the straight line L constituting the outer contour of the frame image Wa. For example, the straight line specifying unit c2 selects any two vertices B, B from among the six vertices B, ... constituting the convex hull area A1, and specifies a straight line L connecting these two vertices B, B as a frame. Candidates for the straight line L of the square outer contour of the image Wa (candidate straight line L). Then, the straight line specifying unit c2 calculates the number of pixels overlapping the plurality of pixels constituting the convex hull region A1 for all the specified straight line candidates L, and determines the straight line candidate L whose calculated number of pixels is larger than a predetermined value as the frame image. The straight line L of the outer contour of Wa. Thus, for example, the candidate straight line La corresponding to a portion of the imprint Si where the void occurs, and the candidate straight line Lc other than the relatively short candidate straight line Lb are determined as the straight line L constituting the outer contour of the frame image Wa (see FIG. 5A ).

In addition, since the line detection processing of the RANSAC method is a well-known technique, detailed description is omitted here.

In addition, the straight line specifying unit c2 specifies, among the plurality of straight lines L, . A straight line L whose angle is approximately equal to the inner angle of the polygonal frame, weights the pixels that overlap the polygonal convex hull area A1 among the pixels constituting the straight line L, and calculates the evaluation value of each straight line L, and calculates The straight line L with a high evaluation value obtained is determined as the straight line L constituting the outer contour of the frame image Wa.

Specifically, the line specifying unit c2 specifies, for example, a candidate line Ld whose angle with an adjacent line L is approximately equal to the inner angle (90°) of the square frame of the stamp S for all candidate lines L. Then, the line specifying unit c2 adds weights to pixels overlapping the polygonal convex hull region A1 among the pixels constituting the specified candidate line Ld, and calculates an evaluation value for each candidate line L according to a predetermined calculation formula. For example, when a part of the frame image Wa of the imprint Si (for example, the lower part in FIG. , B and the candidate straight line L is determined, it becomes a state that three candidate straight lines L, . . . are determined. Then, for each of these three candidate straight lines L, ... , the straight line determination unit c2 calculates the candidate straight lines L adjacent to each end (in FIG. The angle formed by the straight lines Lc) determines the candidate straight line L whose calculated angle is approximately 90° (in FIG. 5B, the candidate straight line Ld indicated by the dotted line). Furthermore, the straight line specifying unit c2 specifies, among the pixels constituting the specified candidate straight line L, pixels overlapping the polygonal convex hull region A1 (for example, in FIG. The pixels are weighted, and the evaluation value is calculated according to a predetermined calculation formula.

Then, the line specifying unit c2 compares the calculated evaluation values of the candidate straight lines L, and specifies the candidate straight line Ld with the highest evaluation value as the straight line L constituting the outer contour of the frame image Wa. For example, the candidate straight line Ld indicated by the dashed-dotted line does not have a shape along the edge of the polygonal convex-hull area A1, but the pixels on the left and right end sides overlap with the polygonal-shaped convex-hull area A1. The other candidate straight line Ld indicated by the dotted line is higher and determined as the straight line L constituting the outer contour of the frame image Wa.

In this way, the straight line estimating unit 6c estimates four straight lines L, . Specifically, the straight line estimation unit 6c estimates the straight line L1 corresponding to the upper side, the straight line L2 corresponding to the lower side, the straight line L3 corresponding to the left side, and the straight line L4 corresponding to the right side as straight lines L corresponding to the upper, lower, left, and right sides of the square (see FIG. 6A ).

The frame detection unit 6d detects the frame image Wa of the captured image Ia of the footprint Si composed of a predetermined number of straight lines L estimated by the straight line estimation unit 6c. Specifically, the frame detection unit 6d includes a vertex determination unit d1 and a map conversion unit d2.

The vertex specifying unit d1 specifies the vertex C of the frame image Wa of the captured image Ia (see FIG. 6B ).

That is, the vertex determination unit d1 determines a predetermined number of points C intersecting the predetermined number of straight lines L determined by the straight line determination unit c2 as vertices C of the frame portion Sw (frame image Wa) of the footprint Si. Specifically, the vertex specifying unit d1 specifies four points formed by the intersection of adjacent straight lines L among the four straight lines L constituting the outer contour specified by the straight line specifying unit c2 in the captured image Ia as Vertex C of frame image Wa. At this time, between the non-intersecting straight lines L among the four straight lines L, ... (for example, the straight line L1 corresponding to the upper side and the straight line L3 corresponding to the left side, etc.), the vertex specifying part d1 extends at least one of the straight lines L in a predetermined direction. to find the point of intersection.

Then, the frame detection unit 6d detects the frame image Wa of the captured image Ia based on the predetermined number of vertices C specified by the vertex specifying unit d1. That is, the frame detection unit 6 d detects the region having the identified four points as the vertices C as the frame portion Sw of the footprint Si (frame image Wa).

In addition, the vertex specifying unit d1 may specify the vertex C of the frame portion Sw (frame image Wa) based on the coordinate position of the mark image Ma corresponding to the mark Sm of the imprint Si in the captured image Ia.

That is, for example, as shown in FIG. 7A , when a part of the imprint Si (for example, the upper right part) bleeds out, this part becomes a black pixel with a pixel value "0" at the time of binarization, and it may not be properly The straight line L constituting the outer contour of the frame image Wa is estimated from the captured image Ia of the footprint Si. Therefore, on the print surface, utilizing the fact that there are vertices of a polygonal frame near the mark Sm (within a predetermined range), the vertex specifying unit d1 determines the frame image Wa in consideration of the coordinate position of the mark image Ma in the captured image Ia. Vertex C.

Specifically, for example, the vertex specifying unit d1 prepares a pattern image Pa corresponding to the shape of the mark Sm (see FIG. 7B ), and utilizes feature information (such as SIFT (Scale-Invariant Features Transform: Scale-Invariant Features Transform) feature of the pattern image Pa to amount) to identify a region including a logo image Ma similar to the pattern image Pa in the captured image Ia. Then, the vertex specifying unit d1 specifies the vertex C of the frame image Wa within a predetermined range based on the coordinate position of the marker image Ma in the captured image Ia. Thereby, the vertex specifying unit d1 can specify the vertex C of the corresponding frame image Wa from the portion where the blotch Si occurs, and the frame image Wa of the captured image Ia can be appropriately detected by the frame detecting unit 6d (see FIG. 7C ).

The map conversion unit d2 performs map conversion processing for generating a map-converted image Ic (see FIG. 6B ).

That is, the mapping conversion unit d2 performs mapping conversion processing on the captured image Ia acquired by the image acquisition unit 6a based on the predetermined number of vertices C specified by the vertex specifying unit d1, and generates a polygonal map-transformed image Ic. Specifically, the mapping conversion unit d2 calculates the coordinate positions of the four vertices C, . . . of the four vertices C, . Coordinate transformation formula. Then, the mapping transformation unit d2 performs mapping transformation processing on the captured image Ia of the footprint Si according to the calculated coordinate transformation formula, and generates a map-transformed image in which the outer shape of the frame image Wa corresponding to the frame portion Sw of the footprint Si is transformed into a square. Ic.

Then, the frame detection unit 6d detects a square frame portion Sw (frame image Wa) corresponding to the frame of the stamp S in the map-converted image Ic (polygonal captured image) generated by the map conversion unit d2.

In addition, the above-mentioned mapping conversion processing is a well-known technique, and a detailed description thereof will be omitted here.

The information reading unit 6e performs reading processing, that is, reading original predetermined information from the code information Sc.

That is, the information reading unit 6e reads predetermined information from the code information Sc in the frame portion Sw (frame image Wa) of the captured image Ia of the imprint Si detected by the frame detecting unit 6d. Specifically, the information reading unit 6e reads predetermined information from the code information Sc in the frame image Wa of the second binary image Id corresponding to the mapped image Ic generated by the mapping converting unit d2.

For example, the information reading unit 6e detects two substantially parallel edges constituting the frame image Wa detected by the frame detecting unit 6d in the second binarized image Id, and connects the intermediate points of these two edges. Lines of a predetermined shape (such as a square, for example) formed by rising up are defined as a reading area D of the code information Sc (see FIG. 6C ). Then, the information reading unit 6e scans along a predetermined direction from a predetermined position (for example, the upper left corner) of the reading area D, and respectively determines a set of white pixels with a pixel value "1" and a set of black pixels with a pixel value "0". The coordinate position where the collection of pixels exists. The information reading unit 6e performs decoding processing corresponding to the coding method of the code information Sc for the determined arrangement of the set of white pixels and the set of black pixels, and reads the original predetermined information represented by the code information Sc (for example, URL wait).

At this time, the information reading unit 6e performs reading processing for each area corresponding to each side Sa of the frame portion Sw in the reading area D. As shown in FIG. That is, reading of the original predetermined information from the code information Sc is performed at the number of times (for example, 4 times) corresponding to the number of sides Sa of the frame portion Sw. Here, the same code information Sc is attached to each side Sa of the frame portion Sw. Therefore, the information reading unit 6e may be configured to read out a plurality of original predetermined information after detecting two or more original predetermined information, for example. In this case, it is determined that the predetermined information can be properly read.

In addition, since the above-mentioned reading process is a well-known technique, detailed description is abbreviate|omitted here.

The operation processing part 7 executes a predetermined operation according to the original predetermined information of the code information Sc read by the information reading part 6e.

That is, the operation processing unit 7 controls the execution of the processing corresponding to the predetermined information when the information reading unit 6 e has read a predetermined number or more of predetermined information. Specifically, for example, when a URL is read as predetermined information, the operation processing unit 7 controls the communication control unit 11 to access a specific web page on the Internet specified by the acquired URL. Then, the operation processing unit 7 controls the display control unit 9, the voice communication unit 10, etc. to execute various processes according to execution instructions of various processes set in advance (for example, playback of specific sounds and images, etc.).

The display unit 8 is composed of, for example, a liquid crystal display panel or the like, and displays an image captured by the imaging unit 3 (for example, a live image, etc.) on a display screen based on a video signal from the display control unit 9 .

The display control unit 9 performs control to read out display image data temporarily stored in the memory 2 and display it on the display unit 8 .

Specifically, the display control unit 9 includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. In addition, under the control of the central control unit 1, the digital video encoder reads from the VRAM at a predetermined playback frame rate (for example, 30 fps) via the VRAM controller. The luminance signal Y and the color difference signals Cb and Cr are based on these data to generate a video signal and output it to the display unit 8 .

For example, the display control unit 9 displays a plurality of frame images captured by the imaging unit 3 and the imaging control unit 4 and generated by the image data generation unit 5 on the display unit 8 while updating them sequentially at a predetermined display frame rate. superior.

The audio communication unit 10 communicates with an external user of an external device connected via the communication network N.

Specifically, the audio communication unit 10 includes a microphone 10a, a speaker 10b, a data conversion unit 10c, and the like. In addition, the voice communication unit 10 performs A/D conversion processing on the user's speech voice input from the microphone 10a through the data conversion unit 10c, outputs the speech voice data to the central control unit 1, and under the control of the central control unit 1 The data conversion unit 10c performs D/A conversion processing on audio data such as answering audio data output and input from the communication control unit 11, and outputs it from the speaker 10b.

The communication control unit 11 transmits and receives data via the communication network N and the communication antenna 11a.

That is, the communication antenna 11a is capable of performing a predetermined communication method (for example, W-CDMA (Wideband Code Division Multiple Access: Wideband Code Division Multiple Access) method, An antenna for sending and receiving data corresponding to GSM (Global System for Mobile Communications: Global System for Mobile Communications; registered trademark) method, etc.). In addition, the communication control unit 11 transmits and receives data to and from the wireless base station via the communication antenna 11 a through the communication channel set in the communication method according to the communication protocol corresponding to the predetermined communication method. In other words, the communication control unit 11 transmits and receives voice during a call with an external user of the external device and transmits and receives data of an e-mail to the external device of the communication partner based on the instruction signal output and input from the central control unit 1 .

In addition, the configuration of the communication control unit 11 is an example and is not limited thereto, and can be changed arbitrarily as appropriate. For example, although not shown in the figure, it can also be configured to be equipped with a wireless LAN module to allow access to communication via an access point (Access Point). Network N.

The communication network N is, for example, a communication network that connects the mobile terminal 100 to external devices via a wireless base station, a gateway server (not shown), or the like.

In addition, the communication network N is, for example, a communication network constructed using dedicated lines and existing general public lines, and various line types such as LAN (Local Area Network) and WAN (Wide Area Network) can be applied. In addition, the communication network N includes, for example, various communication networks such as a telephone line network, an ISDN line network, a dedicated line, a mobile communication network, a communication satellite line, and a CATV line network, an IP network, and VoIP (Voice over Internet Protocol: Internet Telephony). Gateways, Internet Service Providers, etc.

The operation input unit 12 is used to input various instructions to the terminal body.

Specifically, the operation input unit 12 includes a shutter button for instructing photography of a subject, up, down, left, and right cursor buttons and a decision button for selection instructions such as modes and functions, and execution instructions such as telephone calls and e-mail transmission and reception. Related communication related keys, various keys such as number keys and mark keys related to text input instructions (both are omitted from the illustration).

Also, when various keys are operated by the user, the operation input unit 12 outputs an operation instruction corresponding to the operated key to the central control unit 1 . The central control unit 1 causes each unit to perform a predetermined operation (for example, imaging a subject, receiving a telephone call, sending and receiving an e-mail, etc.) in accordance with an operation instruction output and input from the operation input unit 12 .

In addition, the operation input unit 12 may have a touch screen integrally provided with the display unit 8 , or may output an operation instruction corresponding to the predetermined operation to the central control unit 1 according to a user's predetermined operation on the touch screen.

<Code read processing>

Next, code reading processing of the mobile terminal 100 will be described with reference to FIGS. 2 to 7 .

FIG. 2 is a flowchart showing an example of the operation of code reading processing.

In addition, it is assumed that the imprint Si imaged by the following code reading process is stamped, for example, at a predetermined position on a recording medium P such as a postcard (see FIG. 3A ). In addition, it is assumed that the same code information Sc is added to each side Sa of the frame portion Sw of the imprint Si.

As shown in FIG. 2, first, if an imaging instruction is input according to a predetermined operation of the user on the operation input unit 12, the imaging control unit 4 causes the imaging unit 3 to image the imprint Si, and the image data generation unit 5 generates the image data transmitted from the electronic imaging unit 3b. The image data of the image Ia is captured (step S1; refer to FIG. 3B).

Then, the image data generation unit 5 outputs the generated YUV data of the captured image Ia to the memory 2 and stores it in the memory 2 .

In addition, in order to easily image the imprint Si in a more square state, the display control unit 9 may cause the display unit 8 to display a guide display corresponding to the outer shape of the imprint Si.

Next, the image acquisition unit 6a of the code processing unit 6 acquires image data (for example, luminance data) of a predetermined resolution of the captured image Ia generated by the image data generation unit 5 from the memory 2 (step S2; see FIG. 4A ). Next, the binarization unit 6b performs a binarization process of binarizing the image data of the captured image Ia acquired by the image acquisition unit 6a with a predetermined threshold to generate a first binarized image Ib (see FIG. 4B ). Image data (step S3).

Next, the contour determination unit c1 of the straight line estimation unit 6c performs convex hull processing on the image data of the first binarized image Ib generated by the binarization unit 6b, thereby specifying the polygonal shape corresponding to the outer contour of the frame portion Sw. Convex hull area A1 (see FIG. 4C) (step S4). Specifically, the contour determination unit c1 determines the convex hull region A having the largest area among the plurality of convex hull regions A, ... formed by the convex hull processing, as a polygonal shape corresponding to the outer contour of the frame portion Sw of the imprint Si. (for example, a hexagonal shape) convex hull area A1.

Next, the straight line specifying unit c2 divides the plurality of straight lines L, . A straight line L in which pixels overlap more than a predetermined value is determined as a straight line L constituting the outer contour of the frame image Wa corresponding to the frame portion Sw (step S5). Specifically, the straight line specifying unit c2 selects any two vertices B, B from among the six vertices B, ... constituting the convex hull area A1, and specifies a straight line L connecting these two vertices B, B as a constituting frame. The candidate straight line L of the outer contour of the square of the image Wa. Then, the straight line determination unit c2 determines, among the determined candidate straight lines L, a candidate straight line L whose number of pixels overlaps with a plurality of pixels constituting the convex hull area A1 is larger than a predetermined value, as a straight line constituting the outer contour of the frame image Wa. L.

Next, among the plurality of straight lines L, ... that pass through any two vertices B, B constituting the polygonal convex hull area A1, the straight line determination unit c2 determines the constituent frame in consideration of the angle formed with the adjacent straight line L The straight line L of the outer contour of the image Wa (step S6). Specifically, for all the candidate straight lines L, the straight line determination unit c2 determines the candidate straight line L whose angle with the adjacent candidate straight line L is approximately equal to the inner angle (90°) of the square frame of the stamp S, and for the polygonal shape The overlapping pixels of the convex hull area A1 are weighted, and the evaluation value of each candidate line L is calculated according to a predetermined calculation formula. Then, the line specifying unit c2 compares the calculated evaluation values of the candidate straight lines L, and specifies the candidate straight line L with the highest evaluation value as the straight line L constituting the outer contour of the frame image Wa.

In addition, the order of the process of specifying the straight line L in step S5 and the process of specifying the straight line L in step S6 is an example, and is not limited thereto, and may be reversed, for example.

Then, the vertex determination unit d1 of the frame detection unit 6d determines a predetermined number of points of intersection between the predetermined number of straight lines L constituting the outer contour determined by the straight line determination unit c2 as the frame portion Sw of the imprint Si (frame image Wa ) of the vertex C (step S7). At this time, the vertex specifying unit d1 may specify the vertex C of the frame image Wa in consideration of the coordinate position of the marker image Ma in the captured image Ia (see FIGS. 7A to 7C ).

Next, the frame detection unit 6d determines whether or not the four vertices C of the frame image Wa have been identified based on the identification result of the vertex identification unit d1 (step S8). Here, if it is determined that the four vertices C of the frame image Wa have been determined (step S8; Yes), the mapping transformation unit d2 performs a process on the captured image Ia based on the coordinate positions of the determined four vertices C, . . . Mapping transformation processing (step S9). Specifically, the mapping conversion unit d2 calculates a coordinate transformation formula such that the coordinate positions of the four vertices C, ... of the frame image Wa having a skewed rectangular shape become the coordinate positions of the four vertices C, ... of a square, according to The calculated coordinate transformation formula performs mapping transformation processing on the captured image Ia of the footprint Si to generate a mapped image Ic in which the outer shape of the frame image Wa of the footprint Si is converted into a square.

In addition, if it is determined in step S8 that the four vertices C of the frame image Wa have not been identified (step S8; NO), the CPU of the central control unit 1 skips the subsequent processing, and ends the code reading processing.

Next, the frame detection unit 6d detects a square frame portion Sw (frame image Wa) corresponding to the frame of the stamp S in the map-converted image Ic generated by the map conversion unit d2 (step S10). Next, the binarization unit 6b performs a binarization process of binarizing the image data of the map-transformed image Ic generated by the map conversion unit d2 by performing binarization at a predetermined threshold, thereby generating image data of the second binarized image Id. (step S11).

Next, the information reading unit 6e performs a reading process of reading from the frame portion Sw (frame image Wa) of the second binarized image Id corresponding to the mapped transformed image Ic generated by the map converting unit d2. The code information Sc reads predetermined information (step S12). Specifically, the information reading unit 6e determines a square reading area D in the second binarized image Id, and scans along a predetermined direction from a predetermined position (such as the upper left corner) of the reading area D, Coordinate positions at which a set of white pixels of pixel value "1" and a set of black pixels of pixel value "0" exist are respectively determined. Then, the information reading unit 6e decodes the specified arrangement of the set of white pixels and the set of black pixels, and reads the original predetermined information (for example, URL, etc.) indicated by the code information Sc.

Next, the information reading unit 6e determines whether or not the original schedule information has been read a plurality of times from the code information Sc through the reading process (step S13).

Here, if it is determined that the original scheduled information has been read multiple times (step S13; YES), the information reading unit 6e determines that the scheduled information can be read appropriately, and the operation processing unit 7 follows the steps passed by the information reading unit. 6e executes a predetermined action (such as accessing the Internet, replaying a specified sound or image) on the read predetermined information (for example, URL, etc.) (step S14).

On the other hand, if it is determined in step S13 that the original predetermined information has not been read multiple times (step S13; NO), the CPU of the central control unit 1 skips the process of step S14 and ends the code reading process.

As described above, according to the mobile terminal 100 of the present embodiment, the captured image Ia is obtained by capturing the imprint Si obtained by adding the code information Sc to the polygonal (for example, square) frame portion Sw around the predetermined print image Sp. Inside, constitute the outer contour of the frame portion Sw (frame image Wa) corresponding to the polygonal frame, estimate a predetermined number of straight lines L corresponding to the number of corners of the polygonal frame, and detect the estimated predetermined number of straight lines L The frame image Wa of the captured image Ia constituted by L, therefore, the frame image Wa of the captured image Ia can be appropriately detected by using a predetermined number of straight lines L constituting the outer contour of the frame image Wa. That is, for example, in the case where the vertex C of the frame image Wa cannot be properly detected due to blurring or blanking of the imprint Si, a predetermined number of straight lines L constituting the outer contour of the frame image Wa can be estimated, and the straight lines L can be used. L appropriately detects the frame portion Sw (frame image Wa) from the captured image Ia.

In addition, a polygonal region (convex hull region A1) corresponding to the outer contour of the frame portion Sw (frame image Wa) of the captured image Ia is specified, and a plurality of vertices B constituting the specified polygonal region, ... The positions determine the predetermined number of straight lines L constituting the outer contour of the frame image Wa, so that the frame image Wa can be appropriately detected from the captured image Ia using the specified predetermined number of straight lines L. Specifically, among a plurality of straight lines L, ... that pass through any two vertices B and B constituting the polygonal area, the number of pixels that each straight line L overlaps with the polygonal area and the number of adjacent straight lines At least one of the relative relationships between L can specify a predetermined number of straight lines L constituting the outer contour of the frame image Wa.

That is, among the plurality of straight lines L passing through any two vertices B, B constituting the polygonal area, ..., determine the straight line L in which the number of overlapping pixels constituting the polygonal area is greater than a predetermined value, This straight line L is determined as the straight line L constituting the outer contour of the frame image Wa, so the degree of overlap between the candidate straight line L constituting the straight line L of the outer contour of the frame image Wa and a plurality of pixels constituting the polygonal area can be considered. , the straight line L constituting the outer contour of the frame image Wa is appropriately determined.

In addition, among the plurality of straight lines L, ... that pass through any two vertices B, B constituting the polygonal area, a straight line whose angle with the adjacent straight line L is substantially equal to the inner angle of the polygonal frame is determined. L, weighting is added to the pixels overlapping the polygonal area among the pixels constituting the straight line L, the evaluation value of each straight line L is calculated, and the straight line L with the higher calculated evaluation value is determined as the outer contour constituting the frame image Wa Therefore, the straight line L constituting the outer contour of the frame image Wa can be appropriately determined in consideration of the relative relationship between the candidate straight lines L constituting the straight line L of the outer contour of the frame image Wa.

In addition, a predetermined number of points intersecting a predetermined number of straight lines L constituting the outer contour of the frame portion Sw (frame image Wa) is determined as vertices C of the frame image Wa, and based on the determined predetermined number of vertices C , the frame image Wa of the captured image Ia is detected, and therefore the frame image of the captured image Ia can be appropriately detected using a predetermined number of points (vertices C) that intersect with a predetermined number of straight lines L constituting the outer contour of the frame image Wa. Wa. At this time, the vertex C of the frame image Wa is specified based on the coordinate position of the mark image Ma corresponding to the mark Sm of the predetermined shape at the corner of the polygonal frame in the captured image Ia. In the captured image Ia obtained by imaging the footprint Si, the vertex C of the frame image Wa can also be specified appropriately, and the frame image Wa of the captured image Ia can be appropriately detected.

Furthermore, the captured image Ia is subjected to mapping conversion processing based on the determined predetermined number of vertices C to generate a polygonal captured image (mapped image Ic), and a frame corresponding to the generated polygonal captured image Ia is detected. Therefore, even if it is the captured image Ia obtained by imaging the imprint Si with bleed-out and white spots, it is possible to use a predetermined number of intersections between a predetermined number of straight lines L constituting the outer contour of the frame image Wa. The number of points (vertex C) is properly mapped and converted, and as a result, the polygonal frame image Wa can be properly detected.

In addition, predetermined information can be appropriately read from the code information Sc in the frame portion Sw (frame image Wa) of the captured image Ia. At this time, when more than a predetermined number of predetermined information is read from the captured image Ia of the imprint Si to which the same plurality of code information Sc is added to the frame portion Sw, execution of processing corresponding to the predetermined information is controlled. Therefore, by embedding a plurality of code information Sc, . .

In addition, the present invention is not limited to the above-described embodiments, and various improvements and design changes can be made without departing from the scope of the present invention.

Hereinafter, a second embodiment of the mobile terminal 100 will be described.

<Second Embodiment>

FIG. 8 is a block diagram showing a schematic configuration of a mobile terminal 200 according to the second embodiment.

As shown in FIG. 8, the code processing unit 206 of the mobile terminal 200 according to the second embodiment includes an image acquisition unit 6a, a binarization unit 6b, a straight line estimation unit 6c, a frame detection unit 6d, and an information reading unit 6e. Equipped with pixel number reduction unit 6f.

In addition, the configuration of the mobile terminal 200 of the second embodiment is substantially the same as that of the mobile terminal 100 of the above-mentioned embodiment except for the detailed description below, and the detailed description is omitted.

The number-of-pixels reduction unit 6f performs a number-of-pixels reduction process of reducing the number of pixels existing in the background of the captured image Ia.

That is, the number-of-pixels reduction unit 6f performs a process of reducing the number of pixels existing in the background relative to the first binarized image Ib corresponding to the captured image Ia acquired by the image acquisition unit 6a. Specifically, when the imprint Si is not stamped on a recording medium of the same color and design, but is stamped on a grid line N of a notebook, for example (see FIG. 10A ), the straight line estimation unit 6c may not be able to properly A straight line L constituting the outer contour of the frame portion Sw (frame image Wa) is estimated.

Therefore, the number-of-pixels reduction unit 6f acquires the image data of the first binarized image Ib generated by the binarization unit 6b, performs a process of inverting white pixels and black pixels (see FIG. 10B ), and performs Expansion processing and contraction processing of a set of pixels smaller than a predetermined value in the background of the first binarized image Ib (refer to FIG. 10C etc.). For example, the number-of-pixels reduction unit 6f performs dilation processing by increasing pixels by one turn for each pixel to be processed in the black-and-white reversed image Ie (see FIG. 10B ) of the first binarized image Ib (see FIG. 10B ). 10C), performing shrinkage processing (refer to FIG. 11A ) for stripping pixels in two circles, and then performing expansion processing (see FIG. 11B ) for adding pixels in one circle. As a result, the number of pixels existing in the background of the first binarized image Ib is relatively reduced relative to the number of pixels constituting the frame image Wa, and the straight line L constituting the outer contour of the frame image Wa is estimated by the straight line estimation unit 6c. When , the influence of pixels existing in the background of imprinted Si is sought to be reduced.

Then, the straight line estimation unit 6c estimates four lines constituting the outer contour of the frame portion Sw (frame image Wa) corresponding to the square frame of the stamp S in the first binarized image Ib processed by the number of pixels reduction unit 6f. After the straight line L, ..., the frame detection unit 6d detects the frame image Wa of the captured image Ia (see FIG. 11C ).

<Code read processing>

Next, code reading processing of the mobile terminal 200 according to the second embodiment will be described with reference to FIG. 9 .

FIG. 9 is a flowchart showing an example of the operation of code reading processing.

In addition, the code reading process of the mobile terminal 200 of the second embodiment is substantially the same as the code reading process of the mobile terminal 100 of the above-mentioned embodiment except for the detailed description below, and the detailed description is omitted.

As shown in FIG. 9 , the code processing unit 206 performs the respective processes of steps S1 to S3 in the same manner as the code reading process of the mobile terminal 100 in the above-mentioned embodiment, and generates image data of the first binarized image Ib (see FIG. 4B ). .

The number-of-pixels reduction unit 6f performs pixel reduction processing on the generated image data of the first binarized image Ib (step S21). Specifically, the number-of-pixels reduction unit 6f performs the processing of inverting white pixels and black pixels on the image data of the first binarized image 1b, and then performs expansion processing and contraction processing so that the pixels existing in the first binarized image The number of pixels in the background of Ib is relatively reduced with respect to the number of pixels constituting the frame portion Sw (frame image Wa).

Next, the contour specifying unit c1 performs convex hull processing on the image data of the first binarized image Ib after the pixel number reduction process, thereby specifying a polygonal convex hull region A1 corresponding to the outer contour of the frame portion Sw (see FIG. 4C) (step S4).

Then, the code processing unit 206 performs the processes after step S4 similarly to the code reading process of the mobile terminal 100 in the above-mentioned embodiment, and detects the straight line L constituting the outer contour of the frame portion Sw (frame image Wa) (step S5 ). , S6), detection of frame portion Sw (frame image Wa) (step S10), reading of predetermined information from code information (step S12), and the like.

Therefore, according to the mobile terminal 200 of the second embodiment, a process of relatively reducing the number of pixels existing in the background of the captured image Ia is performed, and in the processed captured image Ia, the frame portion Sw (frame) is estimated. The predetermined number of straight lines L of the outer contour of the image Wa), therefore, in the case where the imprint Si is not stamped on a recording medium with the same color and pattern, but is for example stamped on a notebook with a line grid N, by making The number of pixels existing in the background of the captured image Ia is relatively reduced, and the influence of the pixels existing in the background of the captured image Ia can also be reduced to appropriately estimate the straight line L constituting the outer contour of the frame image Wa.

In addition, in the above-described embodiment, the straight line specifying unit c2 uses each of the straight lines L and the polygonal convex hull area A1 among the plurality of straight lines L passing through any two vertices B and B constituting the polygonal convex hull area A1 A predetermined number of straight lines L constituting the outer contour of the frame image Wa are determined based on both the number of overlapping pixels in the packet area A1 and the relative relationship between adjacent straight lines L, but either one may be used as a reference. .

Furthermore, in the above-mentioned embodiment, the shape of the frame part Sw was made into a square, but this is only an example, and it is not limited to this, For example, it may be a polygonal shape other than a square.

In the above embodiment, the same code information Sc is added to each side Sa of the frame portion Sw of the imprint Si, but this is an example, not limited thereto, and different code information Sc may be added, for example. In this case, the amount of code information (original predetermined information) Sc embedded into the frame portion Sw can be increased.

Furthermore, in the above-mentioned embodiment, the mobile terminals 100 and 200 were exemplified as image processing devices, but this is not limited to this as an example, and as long as the execution of the detection process of the frame portion Sw (frame image Wa) can be controlled, it can be appropriately processed. change arbitrarily.

In addition, in the above-mentioned embodiment, under the control of the central control unit 1 of the mobile terminal 100 (200), the image acquisition unit 6a, the straight line estimation unit 6c, and the frame detection unit 6d are driven to achieve The functions of the estimating unit, the estimating unit, and the detecting unit are not limited thereto, and may be realized by the CPU of the central control unit 1 executing a predetermined program or the like.

That is, a program including an acquisition process, an estimation process, and a detection process is stored in a program memory storing the program. Then, the CPU of the central control unit 1 can be made to function as a means for acquiring the captured image Ia obtained by imaging the imprint Si of the seal S forming the periphery of a predetermined seal through the acquisition process. The polygonal frame is such that code information Sc that encodes predetermined information as a set of pixels in a regular arrangement is added to the print Si. In addition, the CPU of the central control unit 1 may function as a means for estimating the outer contour of the frame part Sw corresponding to the polygonal frame in the acquired captured image Ia through the estimation process, and estimating the contour corresponding to the polygonal shape. A predetermined number of straight lines L corresponding to the number of corners of the box. In addition, the CPU of the central control unit 1 may function as means for detecting the frame portion Sw of the captured image Ia composed of the estimated predetermined number of straight lines L through the detection process.

Similarly, the contour specifying unit, straight line specifying unit, vertex specifying unit, generating unit, reading unit, processing unit, and pixel count reducing unit may also be realized by the CPU of the central control unit 1 executing a predetermined program or the like.

Furthermore, as a computer-readable medium storing a program for executing each of the above-mentioned processes, in addition to a ROM, a hard disk, etc., a nonvolatile memory such as a flash memory, and a portable recording medium such as a CD-ROM can also be applied. . In addition, a carrier wave (Carrier wave) is also used as a medium for providing program data via a predetermined communication line.

Next, specific aspects of a third embodiment of the present invention will be described using the drawings. However, the scope of the invention is not limited to the illustrated examples.

FIG. 12 is a block diagram showing a schematic configuration of a mobile terminal 300 to which an embodiment of the present invention is applied.

As shown in FIG. 12 , the code processing unit 206 of the mobile terminal 300 according to the third embodiment includes an edge detection unit 6g and a parallel edge extraction unit in addition to the image acquisition unit 6a, the binarization unit 6b, and the information reading unit 6e. 6h, the reading area determination unit 6i.

In addition, the configuration of the mobile terminal 300 according to the third embodiment is substantially the same as that of the mobile terminal 100 according to the above-mentioned embodiment except for the detailed description below, and the detailed description will be omitted.

The edge detection unit 6g detects the edge E of the print image.

That is, the edge detection unit 6g detects a plurality of edges E from the binarized image Ib corresponding to the captured image Ia acquired by the image acquisition unit 6a. Specifically, for example, the edge detection unit 6g performs differential calculation on the image data of the binarized image 1b generated by the binarization unit 6b using a predetermined differential filter (for example, a Laplacian filter, etc.), and detects the brightness value, A place where the color and density change sharply is referred to as an edge E. Then, the edge detection unit 6g generates image data of an edge image Ic (see FIG. 15C ) based on the detected edge E.

In addition, the content of the edge detection process mentioned above is an example, it is not limited to this, and it can change arbitrarily suitably.

The parallel edge extraction part 6h extracts two parallel edges E, E.

That is, the parallel edge extraction unit 6h extracts two substantially parallel edges E, E at intervals substantially equal to the width of the frame portion Sw from among the plurality of edges E, . . . detected by the edge detection unit 6g. Specifically, the parallel edge extracting unit 6h sequentially applies the parallel edge filter F to the edge image Ic from a predetermined position (for example, the upper left corner, etc.) to a predetermined direction (for example, downward, etc.), and extracts the edge image Ic having a frame portion Sw. Two substantially parallel edges E, E (see FIGS. 16A and 16B ) at intervals of substantially equal width.

In the parallel edge filter F, two edge detection regions Fa, Fa having a predetermined width (for example, 5 pixels) are added at predetermined intervals with a predetermined length (for example, 20 pixels). By adjusting the interval between these two edge detection areas Fa, Fa to be approximately equal to the width of the frame portion Sw, the parallel edge extracting unit 6h extracts an edge having an approximately equal interval to the width of the frame portion Sw from the edge image Ic. The two substantially parallel edges E, E. At this time, the parallel edge extraction unit 6h rotates the two edge detection areas Fa, Fa around the center Fc by a predetermined angle (for example, 90°), thereby extracting two parallel edges E, E corresponding to the frame portion Sw.

In addition, in FIG. 16A and FIG. 16B, only a part of the upper left corner of the edge image Ic is enlarged and shown.

In addition, the content of the processing using the above-mentioned parallel edge filter F is an example, is not limited thereto, and can be changed arbitrarily as appropriate. For example, it is also possible to prepare a plurality of filters in which two edge detection areas Fa and Fa widths and rotation angles have been changed as parallel edge filters F, and use all of these parallel edge filters F to extract the corresponding frame portion Sw. Two parallel edges E, E.

The reading area specifying unit 6i specifies the reading area A of the code information Sc.

That is, the reading area specifying unit 6i specifies the reading area A of the code information Sc in the captured image Ia based on the two substantially parallel edges E, E extracted by the parallel edge extracting unit 6h. Specifically, the reading area specifying unit 6i assigns, within the binarized image Ib corresponding to the captured image Ia, the area corresponding to the line connecting the intermediate points of the two substantially parallel edges E, E (the two The area corresponding to the area inside the edge E, E) is determined as the reading area A.

For example, the reading area specifying unit 6i connects the intermediate points of two substantially parallel edges E, E extracted by the parallel edge extracting unit 6h to determine a line of a predetermined shape (for example, a square, etc.), and makes the determined line Corresponding to the binarized image Ib, the reading area A of the code information Sc is determined within the binarized image Ib (see FIG. 16C ).

The information reading unit 6e performs a reading process of reading original predetermined information from the code information Sc.

That is, the information reading section 6e reads predetermined information from the code information Sc in the reading area A specified by the reading area specifying section 6i. Specifically, the information reading unit 6e reads predetermined information from the pixel values (code information Sc) of pixels existing in the reading area A in the binarized image Ib corresponding to the captured image Ia. For example, the information reading unit 6e scans along a predetermined direction from a predetermined position (for example, the upper left corner) of the reading area A, and respectively determines a set of white pixels with a pixel value of "1" and a set of black pixels with a pixel value of "0". The coordinate position where the collection of . The information reading unit 6e performs decoding processing corresponding to the coding method of the code information Sc for the determined arrangement of the set of white pixels and the set of black pixels, and reads the original predetermined information represented by the code information Sc (for example, URL wait).

At this time, the information reading unit 6e performs reading processing for each area corresponding to each side Sa of the frame portion Sw in the reading area A. As shown in FIG. That is, the original predetermined information is read from the code information Sc at the number of times (for example, 4 times) corresponding to the number of sides Sa of the frame portion Sw. Here, the same code information Sc is attached to each side Sa of the frame portion Sw. Therefore, the information reading unit 6e may be configured to read out a plurality of original predetermined information after detecting two or more original predetermined information, for example. In this case, it is determined that the predetermined information can be properly read.

In addition, since the above-mentioned reading process is a well-known technique, detailed description is abbreviate|omitted here.

The operation processing unit 7 executes a predetermined operation according to the original predetermined information of the code information Sc read by the information reading unit 6e.

That is, the operation processing unit 7 controls the execution of the processing corresponding to the predetermined information when the information reading unit 6 e has read a predetermined number or more of predetermined information. Specifically, for example, when a URL is read as predetermined information, the operation processing unit 7 controls the communication control unit 11 to access a specific web page on the Internet specified by the acquired URL. Then, the operation processing unit 7 controls the display control unit 9 and the audio communication unit 10 to execute various processes according to execution instructions of various processes set in advance (for example, playback of specific sounds and images, etc.).

The display unit 8 is composed of, for example, a liquid crystal display panel or the like, and displays an image captured by the imaging unit 3 (for example, a live image, etc.) on a display screen based on a video signal from the display control unit 9 .

The display control unit 9 performs control to read out display image data temporarily stored in the memory 2 and display it on the display unit 8 .

Specifically, the display control unit 9 includes a VRAM (Video Random Access Memory), a VRAM controller, a digital video encoder, and the like. In addition, under the control of the central control unit 1, the digital video encoder reads from the VRAM at a predetermined playback frame rate (for example, 30 fps) via the VRAM controller. The luminance signal Y and the color difference signals Cb and Cr are based on these data to generate a video signal and output it to the display unit 8 .

For example, the display control unit 9 displays live on the display unit 8 while sequentially updating a plurality of frame images captured by the imaging unit 3 and the imaging control unit 4 and generated by the image data generation unit 5 at a predetermined display frame rate. superior.

The audio communication unit 10 communicates with an external user of an external device connected via the communication network N.

Specifically, the audio communication unit 10 includes a microphone 10a, a speaker 10b, a data conversion unit 10c, and the like. In addition, the voice communication unit 10 performs A/D conversion processing on the user's speech voice input from the microphone 10a through the data conversion unit 10c, outputs the speech voice data to the central control unit 1, and under the control of the central control unit 1 The data conversion unit 10c performs D/A conversion processing on audio data such as answering audio data output and input from the communication control unit 11, and outputs it from the speaker 10b.

The communication control unit 11 transmits and receives data via the communication network N and the communication antenna 11a.

That is, the communication antenna 11a is capable of performing a predetermined communication method (for example, W-CDMA (Wideband Code Division Multiple Access: Wideband Code Division Multiple Access) method, An antenna for sending and receiving data corresponding to GSM (Global System for Mobile Communications: Global System for Mobile Communications; registered trademark) method, etc.). In addition, the communication control unit 11 transmits and receives data to and from the wireless base station via the communication antenna 11 a through the communication channel set in the communication method according to the communication protocol corresponding to the predetermined communication method. That is, the communication control unit 11 transmits and receives audio during a call with an external user of the external device and transmits and receives data of an e-mail to an external device of a communication partner based on an instruction signal output and input from the central control unit 1 .

In addition, the configuration of the communication control unit 11 is an example and is not limited thereto, and can be changed arbitrarily as appropriate. For example, although not shown in the figure, it can also be configured to be equipped with a wireless LAN module to allow access to communication via an access point (Access Point). Network N.

The communication network N is, for example, a communication network in which the mobile terminal 300 is connected to external devices via a wireless base station, a gateway server (not shown), or the like.

In addition, the communication network N is, for example, a communication network constructed using dedicated lines and existing general public lines, and various line types such as LAN (Local Area Network) and WAN (Wide Area Network) can be applied. In addition, the communication network N includes, for example, various communication networks such as a telephone line network, an ISDN line network, a dedicated line, a mobile communication network, a communication satellite line, and a CATV line network, an IP network, and VoIP (Voice over Internet Protocol: Internet Telephony). Gateways, Internet Service Providers, etc.

The operation input unit 12 is used to input various instructions to the terminal body.

Specifically, the operation input unit 12 includes a shutter button for instructing photography of a subject, up, down, left, and right cursor buttons for instructing selection of modes, functions, etc., a decision button, and execution functions such as telephone calls and e-mail transmission and reception. Various keys such as communication-related keys related to indications, number keys and mark keys related to text input indications (both are not shown).

Also, when various keys are operated by the user, the operation input unit 12 outputs an operation instruction corresponding to the operated key to the central control unit 1 . The central control unit 1 causes each unit to perform a predetermined operation (for example, imaging a subject, calling a telephone, sending and receiving an e-mail, etc.) in accordance with an operation instruction output and input from the operation input unit 12 .

In addition, the operation input unit 12 may have a touch screen integrally provided with the display unit 8 , or may output an operation instruction corresponding to the predetermined operation to the central control unit 1 according to a user's predetermined operation on the touch screen.

<Code read processing>

Next, code reading processing of the mobile terminal 300 will be described with reference to FIGS. 13 to 16 .

FIG. 13 is a flowchart showing an example of the operation of code reading processing.

In addition, it is assumed that the imprint Si imaged by the following code reading process is stamped, for example, at a predetermined position on a recording medium P such as a postcard (see FIG. 14A ). In addition, it is assumed that the same code information Sc is added to each side Sa of the frame portion Sw of the imprint Si.

As shown in FIG. 13, first, if an imaging instruction is input according to a predetermined operation of the user on the operation input unit 12, the imaging control unit 4 causes the imaging unit 3 to image the imprint Si, and the image data generating unit 5 generates the image data transmitted from the electronic imaging unit 3b. Image data of the captured image Ia (step S1; refer to FIG. 14B ).

Then, the image data generation unit 5 outputs the generated YUV data of the captured image Ia to the memory 2 and stores it in the memory 2 .

In addition, in order to easily image the imprint Si in a more square state, the display control unit 9 may cause the display unit 8 to display a guide display corresponding to the outer shape of the imprint Si.

Next, the image acquisition unit 6a of the code processing unit 6 acquires image data (for example, luminance data) of a predetermined resolution of the captured image Ia generated by the image data generation unit 5 from the memory 2 (step S2; see FIG. 15A ). Next, the binarization unit 6b performs a binarization process of binarizing the image data of the captured image Ia acquired by the image acquisition unit 6a with a predetermined threshold to generate image data of the binarized image Ib (step S3).

Next, the edge detection unit 6g detects a plurality of edges E from the binarized image Ib generated by the binarization unit 6b, and generates image data of an edge image Ic (see FIG. 15C) (step S4).

Next, the parallel edge extraction unit 6h sequentially applies the parallel edge filter F to the image data of the edge image Ic from a predetermined position (for example, upper left corner, etc.), and extracts two substantially parallel edges E, E (step S5; See Figures 16A and 16B).

Next, the read area specifying unit 6i specifies, in the binarized image Ib, an area corresponding to a line connecting the intermediate points of two substantially parallel edges E, E extracted by the parallel edge extracting unit 6h as code information. The area A of Sc is read (step S6).

Next, the information reading unit 6e performs a reading process of reading predetermined information from the code information Sc in the reading area A specified by the reading area specifying unit 6i of the binarized image Ib (step S7). Specifically, the information reading unit 6e scans in a predetermined direction from a predetermined position (for example, the upper left corner) of the reading area A which is a line of a predetermined shape, and specifies a set of white pixels with a pixel value "1" respectively. and the coordinate position where a set of black pixels with a pixel value "0" exists. Then, the information reading unit 6e performs decoding processing on the specified arrangement of the set of white pixels and the set of black pixels, and reads original predetermined information (for example, URL, etc.) indicated by the code information Sc.

In addition, in order to more efficiently perform the reading process by the information reading unit 6e, before the reading process, the binarized image Ib may be subjected to a mapping conversion process such that the outer shape of the frame portion Sw becomes a square.

Next, the information reading unit 6e judges whether or not original predetermined information has been read from the code information Sc a plurality of times through the reading process (step S8).

Here, if it is determined that the original scheduled information has been read multiple times (step S8; Yes), the information reading unit 6e determines that the scheduled information can be read appropriately, and the operation processing unit 7 follows the steps passed by the information reading unit. 6e executes a predetermined action (such as accessing the Internet, replaying a certain sound or image) with the read predetermined information (for example, URL, etc.) (step S9).

On the other hand, if it is determined in step S8 that the original predetermined information has not been read multiple times (step S8; NO), the CPU of the central control unit 1 skips the process of step S9 and ends the code reading process.

As described above, according to the mobile terminal 300 of the present embodiment, the captured image Ia is binarized from the footprint Si obtained by adding the code information Sc to the frame portion Sw having a predetermined width around the predetermined print image Sp. A plurality of edges E are detected in the image Ib, and are determined in the binarized image Ib based on two substantially parallel edges E, E extracted from the plurality of edges E, . The reading area A of the code information Sc in which predetermined information is read from the code information Sc, therefore, by utilizing the edge shape of the frame portion Sw of the imprint Si stamped on the recording medium P, it is possible to In the valued image Ib, the reading area A where the code information Sc exists is appropriately determined. That is, since the code information Sc exists inside the frame portion Sw, by specifying the two edges E, E of the frame portion Sc, the reading area A of the code information Sc in the binarized image Ib can also be appropriately specified. Specifically, in the binarized image Ib, the region corresponding to the region inside the two edges E, E, more specifically, corresponds to the line connecting the intermediate points of the two edges E, E The area of is determined as the reading area A, so the reading area A can be more appropriately specified in the captured image Ia.

In this way, it is only necessary to read the original predetermined information from the code information Sc in the reading area A, so that the predetermined information can be appropriately read from the captured image Ia.

In addition, when a predetermined number or more of predetermined information is read from the captured image Ia of imprint Si to which the same plurality of code information Sc is added to the frame portion Sw, the execution of processing corresponding to the predetermined information is controlled. Therefore, by embedding a plurality of code information Sc, . .

In addition, the present invention is not limited to the above-described embodiments, and various improvements and design changes can be made without departing from the scope of the present invention.

That is, as long as the frame portion Sw can be specified from the captured image Ia and information can be read from the specified frame portion, the method of specifying the frame portion Sw may be any method.

For example, in the above-described embodiment, a frame template image corresponding to a predetermined polygonal frame may be prepared, and the frame portion Sw may be specified by matching between the frame template image and the captured image Ia.

In addition, for example, in the above-mentioned embodiment, as the reading area A of the code information Sc, an example is shown that connects an intermediate point between two substantially parallel edges E, E in the captured image Ia (binarized image Ib). The area corresponding to the raised line is not limited thereto as an example, and can be changed arbitrarily as long as it is an area corresponding to the inner area of the two substantially parallel edges E, E.

In addition, in the above-mentioned embodiment, the shape of the frame part Sw was made into a square, but this is just an example, and it is not limited to this, As long as it has predetermined width, it can change arbitrarily suitably. That is, the shape of the frame portion Sw may be, for example, a polygonal shape other than a square, or may be a ring shape.

Furthermore, in the above-described embodiment, the same code information Sc is added to each side Sa of the frame portion Sw of the imprint Si, but it is not limited to this as an example, and for example, different code information Sc may be added. Next, the amount of code information (original predetermined information) Sc embedded in the frame portion Sw can be increased.

Furthermore, in the above-mentioned embodiment, the mobile terminal 300 was exemplified as the image processing device, but it is not limited to this as an example, as long as the execution of the code information Sc reading process can be controlled, it can be changed arbitrarily as appropriate.

In addition, in the above-described embodiment, under the control of the central control unit 1 of the mobile terminal 300, the image acquisition unit 6a, the edge detection unit 6g, the parallel edge extraction unit 6h, the reading area determination unit 6i, The information reading unit 6e is driven to realize functions as an acquisition unit, a detection unit, an extraction unit, a determination unit, and a reading unit. Wait for it to come true.

That is, a program including acquisition processing, detection processing, extraction processing, identification processing, and reading processing is stored in a program memory storing the programs. Then, the CPU of the central control unit 1 can be made to function as a means for acquiring a print image obtained by imaging a print Si having a predetermined width to the periphery of a predetermined print image Sp through an acquisition process. The frame portion Sw of the frame is formed by adding code information Sc that encodes predetermined information as a set of pixels regularly arranged. In addition, the CPU of the central control unit 1 may function as a means for detecting a plurality of edges E from the obtained print image through the detection processing procedure. In addition, the CPU of the central control unit 1 may function as a means for extracting, among the detected plurality of edges E, . The 2 edges E, E of . In addition, the CPU of the central control unit 1 may function as means for specifying the reading area A of the code information Sc in the stamp image based on the two extracted edges E, E by specifying the processing procedure. In addition, the CPU of the central control unit 1 may function as a means for reading predetermined information from the code information Sc in the determined reading area A through the reading process,

Similarly, the processing unit may also be realized by a configuration in which the CPU of the central control unit 1 executes a predetermined program or the like.

Furthermore, as a computer-readable medium storing a program for executing each of the above-mentioned processes, in addition to a ROM, a hard disk, etc., a nonvolatile memory such as a flash memory, and a portable recording medium such as a CD-ROM can also be applied. . In addition, a carrier wave (Carrier wave) is also used as a medium for providing program data via a predetermined communication line.

Claims (19)

1. an image processing apparatus, is characterized in that, possesses:

Acquisition unit, it obtains the print image trace that addition of code information in frame part being carried out to shooting gained;

Determining unit, the frame image-region corresponding with the trace of above-mentioned frame part in the print image obtained by above-mentioned acquisition unit is defined as the reading area reading above-mentioned code information by it;

Reading unit, it reads above-mentioned code information from the above-mentioned reading area determined by above-mentioned determining unit.

2. image processing apparatus according to claim 1, is characterized in that,

Above-mentioned determining unit possesses:

Detecting unit, it detects multiple edge from the print image obtained by above-mentioned acquisition unit;

Extract unit out, they are in the multiple edges detected by above-mentioned detecting unit, extract out and 2 almost parallel edges at interval that the width of above-mentioned frame part is roughly equal, wherein

According to 2 edges extracted out by above-mentioned extraction unit, in above-mentioned print image, determine the reading area of above-mentioned code information,

Above-mentioned reading unit, in the above-mentioned reading area determined by above-mentioned determining unit, reads above-mentioned predetermined information from above-mentioned code information.

3. image processing apparatus according to claim 2, is characterized in that,

The region corresponding with the region of the inner side clamped with above-mentioned 2 edges, in above-mentioned print image, is defined as above-mentioned reading area by above-mentioned determining unit.

4. image processing apparatus according to claim 2, is characterized in that,

Above-mentioned code information is added along the vertical direction substantially vertical with this Width in the Width substantial middle side of above-mentioned frame part,

The region corresponding with the line between the intermediate point connecting above-mentioned 2 edges, in above-mentioned marking image, is defined as above-mentioned reading area by above-mentioned determining unit.

5. image processing apparatus according to claim 1, is characterized in that,

Identical multiple above-mentioned code information is added to above-mentioned frame part,

Also possess: processing unit, it, when be have read the above-mentioned predetermined information of more than predetermined number by above-mentioned reading unit, controls the execution of the process corresponding with this information.

6. image processing apparatus according to claim 1, is characterized in that,

Above-mentioned determining unit possesses:

Presumption unit, it is in the print image obtained by above-mentioned acquisition unit, forms the lateral profile of above-mentioned frame part, estimates the straight line of the predetermined number corresponding with the angle number of the frame of above-mentioned polygon;

Detecting unit, it detects the above-mentioned frame part of the above-mentioned print image be made up of the straight line of the predetermined number deduced by above-mentioned presumption unit.

7. image processing apparatus according to claim 6, is characterized in that,

Above-mentioned presumption unit possesses:

Outline specifying unit, it determines the region of the polygon corresponding with the lateral profile of the above-mentioned frame part of the print image obtained by above-mentioned acquisition unit;

Straight line determining unit, the position on multiple summits in the region of its polygon determined by above-mentioned outline specifying unit according to formation, determines the straight line of the predetermined number of the lateral profile forming above-mentioned frame part.

8. image processing apparatus according to claim 6, is characterized in that,

Above-mentioned straight line determining unit is in the multiple straight lines on any 2 summits that have passed the region forming the polygon determined by above-mentioned outline specifying unit, according at least one party in the relativeness between the number of the pixel of the region overlap of each straight line and above-mentioned polygon and adjacent straight line, determine the straight line of the predetermined number of the lateral profile forming above-mentioned frame part.

9. image processing apparatus according to claim 8, is characterized in that,

Above-mentioned straight line determining unit is also in the multiple straight lines on any 2 summits that have passed the region forming the polygon determined by above-mentioned outline specifying unit, determine this straight line to be defined as the straight line of the lateral profile forming above-mentioned frame part by the straight line that the number of multiple pixel overlaps in the region forming above-mentioned polygon is more than predetermined value.

10. image processing apparatus according to claim 8, is characterized in that,

Above-mentioned straight line determining unit is also in the multiple straight lines on any 2 summits that have passed the region forming the polygon determined by above-mentioned outline specifying unit, determine the straight line roughly equal with the interior angle of the frame of adjacent straight line angulation and above-mentioned polygon, the pixel additional weight overlapping with the region of above-mentioned polygon in the pixel forming this straight line, calculate the evaluation of estimate of each straight line, straight line high for the evaluation of estimate calculated is defined as the straight line of the lateral profile forming above-mentioned frame part.

11. image processing apparatus according to claim 7, is characterized in that,

Above-mentioned detecting unit possesses: summit determining unit, and the point of the predetermined number intersected between the straight line of the predetermined number determined by above-mentioned straight line determining unit is defined as the summit of above-mentioned frame part by it, wherein

According to the summit of the predetermined number determined by above-mentioned summit determining unit, detect the above-mentioned frame part of above-mentioned print image.

12. image processing apparatus according to claim 11, is characterized in that,

Above-mentioned trace forms the mark of reservation shape in the bight of the frame of above-mentioned polygon,

Above-mentioned summit determining unit also determines the position of the sign image corresponding with above-mentioned mark in the print image obtained by above-mentioned acquisition unit, according to the position of determined above-mentioned sign image, determines the summit of above-mentioned frame part.

13. image processing apparatus according to claim 12, is characterized in that,

Also possess: the first generation unit, it, according to the summit of the predetermined number determined by above-mentioned summit determining unit, carries out mapping transformation process to the print image obtained by above-mentioned acquisition unit, generates the print image of polygon,

Above-mentioned detecting unit detects the frame part corresponding with the frame of the print image of the polygon generated by above-mentioned first generation unit.

14. image processing apparatus according to claim 6, is characterized in that,

Also possess: reading unit, it reads predetermined information from the above-mentioned code information in the above-mentioned frame part of the above-mentioned print image detected by above-mentioned detecting unit.

15. image processing apparatus according to claim 1, is characterized in that,

Additional multiple above-mentioned code information on the frame of the polygon of above-mentioned trace,

Possess: processing unit, it, when be have read the above-mentioned predetermined information of more than predetermined number by above-mentioned reading unit, controls the execution of the process corresponding with this predetermined information.

16. image processing apparatus according to claim 6, is characterized in that,

Also possess: pixel count reduces unit, and it is to the print image obtained by above-mentioned acquisition unit, carries out processing the number of the pixel be present in the background of this print image is reduced relatively,

In the print image of above-mentioned presumption unit after above-mentioned pixel count reduces the process of unit, estimate the straight line of above-mentioned predetermined number.

17. image processing apparatus according to claim 1, is characterized in that,

Above-mentioned frame is roughly circular,

Possess: mark determining unit, it determines to be formed in the mark in above-mentioned ring part in the image obtained by above-mentioned acquisition unit,

Above-mentioned reading unit, according to the mark determined by above-mentioned mark determining unit, reads above-mentioned code information.

18. image processing apparatus according to claim 17, is characterized in that,

Also possess: the second generation unit, the position of mark determined by above-mentioned mark determining unit as benchmark, is carried out mapping transformation process to the image obtained by above-mentioned acquisition unit, is generated the image above-mentioned ring part being transformed to positive toroidal by it.

19. 1 kinds of image processing methods employing image processing apparatus, is characterized in that, comprising:

Obtain to the trace that addition of code information in frame part carry out make a video recording gained print image obtain step;

The frame image-region corresponding with the trace of above-mentioned frame part in the print image this obtained is defined as the determining step of the reading area reading above-mentioned code information;

The read step of above-mentioned code information is read from this above-mentioned reading area determined.