JP6287529B2 - Keyword determining apparatus, keyword determining program, and keyword determining method - Google Patents

Description

  The present invention relates to a keyword determination device, a keyword determination program, and a keyword determination method.

  When managing images, it is easier to manage by associating keywords with images, and the accessibility to the images is also improved. If the keyword is linked manually by the user, it takes time and effort. Therefore, if the keyword can be automatically extracted from the image and linked, the convenience for the user is improved.

  Conventionally, a method of extracting a character string from image information obtained by photographing a station name plate and extracting a station name based on the size and position of the character string is known (see, for example, Patent Document 1).

JP 2009-130697 A

  However, as in Patent Document 1, even if an image keyword is specified based on the position and size of the character string, the position and size of the character string that can be a keyword vary depending on the type of signboard included in the photographed image. For this reason, even if the said patent document 1 is used, the keyword of an image may not be correctly specified.

  In one aspect, an object of the present invention is to provide a keyword determination device, a keyword determination program, and a keyword determination method capable of determining an appropriate keyword as an image keyword.

In one aspect, the keyword determination device includes a storage unit that associates and stores a word that can be a keyword and a character string of one or more character systems that indicate the word, and when the image includes a plurality of character strings, Referring to the storage unit, a specifying unit for specifying a word indicated by each character string included in the image, and a word having a large number of types or number of character strings included in the image indicating the word specified by the specifying unit. A determination unit that determines a keyword indicating the image from among the specified words in preference to a word having a small number of types or number of character strings .

In one aspect, when a plurality of character strings are included in an image, the keyword determination program refers to a storage unit that stores a word that can be a keyword and a character string of one or more character systems that indicate the word in association with each other. Then, a word indicated by each character string included in the image is specified, and a word having a large number or number of character strings included in the image indicating the specified word is a word having a small number or number of character strings. This is a program that causes a computer to execute a process of determining a keyword indicating the image from among the identified words with priority over the word.

In one aspect, when a plurality of character strings are included in an image, the keyword determination method refers to a storage unit that associates and stores a word that can be a keyword and a character string of one or more character systems that indicate the word. Then, a word indicated by each character string included in the image is specified, and a word having a large number or number of character strings included in the image indicating the specified word is a word having a small number or number of character strings. This is a keyword determination method in which a computer executes a process of determining a keyword indicating the image from among the specified words with priority over the word.

  An appropriate keyword can be determined as an image keyword.

It is a block diagram which shows roughly the hardware constitutions of the electronic device which concerns on one Embodiment. It is a functional block diagram of CPU. It is a figure which shows the data structure of word DB. It is a flowchart which shows the keyword determination process by CPU. FIG. 5A to FIG. 5C are diagrams for explaining the processing of FIG. It is a figure which shows another example of a signboard area | region.

  Hereinafter, an embodiment will be described in detail with reference to FIGS. FIG. 1 schematically shows a hardware configuration of an electronic device 100 according to the present embodiment. The electronic device 100 is assumed to be, for example, a terminal such as a smartphone or a mobile phone, or a photographing device such as a compact camera or a single-lens reflex camera.

  As shown in FIG. 1, the electronic device 100 includes a CPU (Central Processing Unit) 90, a ROM (Read Only Memory) 92, a RAM (Random Access Memory) 94, a storage unit (here, an HDD (Hard Disk Drive)) 96, A network interface 97, a display unit 93, an input unit 95, a photographing unit 80, a portable storage medium drive 99, and the like are provided. Each component of the electronic device 100 is connected to a bus 98. The display unit 93 includes a liquid crystal display or the like, and the input unit 95 includes a touch panel, an input button, or the like. The imaging unit 80 performs imaging based on an instruction from the user via the input unit 95. In the electronic device 100, when managing an image captured by the capturing unit 80 with the HDD 96, the CPU 90 determines a keyword of the image and manages the image in a state in which the keyword is associated. That is, the CPU 90 functions as a keyword determination device. Specifically, in the electronic device 100, a program (including a keyword determination program) stored in the ROM 92 or the HDD 96 or a program read from the portable storage medium 91 by the portable storage medium drive 99 (a keyword determination program is stored). 2) is realized by the CPU 90 executing the function shown in FIG. Note that the keyword means a word that is assumed to be taken by the photographer who has taken the image (though the photographer wanted to photograph) among the words included in the image.

  FIG. 2 schematically shows a functional block diagram of the CPU 90. As shown in FIG. 2, the CPU 90 includes an image acquisition unit 12, a signboard region estimation unit 14, a character string extraction unit 16, a character recognition unit 18, a word detection unit 20 as a specifying unit, and a keyword determination unit 22 as a determination unit. , Function as. 2 also shows a character recognition dictionary DB 30 stored in the HDD 96 or the like and a word DB 32 as a storage unit.

  The image acquisition unit 12 acquires the image captured by the imaging unit 80 and transmits it to the signboard area estimation unit 14.

  The signboard area estimation unit 14 estimates and extracts a signboard area from the image received from the image acquisition unit 12. As a signboard area estimation method, a known method (for example, Zhou et al., “Extracting Signboard Area and Information in Signboard from Scene Image”, PRMU, No. 102, 2003, etc.) can be employed.

  The character string extraction unit 16 separates the signboard area cut out by the signboard area estimation unit 14 into characters and the others and extracts only the character part as an image. The image extracted by the character string extraction unit 16 is referred to as a character string image.

  The character recognition unit 18 performs character recognition on the character string image extracted by the character string extraction unit 16 with reference to the character recognition dictionary DB 30 to obtain a character recognition result. The character recognition dictionary DB 30 stores dictionary data used in normal character recognition processing.

  The word detection unit 20 detects a word existing in the signboard area based on the character recognition result by the character recognition unit 18 and the word DB 32. FIG. 3 shows the data structure of the word DB 32. As shown in FIG. 3, the word DB 32 has fields of “word”, “character string serial number”, and “character string”. In the “word” field, words that can be keywords of the image, such as place names and station names, are stored. The “character string serial number” field stores a serial number assigned to the registered character string. The “character string” field stores a character string of one or more character systems corresponding to a word that can be a keyword of an image. The character system means kanji notation, hiragana notation, katakana notation, romaji notation, English notation, and the like. For example, a common word indicated (meaning) by a character string, such as the character strings “Fujitsu Castle”, “Fujitsu Ujo”, and “Fujitsu Castle”, even if the character types used are different. That is, it can be said that the “word” is a character string representing one or more character strings having a common meaning.

  The keyword determination part 22 determines the word as a keyword, when the number of the detected words is single. On the other hand, when the number of detected words is plural, the keyword determination unit 22 determines a keyword based on the character string existing in the signboard area. Details of the keyword determination method will be described later. Note that the CPU 90 manages the image in the HDD 96 by associating the keyword determined by the keyword determination unit 22 with the image.

  Next, the keyword determination process by the CPU 90 will be described in detail along the flowchart of FIG.

  In the process of FIG. 4, in step S <b> 10, the image acquisition unit 12 waits until an image is input from the imaging unit 80. In this case, when the imaging unit 80 captures an image and the image is transmitted from the imaging unit 80, the image acquisition unit 12 proceeds to step S12.

  In step S12, the image acquisition unit 12 acquires the image transmitted from the imaging unit 80. The image acquisition unit 12 transmits the acquired image to the signboard area estimation unit 14.

  Next, in step S14, the signboard area estimation unit 14 estimates a signboard area. For example, when the image acquisition unit 12 acquires an image obtained by photographing a signboard as shown in FIG. 5A, the signboard area estimation unit 14 estimates a range indicated by a broken line in FIG. 5B as a signboard area. In addition, the signboard area estimation part 14 estimates a signboard area by a well-known method as mentioned above. In step S14, when the signboard area estimation unit 14 cannot estimate the signboard area, that is, when the signboard area does not exist in the image, the entire process of FIG.

  Next, in step S16, the character string extraction unit 16 extracts a character string image from the signboard area. Then, the character recognition unit 18 refers to the character recognition dictionary DB 30 and performs character recognition of the character string image. In the case of the signboard region of FIG. 5B, the character recognition result by the character recognition unit 18 is as shown in FIG. In FIG. 5C, a character string registered in the word DB 32 in FIG. 3 (referred to as a registered character string) is underlined.

  Next, in step S18, the character recognition unit 18 determines whether there are a plurality of registered character strings. In the case of FIG. 5C, since there are three registered character strings, the determination in step S18 is affirmed.

  When the determination in step S18 is affirmed, the process proceeds to step S20, and the word detection unit 20 refers to the word DB 32 and identifies a word corresponding to the character string (registered character string) of the character recognition result. In the case of FIG. 5C, the word detection unit 20 identifies the word “Fujitsu Castle” corresponding to the character strings “Fujitsu Castle” and “Fujitsujo” and the word “Sagami Lake” corresponding to the character string “Lake Sagami”. To do.

  Next, in step S22, the keyword determination unit 22 determines a keyword from the identified words. Here, the keyword determination part 22 determines the keyword of an image with the following methods.

First, the keyword determining unit 22 obtains the detection number N _i in the word image specified in step S20. In the example of FIG. 5C, the keyword determination unit 22 obtains the number of detections N ₁ (= 2) of the _first word “Fujitsu Castle” out of the words “Fujitsu Castle” and “Sagamiko” and 2 The number of detections N ₂ (= 1) of the first word “Lake Sagami” is obtained.

Then, the keyword determining section 22 calculates from the keyword likelihood degree f, defined (N _i) the following equation (1) by N _i.

Here, f (N _i ) is a value obtained by normalizing the number of detections Ni to a range from 0 to 1. In the example of FIG. 5C, f (N ₁ ) of the _first word “Fujitsu Castle” is
f (N ₁ ) = 2 / (2 + 1) = 2 / 3≈0.67 (2)
And f (N ₂ ) of the _second word “Sagamiko” is
f (N ₂ ) = 1 / (2 + 1) = 1 / 3≈0.33 (3)
It becomes.

Next, the keyword determination unit 22 determines a word having the largest f (N _i ) as a keyword. In the above example, “Fujitsu Castle” is determined as a keyword associated with the image of FIG.

When a plurality of image keywords can be set, the keyword determination unit 22 may determine a word having f (N _i ) larger than a predetermined threshold as a keyword.

When the word having the largest f (N _i ) is determined as a keyword, the keyword determination unit 22 may use _Ni itself as the keyword-like degree f (N _i ) (f (N _i )). = N _i ).

In addition, when the word having the largest f (N _i ) is determined as a keyword, the keyword may not be determined because the values of f (N _i ) of two or more words are equal. In such a case, the keyword may be determined by further considering information on the position and size of the character string as in a known technique (the above-mentioned Patent Document 1).

In the above formula (1), for example, “Fujitsu Castle” may be detected twice and “Fujitsujo” may be detected once as a character string corresponding to the word “Fujitsu Castle”. In such a case, to the number of times of detection N _i may be counted three times the total number of detections, number of types of registered character string is detected, i.e., Chinese characters "Fujitsu castle" and hiragana "Fuji It may be counted as 2 times.

  Thus, the process of FIG. 4 ends. On the other hand, if the determination in step S18 of FIG. 4 is negative, that is, if only one registered character string is recognized (detected), the keyword determination unit 22 proceeds to step S24 and stores it in the word DB 32. The keyword is determined based on this. That is, the keyword determination unit 22 determines a word uniquely determined by one recognized registered character string as a keyword. Thereafter, all the processes in FIG. 4 are terminated. If no registered character string is recognized, the keyword cannot be determined, and therefore the entire process of FIG. 4 is terminated without determining the keyword in step S24. Note that the CPU 90 manages the images for which keywords have been determined, with the keywords associated with each other in the HDD 96.

FIG. 6 shows another example of the signboard area included in the image. The same processing as described above may be performed in a signboard area (station signboard) as shown in FIG. In the example of FIG. 6, the registered character string of the word “Fujitsu” is detected as “Fujitsu” in kanji, “Fujitsuu” in hiragana, and “Fujitsu” in romaji. As a registered character string of the word “left neighbor”, “kanji left” and “Hidaritonari” are detected as the registered character strings of the word “left neighbor”. Therefore, the degree of keyword likelihood f (N _i ) is the largest for the word “Fujitsu”. Therefore, when determining one word as a keyword, the keyword determination unit 22 determines the word “Fujitsu” as a keyword. .

  As described above in detail, according to the present embodiment, the word DB 32 stores a word that can be a keyword and a character string of one or more character systems indicating the word in association with each other. 20 identifies the word indicated by each character string included in the image with reference to the word DB 32 when the image includes a plurality of character strings. Then, the keyword determination unit 22 determines a keyword indicating an image from among the specified words based on the number of types and the number of character strings indicating the specified words. Thereby, in this embodiment, even if a user has a plurality of character strings included in an image obtained by photographing a signboard such as a specific explanation board or a guide board, words written in various character systems Can be automatically determined as a keyword of the image (a representative character string that the image will mean). Further, even when the keyword cannot be appropriately determined based on the position and size of the word, it is possible to determine an appropriate keyword as the keyword of the image by performing the processing of the present embodiment.

In the above-described embodiment, the case where the keyword determination unit 22 determines one or a plurality of keywords and associates them with an image has been described. However, the present invention is not limited to this. For example, the keyword determination unit 22 prioritizes each word based on f (N _i ) of the word specified in step S20, rearranges each word in the order of arrangement according to the priority order, and associates it with the image. You may make it attach. That may be f (N _i) the word is large to give the cord the image to priority over the word f (N _i) is small.

  In the embodiment described above, the CPU 90 of the electronic device 100 has each function of FIG. 2 and the processing of FIG. 4 is performed in the electronic device 100. However, the present invention is not limited to this. For example, a PC (Personal Computer) or server may have at least a part of the functions in FIG. 2, and at least a part of the processing in FIG. 4 may be executed by the PC or server. For example, the PC or server may have all the functions shown in FIG. 2, and when uploading an image from the electronic device 100 to the PC or server, the keyword of the image may be determined in the PC or server. Further, for example, the functions of the word detection unit 20 and the keyword determination unit 22 may be provided in a PC or a server. In this case, when the image is uploaded from the electronic device 100 to the PC or server, the character recognition result is notified from the electronic device 100 to the PC or server, and the PC or server performs steps S18 to S24 in FIG. Processing may be executed.

  The configuration of FIG. 1 is an example. That is, the configuration may be added, deleted, or changed according to the type of electronic device 100 or a necessary function.

  The above processing functions can be realized by a computer. In that case, a program describing the processing contents of the functions that the processing apparatus should have is provided. By executing the program on a computer, the above processing functions are realized on the computer. The program describing the processing contents can be recorded on a computer-readable recording medium (except for a carrier wave).

  When the program is distributed, for example, it is sold in the form of a portable recording medium such as a DVD (Digital Versatile Disc) or a CD-ROM (Compact Disc Read Only Memory) on which the program is recorded. It is also possible to store the program in a storage device of a server computer and transfer the program from the server computer to another computer via a network.

  The computer that executes the program stores, for example, the program recorded on the portable recording medium or the program transferred from the server computer in its own storage device. Then, the computer reads the program from its own storage device and executes processing according to the program. The computer can also read the program directly from the portable recording medium and execute processing according to the program. Further, each time the program is transferred from the server computer, the computer can sequentially execute processing according to the received program.

  The above-described embodiment is an example of a preferred embodiment of the present invention. However, the present invention is not limited to this, and various modifications can be made without departing from the scope of the present invention.

In addition, the following additional remarks are disclosed regarding description of the above embodiment.
(Additional remark 1) The memory | storage part which memorize | stores in association with the word which can become a keyword, and the character string of the 1 or several character system which shows this word,
When the image includes a plurality of character strings, referring to the storage unit, a specifying unit that specifies a word indicated by each character string included in the image;
A determining unit that determines a keyword indicating the image from among the specified words based on the number of types of character strings included in the image indicating the word specified by the specifying unit;
A keyword determination device comprising:
(Supplementary note 2) The keyword according to supplementary note 1, wherein the determination unit prioritizes a word having a large number of types of character strings over a word having a small number of types of character strings. Decision device.
(Additional remark 3) The said determination part determines the keyword which shows the said image based on the number of the character strings which show the word which the said specific part specified, The keyword determination apparatus of Additional remark 1 or 2 characterized by the above-mentioned.
(Supplementary Note 4) When a plurality of character strings are included in an image, the word is included in the image with reference to a storage unit that stores a word that can be a keyword and a character string of one or a plurality of character systems that indicate the word. Identify the word indicated by each string
Determining a keyword indicating the image from the identified word based on the number of types of character strings included in the image indicating the identified word;
A keyword determination program that causes a computer to execute processing.
(Additional remark 5) The said determination WHEREIN: The word with many types of the said character string is given priority over the word with few types of the said character string, The said keyword is determined, It is characterized by the above-mentioned. Keyword decision program.
(Additional remark 6) The keyword determination program of Additional remark 4 or 5 which determines the keyword which shows the said image based on the number of the character strings which show the specified word in the said process to determine.
(Supplementary Note 7) When a plurality of character strings are included in an image, the word is included in the image with reference to a storage unit that stores a word that can be a keyword and a character string of one or more character systems indicating the word in association with each other. Identify the word indicated by each string
Determining a keyword indicating the image from the identified word based on the number of types of character strings included in the image indicating the identified word;
A keyword determination method in which processing is executed by a computer.
(Additional remark 8) The said determination WHEREIN: The said keyword is determined in preference to the word with many types of the said character string over the word with few said character string types. Keyword decision method.
(Additional remark 9) The keyword determination method of Additional remark 7 or 8 characterized by determining the keyword which shows the said image based on the number of the character strings which show the specified word in the said process to determine.

20 Word detection part (specific part)
22 Keyword decision part (decision part)
32 Word DB (storage unit)
90 CPU (keyword determination device)

Claims (5)

A storage unit that stores a word that can be a keyword and a character string of one or more character systems that indicate the word;
When the image includes a plurality of character strings, referring to the storage unit, a specifying unit that specifies a word indicated by each character string included in the image;
The image indicating the word specified by the specifying unit is prioritized from the word having a large number or number of character strings included in the image over the word having a small number or number of character strings. A determination unit for determining a keyword indicating
A keyword determination device comprising: The keyword determination apparatus according to claim 1, wherein the determination unit determines a word having a number of types or the number of character strings larger than a predetermined threshold as a keyword indicating the image . The determination unit may further be based on the position and / or size of each character string included in the image, the keyword determining apparatus according to claim 1 or 2, characterized in that to determine the keyword indicating the image . When an image includes a plurality of character strings, each character string included in the image is referenced with reference to a storage unit that stores a word that can be a keyword and a character string of one or more character systems that indicate the word Identify the word
The image indicating the identified word is given priority over words having a large number or number of character strings included in the image indicating the identified word over words having a small number or number of character strings. Determine keywords,
A keyword determination program that causes a computer to execute processing. When an image includes a plurality of character strings, each character string included in the image is referenced with reference to a storage unit that stores a word that can be a keyword and a character string of one or more character systems that indicate the word Identify the word
The image indicating the identified word is given priority over words having a large number or number of character strings included in the image indicating the identified word over words having a small number or number of character strings. Determine keywords,
A keyword determination method in which processing is executed by a computer.

Images