JP2008004116A - Method and device for retrieving character in video - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method and a device for retrieving characters in videos for detecting character strings such as captions and flips, which cannot be easily detected by a conventional technology, and allowing retrieval of the detected character strings in a common framework independently of languages/fonts. <P>SOLUTION: By inputting videos and detecting a character area from a frame image of the inputted videos based on the feature of a form of the character area, the image features of the character area are extracted. A retrieval target character string inputted by a character input means is rendered as an image, and the image features are extracted from the character string image. The matching degree is found by comparing the obtained character area image features and those of the character image, and the character area including the character string giving the matching degree is outputted. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a technique for searching for a broadcast video or a video in a database, and more particularly to a video search method capable of performing a search using characters in a video as a clue.

  In video, text information such as subtitles (characters inserted in the screen, also called telop) and flips (characters copied as a subject) is one of important information that symbolizes the contents of the scene in which they appear. One. For this reason, researches such as detection of subtitles appearing and keyword search for character string text used in subtitles have been energetically advanced.

  Here, in detecting the subtitle area, (1) the line figure part that forms the character has high luminance, and (2) the luminance difference between the character part and the boundary is high, thereby forming an edge. In addition, a technique is generally premised on the fact that there are a large number of these edges in the vicinity of the character, and (3) that the character region is stationary for a certain period of time. For keyword search, the character area detected above is converted into character codes such as ASCII (American Standard Code for Information Interchange) and JIS (Japanese Industrial Standards) by character recognition widely used in document processing. However, there is a method of performing collation based on the code.

  Separately from this, a technique for performing a search focusing on the shape of a character is disclosed. For example, in Non-Patent Document 1, one character is cut out from a printed document read by a scanner or the like, and a code based on a shape feature is determined and assigned to each of the characters. The same code is given to each character in the same manner, so that the codes are collated.

Proceedings of Recherche d'Information Assistee par Ordinateur, Cambridge, MA (USA), pp. 248-258 (Y. Tanaka and H. Torii, “Transmedia machine and its keyword search over image texts”)

  The technique of Non-Patent Document 1 can be said to simplify the character recognition in that a code is associated with the shape of each character. In this case, it is necessary to determine in advance what kind of code is to be given for each character, and in order to apply the same search method to other languages, it is necessary to change the method of assigning feature amounts according to each language.

  In addition, in order to assign a feature amount to each character, it is assumed that one character is cut out correctly, but this is a condition that is difficult to satisfy except for a document or the like in which characters are printed on a plain background with high contrast. For example, subtitles are often superimposed on a complex background, and there are very many factors that reduce the accuracy of single character extraction, such as characters being connected by a color blur peculiar to video.

  In addition to such a printed document, there are usually not a few character parts that cannot be detected by the conventional caption detection method in a video. For example, characters on a flip where a paper board drawn with ink is photographed with a camera is not necessarily particularly high in brightness, and therefore, it cannot be said that the brightness difference from the boundary is high. Furthermore, along with the recent enhancement in performance of caption insertion devices, the degree of freedom of the color and shape of characters has increased, and for example, the use of low-luminance characters such as black has been increased. In addition, there are not a few types of subtitles that flow and disappear, and the subtitles are not necessarily stationary.

In addition, in keyword search based on character recognition, character recognition depends on the language and typeface. Therefore, it is necessary to prepare a recognition method suitable for each language and typeface in order to search for multilingual or different typeface characters. is there. In particular, there are many cases in which videos from various countries, such as news, travel and language programs, appear in the video, and you want to search using them as keywords. However, it is wasteful to introduce an optimum character recognition method for each language and type of font.
The purpose of the present invention is to enable detection of character strings such as subtitles and flips, which are difficult to detect with the prior art, and to search the detected character strings in a common framework without depending on language or typeface. Another object of the present invention is to provide a method and an apparatus for searching for characters in video.

  In order to achieve the above object, a character search method in a video according to the present invention includes a step of inputting a video, a step of detecting a character region from a frame image of the input video based on a feature of the shape, and the character region. Extracting a first image feature, a step of drawing a character string to be searched input by a character input means as an image, a step of extracting a second image feature from the character string image, The method includes a step of collating one image feature with the second image feature to obtain a matching degree, and a step of outputting the character region including the character string from which the matching degree is obtained.

  In this way, instead of character by character, a sequence of connected characters is captured as a character string image, and a character region is detected based on the shape of the character string image. This makes it possible to detect subtitles and flip character areas that do not have. Further, since detection is performed for each frame image, it is possible to detect a character area such as a caption that flows without being stationary. Furthermore, since the detected character area is collated using a common framework of image features of the character string image, it is possible to perform a highly accurate search and a search that does not depend on language or typeface.

  According to the present invention, since a search is performed by collation in units of character strings rather than in units of characters, any character string can be searched for characters such as subtitles and flips in video images without character recognition regardless of language. become. In addition, even when there are characters on a complex background, that is, even if there is some noise on the background, the search can be performed stably.

  Hereinafter, a method and apparatus for searching characters in a video according to the present invention will be described in more detail with reference to embodiments of the invention shown in several drawings.

FIG. 1 is a schematic block diagram showing an example of a character search device in a video according to the present invention.
In FIG. 1, 1 is a computer that executes search processing, 2 is a display device such as a CRT (Cathode Ray Tube) that displays an output screen of the computer 1, and 3 is an external command for the computer 1. An input device using a keyboard, a pointing device such as a mouse or a touch panel. A video signal 5 from the video device 4 is input to the computer 1 via a video input device 6.

  The video device 4 is a tuner device for receiving broadcast programs such as terrestrial broadcast, satellite broadcast, and cable television, or a device for reproducing video recorded on an optical disc, a video tape, or the like. The video input device 6 sequentially converts the video signal 5 from the video device 4 into digital image data and digital audio data. When the video output from the video device 4 is digitized from the beginning, conversion from analog to digital is not necessary, but it is also possible to perform conversion into a format that is easy for a computer to handle.

  Inside the computer 1, digital image data from the video input device 6 enters the memory 9 through the interface 8, and a frame image is processed by a CPU (Central Processing Unit) 7 that executes a search processing program stored in the memory 9. Processed every time. In addition, various information and data can be stored in the external information storage device 10 as necessary. The memory 9 stores a program for causing the computer 1 to execute a character search process in the video described below, and stores various data created by the process, and is referred to as necessary.

  The details of the software flow of the search processing according to the present invention, which is executed by the hardware, particularly the CPU 7 of the computer 1, will be described below.

  In the present invention, a character in a subtitle is searched not using a character encoded by recognition but using a character shape, that is, a unique shape of the character. Specifically, the operation is performed as follows.

  First, a user (user) of the present invention inputs a character string to be searched into the computer 1 using the input device 3 without any difference from a normal text search. In the case of alphabets, a single keystroke on the keyboard is used to input characters, and in the case of Japanese, kanji kana mixed text can be input by combining romaji conversion and kana-kanji conversion.

  When a keyword is input, the computer 1 reads and draws character fonts corresponding to the keyword one by one to create a character string image. The character font may be either bitmap format data or vector format data. The search is performed as a similar image search using this character string image as a key. That is, a subtitle including a similar image portion is output as a search result.

  In the current computer operation system, various languages can be handled in a unified framework, and it is extremely easy to create a character string image of an arbitrary language. The character search of the present invention is performed using this character string image, and is not collated in units of one character according to the above-described prior art, so that it is not necessary to cut out each character, and is affected by the cutting accuracy. hard. In addition, since the size of the image to be collated can be made relatively large, even in the case of characters on a complicated background, even if a part of the background is mixed as characters, it can be ignored statistically, and there are few oversights Stable verification is possible.

  The present invention can be broadly divided into two processes. One is preprocessing that analyzes video, detects character areas such as subtitles and flips, and extracts the features, and the other is to match the extracted features with the features of the keywords entered by the user. And a search process for finding a match.

  First, subtitle detection and feature extraction processing, which is preprocessing, will be described. It has already been described that there are not a few cases where the feature amount of captions cannot be detected only by a method using (1) high luminance, (2) edge concentration, and (3) stationary for a certain period. Therefore, in the present invention, more universal features are considered for characters.

  Characters have historically been handed down with brushes and pens, and have a structure that is a combination of lines, common to many languages. In other words, there is a tendency for lines to be dense in portions where characters are present. In addition, the width of the line is usually within a specific range determined by the size of the character. In the present invention, this feature is used as a condition for specifying the presence of a character. In addition, the line color is a single color or a pattern (gradation pattern) that changes gradually. This makes it possible to detect a type of caption that could not be detected in the past.

  A flowchart of caption detection and feature extraction focusing on the shape is shown in FIG. After initialization of various variables in step 200, one frame image is input from the video image (step 201). In the following step 202, filter processing for suppressing blurring of characters peculiar to the video is performed.

  The analog signal video is not only inherently blurry, but also has a deliberate process such as anti-aliasing to make the characters appear smoother on low-resolution television receivers. There are many. Bleeding is a state in which a pixel having an intermediate luminance between both pixels is divided between two pixels forming a boundary, and the difference in luminance between adjacent pixels is reduced to detect an edge. Becomes difficult. This is removed with a filter. A specific processing method will be described later.

  Next, in step 203, lines having a specific width are extracted, and in step 204, it is tested whether or not they are concentrated in a specific area, and the concentrated area is extracted. Then, it is re-inspected whether the extracted area is valid as a character area (step 205). If the extracted area is deemed valid, the image feature of the character portion, that is, the character string included in the area is extracted (step 206). .

  The subtitles that appear in the video continue to exist at least somewhere in the image for a period that can be seen by humans, so check whether they are the same as the characters extracted in the previous frame and select only those that appear newly (Step 207). The above processing returns to step 201 and is repeated until the search target video ends.

  Next, a search process for collating the character features extracted above with the keyword features input by the user and finding a match will be described with reference to FIG.

  First, initialization of variables and the like is performed in step 300, and in step 301, an arbitrary character string is acquired from the user using various input methods widely used in personal computers, word processors, and the like. This character string is drawn as a bitmap image using a character font prepared in advance (step 302). Character string features are extracted from the image thus obtained (step 303), and the degree of matching is obtained for all the character string features stored in step 207 (steps 304 and 305). A search result is obtained by ranking and displaying a list in descending order of the degree of coincidence (step 306).

  Hereinafter, a method for realizing each process of the flowchart will be described in more detail.

First, in the filter process in step 202 of FIG. 2, specifically, for example, a 3 × 3 filter shown in FIG. 4 is used. The figure shows that the central pixel P5 is determined by combining the luminance values of nine pixels adjacent to the pixel P5 from P1 to P9, and the maximum value and the minimum value in the nine pixels are obtained. Correct the brightness value of P5 to a value close to either. This is expressed as follows. That is,
Pmax = MAX {P1, P2, P3, P4, P5, P6, P7, P8, P9}
Pmin = MIN {P1, P2, P3, P4, P5, P6, P7, P8, P9}
If P5−Pmin <Pmax−P5 then P5 ← Pmin
else P5 ← Pmax
It becomes.

  When the image size is represented by the number of pixels by the width w and the height h, when the origin (0, 0) is the top left vertex of the image, the x coordinate is 1 to w-2, and the y coordinate is 1 to h. P5 is corrected for all coordinates in the range up to -2. Since the purpose is to remove the blur at the boundary portion, the correction processing may be performed by paying attention only to the boundary portion, that is, the edge portion for high-speed processing. If necessary, a filter having a size of 5 × 5 or 7 × 7 may be used instead of the 3 × 3 in the above example. By this filter, the intermediate luminance value generated at the boundary due to blurring is unified to the luminance value of any pixel forming the boundary, and the boundary becomes clear.

  Next, the specific line width portion extraction processing in step 203 and the specific line width portion concentrated region extraction processing in step 204 will be described in detail. An outline of the specific line width portion extraction process is shown in FIG. There are various methods for selectively extracting only lines with a specific width, but instead of extracting the lines strictly and finding the width, it is easier to line that satisfies the specific width. Extract the part that has the characteristics related to the existence of.

  Specifically, in each of the vertical and horizontal directions, an equiluminance pixel string that is continuously connected by the length of the predesignated range is extracted by the difference in luminance of the predesignated range, and is marked on each pixel. Add. In the figure, the circled pixels mean the pixels constituting the horizontal equal luminance pixel column, and the X symbols similarly indicate the vertical equal luminance pixel column. In both the vertical and horizontal directions, the lengths of 3 pixels and 4 pixels are within a specified range.

  Characters are present in areas with many pixels marked with “X”. For specifying the area, a projection distribution method generally used in the field of reading and recognition of printed documents can be used. However, since a character is often composed of intricately intertwined vertical lines and horizontal lines, equi-luminance pixel columns in the horizontal and vertical directions are concentrated at the same time in a certain area of the character. That is, the pixels marked with ○ and the pixels marked with x are gathered at the same time. By using this feature, improvement in specific performance can be expected.

  Therefore, in the frame image, a projection distribution in which the number of circles is counted and a projection distribution in which the number of circles is counted are created for each scan based on pixel intervals in the vertical and horizontal directions. A projection distribution composed of values multiplied by is created, and a character area is detected based on the projection distribution. If neither the circle mark nor the x mark exists, the value of the projection distribution becomes 0, and the character area specifying performance is enhanced. Note that in this multiplication, in the scanning in pixel units, even if it is a character area, ○ often does not appear at the same time, so it is better to multiply the sum of values for a plurality of scans with a margin. It may be good.

  It should be noted that the pixels constituting the equiluminance pixel column can be set so that the luminance itself is within a predesignated range in addition to the difference in luminance within a predesignated range.

  The flowchart of step 203 in the specific line width portion extraction process is shown in FIGS. In this process, a two-dimensional array Map (i, j) that matches the size of the frame image to be processed is prepared with an initial value of 0, and a specific-length equiluminance pixel row is detected in two horizontal and vertical directions. Each time, the value of the corresponding position on Map (i, j) is changed to a value other than 0.

  First, in step 400, Map (i, j) is initialized, and in step 401, the loop counter in the y direction is reset. In the subsequent step 402, the x-direction loop counter and the start point of the equiluminance pixel array are set to the initial values.

  In step 403, the absolute value of the difference in luminance between two pixels adjacent in the x direction is obtained, and if it is equal to or less than the threshold value TH1, the process proceeds to step 407 without doing anything. If it is larger than the threshold value TH1, the process proceeds to step 404. In Step 404, it is compared whether or not the difference between the start point of the isoluminance pixel sequence stored last and the current location is larger than the threshold value TH2 and smaller than the threshold value TH3. Assuming that there is a high possibility of being a part of a line having a predetermined width, 1 is substituted into the Map array at the position corresponding to this pixel column (step 405). Then, the starting point of the pixel row is updated to the current point (step 406), the x-direction loop counter is advanced by 1 (step 407), and the above is repeated by the width w of the frame image (step 408) (steps 403 to 408). ). Further, the loop counter in the y direction is advanced by 1 (step 409), and this is repeated by the height h of the frame image (step 410), and the above is repeated (steps 402 to 410).

  Similarly, as shown in FIG. 7, an equiluminance pixel column in the y direction can also be obtained (steps 411 to 420). The distribution of the equiluminance pixel row at this time is stored by adding 2 to the Map array obtained up to 410 (415). That is, when the value of the Map array is 0, it means that both x and y directions are not included in the equiluminance pixel column, 1 is only in the x direction, 2 is only in the y direction, and 3 is in both directions. It will be included in the luminance pixel column.

  Next, the extraction of the specific line width portion concentration area in step 204 is basically performed by finding an area where pixels having values other than 0 in the Map array are concentrated. As described above, the projection distribution is used to extract the character region from the Map array.

  As shown in FIG. 8, when detecting horizontally-written captions in the screen 600, if the projection distribution 602 is taken in the horizontal direction, the peak of the projection distribution tends to appear in the portion where the caption exists. As a result, the starting point and height in the y direction where the subtitle exists are obtained. Then, the projection distribution in the vertical direction is similarly taken for the region in the range of the y-axis obtained in this way, and the start point and the width in the x direction are determined. In the case of vertically written subtitles, the projection distribution in the vertical direction is first taken, the starting point and the width in the x direction are obtained first, and then the starting point and the height in the y direction are determined.

  Specifically, processing is performed according to the flowchart shown in FIG. First, in step 500, the projection distribution ProjY is initialized, and in the subsequent step 501, the loop counter in the y direction is reset. In step 502, initial values 0 are entered in the x-direction loop counter and the variables x_line and y_line for counting the number of equi-luminance pixel columns in the x and y directions, respectively.

  The whole is scanned by pixel unit scanning, and 1 is added to x_line if it is an equiluminance pixel row in the x direction, and 1 is added to y_line if it is an equiluminance pixel row in the y direction (steps 503 to 508).

  Then, the product of x_line and y_line is substituted into the projection distribution ProjY (step 509). The above is repeated for the height of the frame image (steps 510 and 511) (steps 502 to 511). In step 509, the projection distribution does not take a sufficiently large value if there is no equiluminance pixel column in both the x and y directions.

  In step 205, it is determined whether or not the extracted character area is finally valid as a character area. For example, if the height or width of the detected area is extremely small, there is a high possibility that it is not a character. For this reason, when the height or width is smaller than a preset threshold value, the character area is not considered and is discarded.

  In the case of a character region on a complex background with many edges, accurate separation between the character and the background is difficult unless the character is recognized and the shape is determined. Even in such a case, in order to remove the background as much as possible, the inside of the character area is searched starting from each pixel on the rectangular line that encloses the character area with a margin, and pixels of the same color and brightness Is removed as a background. Characters have a clear boundary with their surroundings, so they are not removed together as a background.

  If most of the character area has been removed as the background, it can be determined that the object or background pattern in the image has been misrecognized as a character, and that the character area lacked validity. Therefore, after the background removal process, if there is no character portion of a certain area or more, it is discarded. This validity determination process is performed for each character candidate area obtained by the combination of brightness, line width, vertical writing / horizontal writing in several types of threshold ranges, and the appropriate one is set as the character area.

  In the character string feature extraction in step 206, image features of the character region extracted as subtitles are extracted. Although the subtitle image itself can be stored as a feature as it is and classic image matching such as template matching can be performed, the amount of data to be stored becomes large, and there are many problems in terms of calculation efficiency. When collating characters, the size differs between the subtitles and the template. Therefore, the final matching result must be obtained after determining the value with the highest degree of matching by varying the size. The free-size matching is extremely computationally intensive and is not suitable for search applications that require a large amount of matching.

  Therefore, in the present embodiment, a one-dimensional feature sequence in which the number of vertical edges is counted at regular intervals and arranged horizontally is adopted for a binarized character image as shown in FIG. Of the two dimensions x and y, the y-direction information is represented by the number of edges whose value does not change due to expansion / contraction, thereby expressing the image feature of the character string with a one-dimensional feature string.

  In the case of a vertically written character image, it is possible to employ a one-dimensional feature sequence in which the number of horizontal edges is counted at regular intervals and arranged vertically.

  The character string feature in the video obtained as described above is calculated for the degree of coincidence with the character string feature (step 303) input by the user in step 304 shown in FIG. Verification is performed. When collating, only one-dimensional expansion / contraction collation is sufficient. As for the one-dimensional expansion / contraction verification method, for example, various speed-up methods represented by DP (Dynamic Programming) matching can be applied.

  Each value of this one-dimensional feature sequence is just the number of edges, and the amount of individual information is small, but the number of permutation combinations becomes enormous in proportion to the length of the character string. A meaningful word can be expected for a meaningful word, so high specific performance can be expected. Further, in order to correctly search even a character string consisting of only simple figure characters, for example, the edge is not the simple number of edges but the edges are vertical (|), horizontal (-), diagonally right (/), diagonally left (\). Of these, the type of line formed as a boundary may be added as one of the feature amounts.

  FIG. 11 shows an example of a user interface when using the character search method described above. In FIG. 11, 700 is an example of a search result output screen, and 705 is an example of a search keyword input screen. The user inputs an arbitrary character string into the keyword input text input area 706 using a keyboard or the like. After confirming that the character string has been correctly input, the search process is started by pressing the search button 709, and the keyword character string search result is displayed as 700. When canceling the search, a cancel button 710 is pressed.

  On the screen 700, as a search result, combinations of frame images 701 having subtitles including character strings that match the keyword character strings and the degree of coincidence 704 are displayed in a list in descending order of coincidence. This allows you to see at a glance the scene where the searched subtitle appears. In general, since there are many cases where it is desired to see many results at once, the displayed frame image needs to be displayed as a small thumbnail. However, this makes it difficult to read subtitles and makes it difficult to determine whether the search results are correct. Therefore, only the subtitle portion is enlarged and displayed in another frame 702. Further, it is also possible to highlight the keyword character string searched by enclosing it in a frame 703. Note that the frame 703 surrounds the character string determined by the feature extraction, so that its position with respect to the character string slightly changes depending on the background, font, and the like. Further, by displaying address information such as a time code together with the frame image, the position of the scene in the video may be known. In addition, in order to improve the density of information displayed in the list, if there are multiple search results on one screen that are higher in the list, the one with only one screen and the lower rankings will be advanced. It may be displayed.

  Further, according to the present invention, due to its principle, in the case of a typeface completely different from the font selected when the input keyword is converted into a character string image, it may not be correctly detected from the subtitle. Therefore, the font may be designated in the input field 707. Since the types of main fonts are limited, it is also possible to indicate choices by pressing a button 708 and select from the choices. In the case of a search based on character recognition, the recognition processing algorithm needs to be corrected. However, in the present invention, it is possible to flexibly cope with only the font switching operation.

  FIG. 12 shows an example when a search is performed without using the search keyword input screen 705. By pointing a subtitle displayed on the screen 800 with a pointing device or the like, all subtitles in which the same character string as that subtitle appears are searched. In this method, since the keyword is given as an image from the beginning, there is no need to convert it into a character string image, and no font is required. For example, if an unknown character appears in a subtitle, but it seems that it is a keyword for a program, even if that character cannot be entered, all scenes where the keyword appears can be found. it can. It also helps you to efficiently watch language programs in unfamiliar languages.

  Further, according to the present invention, if a specific keyword is designated in advance, it can be automatically notified at the moment when the keyword appears in the video. For example, if words such as “breaking news”, “typhoon”, “earthquake”, or “disconnection” are stored, emergency information can be detected.

  Similarly, in the case of an advertisement video, a fixed phrase indicating the address of related information such as “www” or “http” can be automatically detected, and only the screen is selectively recorded. And can be prepared for later reference. In particular, in the case of address information such as an Internet domain name or URL (Uniform Resource Locator), the number of companies and organizations that perform advertisements is limited. Therefore, a template pattern of address information of companies and organizations is prepared in advance. It is also possible to specify the company or organization by matching with it, and use the address information encoded in the character code to automatically connect with the Internet browser etc. and browse the homepage. . In this case, it seems to the user that the connection destination of the browser is automatically changed according to the television broadcast. Alternatively, the connection may be made only when the user clicks the address character area with a mouse or the like.

  Furthermore, the present invention can be used not only as a means for enabling scene search by text input without any manual indexing in the field of video databases, but also contributes to the efficiency of manual keyword assignment work. . When indexing is performed, a scene to which a specific keyword is to be added is found by the present invention and registered as an index. As a result, from the next time, it is possible to perform a high-speed search only by pure text search for the index data.

  In addition, the processing target of the present invention is not limited to television broadcast images, and it goes without saying that it can be applied to a wide range of natural images. For example, a character string written on a signboard or the like in the city can be the target of the above processing. It is possible to search from snapshots and video clips taken when visiting sightseeing spots by using the characters on the signboard as keys. In particular, if you shoot along with a nameplate written with a place name written on the historical site, and a description of the origin, you can select a desired photo or photo from a vast library of photos taken in the past using place names that are easy to remember. Find videos.

  In addition, the automatic network connection based on the detection of the address information described above can be performed on such a natural image. For example, the homepage can be easily called by photographing address information written on a signboard or the like and performing the above-described processing on the address information. Alternatively, in this case, it is desirable that shooting, processing of the present invention, and homepage display can be performed collectively in a small device that can be easily carried. One method is to make such a device, but it may be technically or costly difficult to mount high-speed computing performance in a small device so that image processing can be performed quickly. . Therefore, by using a mobile phone or a mobile terminal that can be connected to the network, it is possible to cause a server on the network to perform processing with a heavy calculation load.

  An example is shown in FIG. The mobile phone 900 is equipped with a camera 901 and can photograph any signboard or the like. FIG. 13 is an example of a block diagram of the system. The mobile phone 900 transmits image data captured by the camera 901, relays it at the transmission base station 920, and is connected to the communication network 921. The address information extraction process is performed, and the process result is returned to the mobile phone 900 along the reverse path. A data recording device 923 is connected to the server 922 and stores address information template data and the like. By maintaining the template data up to date, it is possible to automatically cope with new types of address information without any operation on the part of the user of the mobile phone.

The block diagram for demonstrating the example of embodiment of the character search apparatus in the image | video concerning this invention. The flowchart for demonstrating the pre-process in embodiment of the character search method in the image | video concerning this invention. The flowchart for demonstrating the search process in embodiment of the character search method in the image | video concerning this invention. The figure for demonstrating a blur removal filter. The image figure for demonstrating the outline | summary of the character area extraction method. The flowchart for demonstrating specific line width part feature extraction processing. The flowchart following FIG. 6 for demonstrating specific line width part feature extraction processing. The image figure which shows the outline | summary which calculates | requires the area | region where a character exists. The flowchart for demonstrating extraction of a specific line | wire width part concentration area | region. The image figure which shows the outline | summary of the characteristic extraction of a character string. The figure for demonstrating an example of the operation interface at the time of the search by this invention. The figure for demonstrating an example of the operation interface using a touch panel. The figure for demonstrating an example of the utilization method of this invention using a mobile telephone. The figure for demonstrating an example of the system configuration which implement | achieves the utilization method of this invention using a mobile telephone.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Computer, 2 ... Display, 3 ... Input device, 4 ... Image | video apparatus, 5 ... Video signal, 6 ... Video input device, 7 ... CPU, 8 ... Interface, 9 ... Memory, 10 ... External information storage device.

Claims (5)

Means for inputting video,
Means for detecting a character string region from the input frame image;
Means for extracting a first image feature of the character string region;
Display means for displaying video;
Designating means for designating a character string area of the image displayed on the display means;
Means for extracting a second image feature from a character string region of the image designated by the designation means;
A feature matching means for matching the first image feature and the second image feature to obtain a matching degree;
A character search device for video, comprising: output means for outputting a character string region corresponding to an image feature for which a degree of coincidence is obtained or a frame image including the character string region as a search result.   2. The character search device for video according to claim 1, wherein the output unit ranks the matching results obtained by the feature matching unit in descending order and outputs search results.   The means for detecting the character string area is such that there are locally concentrated equal luminance pixel strings in which pixels having a difference in luminance in a predesignated range are continuously connected by the length of the predesignated range. The character search device in the video according to claim 1 or 2, wherein Input video,
Detecting a character string region from the frame image of the input video based on its shape;
Extracting the first image feature without decomposing the character string area character by character;
Drawing the search target character string input by the character input means as an image;
Extracting the second image feature without decomposing the drawn character string image for each character;
Collating the first image feature with the second image feature to determine a degree of coincidence;
And a step of outputting the character string area including the character string obtained with the degree of coincidence. Means for inputting video,
Means for detecting a character string region from the input frame image;
Means for extracting the first image feature without decomposing the character string region for each character;
An input means for inputting an arbitrary character string code;
Character image generation means for drawing a character string corresponding to the character string code as an image;
Means for extracting a second image feature without decomposing the character image character by character;
A feature matching means for matching the first image feature and the second image feature to obtain a matching degree;
A character search device for video, comprising: output means for outputting a character string region corresponding to an image feature for which a degree of coincidence is obtained or a frame image including the character string region as a search result.

Images