KR102209316B1 - Method for storing photo recording using metadata and terminals device for performing the method - Google Patents

Abstract

Disclosed are a method of storing photographic records using metadata and a terminal device performing the same. The disclosed method includes generating metadata for each of a plurality of photographic records; And matching and storing the plurality of photo records with corresponding meta data; wherein the meta data includes a keyword for the photo record, object information present on the photo record, and on the photo record It includes at least one of existing person information. According to the present invention, there is an advantage in that photographic records can be more efficiently stored, managed, and searched using standardized metadata.

Description

A method of storing photographic records using metadata, and a terminal device performing the same.

Embodiments of the present invention relate to a method for storing, managing, and retrieving photographic records more efficiently using metadata, and a terminal device performing the same.

Recently, public records have been recognized as important sources of high historical value, and each local government is making efforts to recognize and preserve the importance of the records.

In the case of general records, that is, text records, there is a standard of metadata, so it is managed with uniformity, but in the case of photographic records, there is no standard for metadata, so it is managed without uniformity. That is, in the case of photographic recordings, metadata is stored in different ways because there is no standard of metadata, making it difficult to interchange photographic recordings between different local governments.

In addition, since metadata is manually input one by one each time a new photo is registered, time and human consumption are large.

In addition, due to the nature of work of local government officials, it is often necessary to search for photos related to a specific person or event. However, due to the absence of sophisticated metadata, it is difficult to search for exact keywords. This is because public officials manage photos based on the file name without a certain metadata design when managing photos, so there is a problem in that the notation method of recording photos and the file name rules are different whenever the person in charge changes.

In order to solve the problems of the prior art as described above, the present invention proposes a method for storing, managing, and retrieving photo records more efficiently using standardized metadata, and a terminal device for performing the same. I want to.

Other objects of the present invention may be derived by those skilled in the art through the following examples.

In order to achieve the above object, according to a preferred embodiment of the present invention, there is provided a method for storing photographic records performed in an apparatus including a processor, the method comprising: generating metadata for each of a plurality of photographic records; And matching and storing the plurality of photo records with corresponding meta data; wherein the meta data includes a keyword for the photo record, object information present on the photo record, and on the photo record There is provided a method for storing photographic records, comprising information about a person present.

In the generating step, character recognition is performed on the photographic record to extract text present on the photographic record, and the keyword may be extracted by performing a morpheme analysis on the extracted text.

In the generating step, character recognition is performed on the photographic record to extract text present on the photographic record, and a single character string including all the extracted texts and no spaces is generated, and the extracted text And morpheme analysis on the one character string to extract the keyword.

The generating may include inputting the photographic recording into a pre-learned deep learning-based algorithm model to extract object information existing on the photographic recording and information on a person existing on the photographic recording.

Among the deep learning-based algorithm models, the object recognition algorithm model may include Regions-CNN (R-CNN), a neural network derived from R-CNN, You Only Look Once (YOLO), SSD Single Shot MultiBox Detector).

When a new object that has not been learned in advance exists in the photographic record, the object recognition algorithm model additionally learns a plurality of images including the name information of the new object and displaying a bounding box surrounding the new object using training data. Can be.

Among the deep learning-based algorithm models, a face recognition algorithm model may include VGGFace, FaceNet, and NAN (Neural Aggregation Network).

The metadata may include a title of the photographic record, a photographer of the photographic record, a photographing date of the photographic record, a photographing location of the photographic record, a description of the photographic record, a name of an event performed on the photographic record, and the photographic record. The original picture number, which is the name of the original photo file of, the creator registering the photo record, the horizontal and vertical size of the photo record, the location where the photo record is stored in the device, the file size of the photo record, and the image of the photo record. It may further include a compression format, the category of the photo record, and the number of views of the photo record.

The metadata may be used when searching for the photographic record.

In addition, according to another embodiment of the present invention, there is provided a memory for storing instructions readable by a computer; And a processor embodied to execute the command, wherein the processor generates metadata for each of a plurality of photographic records, and controls to match and store the plurality of photographic recordings with the corresponding metadata, The meta data includes a keyword for the photographic record, object information present on the photographic record, and information on a person present on the photographic record.

According to the present invention, there is an advantage in that photographic records can be more efficiently stored, managed, and searched using standardized metadata.

In addition, the effects of the present invention are not limited to the above effects, and should be understood to include all effects that can be deduced from the configuration of the invention described in the detailed description or claims of the present invention.

1 is a diagram showing a schematic configuration of a terminal device according to an embodiment of the present invention.
2 is a diagram illustrating a flowchart of a method of storing photographic records according to an embodiment of the present invention.
3 to 7 are diagrams for explaining the concept of generating meta data according to an embodiment of the present invention.

Singular expressions used in the present specification include plural expressions unless the context clearly indicates otherwise. In the present specification, terms such as “consisting of” or “comprising” should not be construed as necessarily including all of the various elements or various steps described in the specification, and some of the elements or some steps It may not be included, or it should be interpreted that it may further include additional elements or steps. In addition, terms such as "... unit" and "module" described in the specification mean units that process at least one function or operation, which may be implemented as hardware or software, or as a combination of hardware and software. .

Hereinafter, various embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram showing a schematic configuration of a terminal device according to an embodiment of the present invention.

Referring to FIG. 1, a terminal device 100 is a device that stores and searches a photographic record, and includes an input unit 110, a memory unit 120, a processor unit 130, and a display unit 140. Hereinafter, a function performed by each component will be described.

The input unit 110 is a component that receives a photographic record from a user and text information from the user. For example, the input unit 110 may receive a photographic record transmitted through wired communication or wireless communication. In addition, the input unit 110 may include a mouse and a keyboard.

The memory unit 120 may be a volatile and/or nonvolatile memory, and stores commands or data related to at least one other component of the terminal device 100. In particular, the memory unit 120 may store instructions or data related to a computer program or a recording medium for generating meta data described below.

The processor unit 130 may include one or more of a central processing unit, an application processor, and a communication processor. For example, the processor unit 130 may control at least one other component of the terminal device 100 and/or perform an operation or data processing related to communication. In particular, the processor unit 130 may execute a command related to execution of the computer program.

The display unit 140 may be composed of a liquid crystal display (LCD), a light emitting diode display (LED), an organic light emitting diode display (OLED), etc., and displays an execution screen and output values of a computer program executed in the processor unit 130 can do.

According to an embodiment of the present invention, the terminal device 100 generates metadata for each of a plurality of photographic records, and controls the plurality of photographic recordings to match and store the corresponding metadata. In this case, the user can search for the photographic record using the metadata.

Hereinafter, a storage operation of a photographic record performed by the processor unit 130 in the terminal device 100 will be described in detail with reference to FIG. 2.

2 is a diagram illustrating a flowchart of a method of storing photographic records according to an embodiment of the present invention. Hereinafter, a process performed for each step will be described.

First, in step 210, metadata for each of a plurality of photographic records is generated.

Meta data includes information on photographic records, and consists of a plurality of text information.

In step 220, a plurality of photographic records are matched with corresponding metadata and stored. Thus, the user can search for photographic records using metadata.

According to an embodiment of the present invention, metadata reflects the characteristics of a photographic record, and includes a title, a photographer, a photographing date, a photographing location, a description, a group name, a keyword, an original picture number, a creator, a horizontal and vertical size, and an image file. It may include location, image file size, image compression format, category, number of views, people present in the photo, and object information in the photo.

The information included in the meta data will be described in more detail as follows.

1) Title: The name of the photographic record (ie, photo file)

2) Creator: The name of the photographer who takes the photographic record, or the name of the group if the photographer is not identified.

3) Date Created: The date the photo record was taken

4) Location Created: The location where the photographic record was taken

5) Group Name: The name of the event performed on the photo record (i.e., the name of the event related to a specific place or person)

6) Original Name: The name of the original photo file when photographing recordings with the camera

7) Registrant: The person who registers the photographic record.

8) Keywords: Information indicating the subject, content, etc. of the photographic record

9) Width & Height Size: The horizontal and vertical length of PPI (Pixels Per Inch), which is the resolution unit of an image that is a photographic record.

10) Image File Location: Location (path) where photo records are stored in the terminal device

11) Image File Size: The size of a photo file that is a photo record

12) Image Compressed Format: Compressed format of photo files in photo records,

13) Description: A description of the situation occurring in the photographic record.

14) Category: Category according to the type of photo record

15) Views: The number of times the photo record was opened (updated)

16) Person Shown in the Image: Information on the person present in the photographic record

17) Object in the Image: Information on objects present on photographic records

Here, 1) to 12) of the meta data may be mandatory items, and 13) to 17) may be optional items.

Hereinafter, an operation of extracting "keyword", "person displayed in photo", and "object in photo" from information included in metadata will be described in more detail.

First, with respect to "keyword", the processor unit 130 extracts text existing on the photographic recording by performing character recognition on the photographic recording.

As an example, the processor unit 130 may extract text from a photographic record using an OCR (Optical Character Recognition) library algorithm or OCR software. In this case, the OCR software may be Google OCR, MS OCR, Abbyy OCR, or Tesseract. 3 shows an example of extracting text from a photographic record.

In addition, the processor unit 130 may extract keywords by performing morpheme analysis on the extracted text. Here, through morpheme analysis, keywords may be extracted based on nouns excluding numbers, surveys, one syllable, and special characters. The extracted keywords are mapped to "keywords" in the meta data. 4 shows an example of extracting a keyword using a morpheme analyzer.

Meanwhile, an error may occur in the keyword extraction process described above.

That is, referring to FIG. 5A, among the extracted keywords, a case in which "collection" is tokenized one syllable may occur. In order to prevent such an error, the processor unit 130 may extract a keyword by generating a single string including all the extracted text and without spaces, and performing a morpheme analysis on the generated single string. FIG. 5B illustrates the concept of generating one character string (ie, text after pre-processing) from the extracted text (ie, text before pre-processing).

However, when a keyword is extracted from one character string, an error may occur in which the keyword that was normally output from the text that has not been preprocessed is tokenized by one syllable after the preprocessing. Figure 6 (a) shows the result of extracting keywords by performing morpheme analysis on the extracted text (i.e., text before pre-processing), and in Figure 6 (b), one character string (i.e., text after pre-processing) The result of extracting keywords by performing morpheme analysis on is shown.

Therefore, according to an embodiment of the present invention, the processor unit 130 extracts text existing on the photographic recording by performing character recognition on the photographic recording, and includes all the extracted texts, and includes a single character string without spaces. Is generated, and a keyword can be extracted by performing a morpheme analysis on both the extracted text and one string. 7 shows an example of extracting keywords by performing morpheme analysis on both the extracted text and one character string.

Next, with respect to the "object in a picture", the processor unit 130 may extract information on an object existing on the photographic recording by inputting the photographic recording into a pre-learned deep learning-based algorithm model. At this time, among the algorithm models based on deep learning, the algorithm model that recognizes the object is R-CNN (Regions-CNN), a neural network derived from R-CNN (Fast R-CNN, FASTER R-CNN, etc.), and YOLO (You only). Look Once), SSD (Single Shot MultiBox Detector) may be included.

In more detail, CNN (Convolutional Neural Network) is a deep learning algorithm created for the purpose of extracting features of an image, and neural networks derived from R-CNN to R-CNN are CNNs that can perform faster computation by proposing regions. YOLO is an algorithm that excludes one regression problem from the bounding box and class probability, selects an image as one object, and calculates the boundary probability by placing several bounding boxes on one convolutional network. It is an algorithm that recognizes objects as multiple boxes and performs recognition in a unified framework. Since the above algorithms are widely known to those skilled in the art, detailed descriptions will be omitted.

Meanwhile, in the process of extracting object information described above, a new object that is not previously defined and thus cannot be recognized may exist in the photographic record. Therefore, in the present invention, additional learning is performed to recognize a new object. In this case, for each of the plurality of images including the new object, a bounding box surrounding the new object is displayed, and a plurality of images including name information of the new object and displaying the bounding box are used as training data.

Next, with respect to the "person displayed in the picture", the processor unit 130 inputs the photographic recording into a face recognition algorithm model among the pre-learned deep learning-based algorithm models to provide information on a specific person present on the photographic recording. Can be extracted. In this case, the face recognition algorithm model may include VGGFace, FaceNet, and NAN (Neural Aggregation Network).

In more detail, VGGFace is a deep learning algorithm that recognizes faces using a neural network structure composed of five convolutional layers, and FaceNet is a Euclidean distance between features extracted from the same portrait picture. It is a deep learning algorithm that recognizes faces based on the feature of being smaller. In addition, NAN (Neural Aggregation Network) is a deep learning algorithm that recognizes a face by extracting a single feature from several frames of a video. Since the above algorithms are widely known to those skilled in the art, detailed descriptions will be omitted.

In short, in the present invention, by standardizing the meta data of the photographic record with the information described above, it enables compatibility of the photographic record between each group, and some of the information of the meta-data is automatically input, and the photographic record is more efficient. It has the advantage of being able to store, manage, and search by.

Further, the embodiments of the present invention may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the present invention, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Examples of program instructions such as magneto-optical, ROM, RAM, flash memory, etc., can be executed by a computer using an interpreter as well as machine code such as those made by a compiler. Contains high-level language code. The above-described hardware device may be configured to operate as one or more software modules to perform the operation of the embodiments of the present invention, and vice versa.

As described above, in the present invention, specific matters such as specific components, etc., and limited embodiments and drawings have been described, but these are provided only to help the general understanding of the present invention, and the present invention is not limited to the above embodiments, Anyone of ordinary skill in the field to which the present invention pertains can make various modifications and variations from this description. Therefore, the spirit of the present invention is limited to the described embodiments and should not be defined, and all things that are equivalent or equivalent to the claims as well as the claims to be described later fall within the scope of the spirit of the present invention. .

Claims (12)

A method of storing photographic records performed in a device including a processor,
Generating metadata for each of the plurality of photographic records; And
Matching the plurality of photographic records with the corresponding meta data and storing; Including,
Generating the meta data,
Extracting a keyword for the photographic recording through character recognition from the photographic recording; And
Extracting object information and person information existing on the photographic recording by applying a pre-learned deep learning-based algorithm model to the photographic recording
Including,
The step of extracting the keyword,
Character recognition is performed on the photographic record to extract text present on the photographic record, and a single character string including all the extracted texts and no spaces is generated, and the extracted text and the single character string are And extracting the keyword by performing a morpheme analysis on the photographic record. delete delete delete The method of claim 1,
Among the deep learning-based algorithm models, the object recognition algorithm model includes R-CNN (Regions-CNN), a neural network derived from R-CNN, YOLO (You Only Look Once), and SSD (Single Shot MultiBox Detector). How to save photo records The method of claim 1,
In the step of extracting the object information and person information,
When a new object that has not been learned in advance exists in the photographic record, a plurality of images including the name information of the new object and displaying a bounding box surrounding the new object are selected from the deep learning-based algorithm model using training data. An object recognition algorithm model is additionally learned. The method of claim 1,
Among the deep learning-based algorithm models, facial recognition algorithm models include VGGFace, FaceNet, and NAN (Neural Aggregation Network). The method of claim 1,
The metadata may include a title of the photographic record, a photographer of the photographic record, a photographing date of the photographic record, a photographing location of the photographic record, a description of the photographic record, a name of an event performed on the photographic record, and the photographic record. The original picture number, which is the name of the original photo file of, the creator registering the photo record, the horizontal and vertical size of the photo record, the location where the photo record is stored in the device, the file size of the photo record, and the image of the photo record. And a compression format, a category of the photographic record, and number of views of the photographic record. The method of claim 1,
The metadata is used when searching for the photographic recording. A memory for storing computer-readable instructions; And
Processor embodied to execute the instruction: including,
The processor controls to generate metadata for each of a plurality of photographic records, and to match and store the plurality of photographic recordings with the corresponding metadata,
The processor extracts a keyword from the photographic record through character recognition for the photographic record, and applies a pre-learned deep learning-based algorithm model to the photographic record to obtain object information and person information existing on the photographic record. Extract and generate the metadata,
The processor performs character recognition on the photographic record to extract text present on the photographic record, generates a single character string including all the extracted texts and without spaces, and generates the extracted text and the A terminal device comprising extracting the keyword by performing morpheme analysis on one character string. delete The method of claim 10,
The metadata may include a title of the photographic record, a photographer of the photographic record, a photographing date of the photographic record, a photographing location of the photographic record, a description of the photographic record, an original picture number that is a name of an original photo file of the photographic record, Creator registering the photographic record, the horizontal and vertical size of the photographic record, the location where the photographic record is stored in the device, the file size of the photographic record, the image compression format of the photographic record, the category of the photographic record, the Terminal device, characterized in that it further comprises the number of views of the photographic record.

Images