KR20200051165A - Method of Internet Information Processing Automation in Internet Explorer - Google Patents

Abstract

The present invention relates to a method of automating Internet information processing in Internet Explorer and, more specifically, to a method of allowing internet Explorer driven by control of an application program to automatically search for or register information which a user requires according to a generated navigation document like autonomous driving of a vehicle when the application program generates the navigation document by collecting and combining HTML-related information corresponding to navigation information and search condition information, set by the user, wherein the search condition information corresponds to sensor information, after a general person without computer expertise searches for required Internet information in Internet Explorer driven by control of the application program and then sets search conditions for automatically searching for or registering the corresponding information, as an artificial intelligent vehicle combines navigation information and sensor information to perform autonomous driving and to a method for automatically using professional Internet information processing such as scrapping for programming.

Description

Method of Internet Information Processing Automation in Internet Explorer

The present invention relates to a method for automating Internet information processing in Internet Explorer (hereinafter referred to as IE), and more specifically, a general public without computer expertise uses an IE driven by application program control to automatically and automatically repeat necessary information on a website. HTML document information that can be searched or registered, item (hereinafter referred to as element) information that provides a user interface (UI) in a basic configuration unit of HTML documents, navigation information that is switching information between HTML documents, and user manipulation information (hereinafter, UI information) and element search condition setting information for retrieving only the information the user needs, element sibling information for recursively repeating equivalent element information, and page information for recursively repeating equivalent element information over multiple pages, etc. Create a document (hereinafter referred to as navigation document) containing It relates to a method of automatically executing Internet information processing by driving IE with application program control according to the navigation document, and a method of using it for writing a program for automatically retrieving web information called scraping or crawling.

In order to automatically search or register Internet information required on the web, the developer tool provided by a web browser such as IE or Chrome analyzes the structure of the HTML document, finds elements within the HTML document, and identifies attributes. , Communication technology such as HTTP protocol, which is a communication protocol based on TCP / IP, to import or register required HTML documents or resources on the web, and parsing technology that extracts necessary information from HTML documents, and programming technology to write scraping programs. This is used, and recently, it uses a library developed to enable web information processing automation programs without specific knowledge of HTTP protocol or HTML parsing.

In addition, in order for the application program for Internet information processing to control IE, the HTML document structure, various types of element attributes, and a technique for obtaining or changing a property according to each element attribute or performing a specific function, Among the various components and screens supported by IE and IE, the HTML screen, which is the client area that provides the HTML document-related UI, and the HTML object, create or obtain an HTML object, connect to the application program, and connect with the IE. There is a need for a technology that collects navigation information and UI information by monitoring events that generate HTML objects.

As described above, in order to automate Internet information processing, programming techniques are required along with HTML document analysis or communication technology, so in the present invention, a modern intelligent vehicle combines navigation information and sensor information with a problem that is difficult for a general user without computer expertise to implement. How to automatically execute Internet information processing repeatedly by generating a navigation document that combines search condition setting information with the way that ordinary users search for information on the Internet in IE driven by application program control, as if autonomous driving is performed. Is to provide

In addition, programmers who write programs such as scraping require a lot of time and effort to find the desired elements in the web browser developer tools and reflect them in the program source code. If you click, it is difficult to find desired element information and reflect it in program development due to navigation.

Therefore, the present invention provides a method for setting a search condition on a selected element or obtaining element information and reflecting it in an information retrieval automation program development without selecting any element on the HTML screen.

As described above, in order to achieve the purpose for a general user who does not have computer expertise to repeatedly search or register necessary information in IE and to develop a scraping program for automating Internet information processing, the user may use the IE screen. Means for collecting and storing HTML document information, element information, and UI information generated in the course of viewing and manipulating, and element information for a user to select any element to set a search condition, or to reflect in program development such as scraping If necessary, it is to provide a means for setting search conditions or obtaining necessary element information without navigation by IE manipulation or generating an IE PouUp menu.

In the present invention, as a means for collecting HTML document information, element information, and UI information, a PC screen search (Enumeration) is performed to search an executed IE screen, obtain an HTML screen and an object belonging to a top-level IE screen, and set the HTML event to be generated. , HTML Event and Program Event, HTML document information and cause of navigation, element information that generated navigation or event or search condition, and parent of multiple levels to enable the element to be specified in the automatic execution of navigation document Element information is collected, and in the case of an input attribute, UI information is collected, such as whether input is terminated by an Enter key or input is terminated by a mouse click.

Also, as a means for setting element search conditions or obtaining information without generating a navigation or IE PouUp menu when a user performs an operation on IE, 0 to 100% transparency can be set as shown in (300) of FIG. It is possible to set at the top of the HTML screen on the order, and all the IE screens (such as the Shield screen), such as a layered window that can prevent navigation or IE PopUp menu from being generated by a user's key or mouse input while viewing the HTML screen Alternatively, except for the toolbar, address input display, menu bar, horizontal / vertical scroll bar 230, and favorites bar, it is located on the top of the HTML screen 220, which is the client area of IE, and when clicked with the right mouse button, popup menu of IE Instead, launch the program PopUp menu, or execute certain functions in combination with the control keys (meaning one of Ctrl, Alt, or Shift), and click the left mouse button. , For example by the user mouse input coordinates on the screen is converted into HTML Shield screen coordinates to obtain the element of the object coordinates from the HTML object to provide a means for achieving the above object.

In the present invention, even an ordinary person without computer expertise can automatically search or register necessary Internet information repeatedly without expert help, so that the price change of a competitor's product can be identified in real time to cope with it, or he or she wants to use the computer Internet screen advertisement part he uses. It can be used as a substitute for an image. Even if the basic HTML document of the target site to process information is changed or the required search conditions or items are changed, only the process of creating a navigation document can be performed without changing the application program.

In addition, programmers who develop web information processing automation programs can view the HTML screen, select the desired element without worrying about the HTML document being switched due to navigation, check the element information, copy the string, or set a certain format to the specified memory. Transferring saves development effort and time.

1 is a step-by-step configuration diagram of a navigation document creation / execution operation.
Figure 2 is a navigation document creation process flow chart
3 is an exemplary view showing screen configuration and selection element information according to an embodiment of the present invention;
4 is an example of an application program PopUp menu
5 is an example of a sibling and page element selection and element search condition setting method.
6 is an example of an external function type
7 is an exemplary method of replacing an IE screen image according to an embodiment of the present invention
8 is an exemplary method of storing Internet audio data according to an embodiment of the present invention.

Hereinafter, a process of creating a navigation document in FIG. 1 will be described step by step, and will be described in detail with reference to related drawings.

(1) Program driving step (S100)

In the driving stage of the application program, a program necessary for generating a navigation document and a program that automatically processes Internet information according to the generated navigation document, two different programs can be used, and the execution parameters are set differently in one program. There is also a way to run it.

(2) IE object creation / connection step (S200)

When the application program is run, the program searches only the IE program that is the "IEFRAM" Class among currently running programs on the PC, and the first IE that is searched is the top-level IE, and obtains an IE object with IE screen information to control the IE. If it is not found, it creates an IE object and obtains IE screen information with the IE object to control the IE, and obtains an HTML screen and an HTML object with the IE screen information to control the HTML screen and the HTML object.

If IE is running as a result of the PC screen search, it is difficult to manage the existing running IE on the program, or if the user selects an unwanted IE by mistake, all existing IEs are executed to prevent mistakes that are considered as navigation. Exit and create a new IE to prevent mistakes.

Once you have obtained the IE and HTML objects, you can process events to occur by connecting to the Event Sinker, respectively, and you can navigate to the basic site pre-registered in the registry by controlling the application program.

In addition, if you obtain the forward / backward toolbar button screen and the position of IE and use the mouse hooking of the toolbar screen in the DLL to generate and process the application program event when the corresponding button is clicked, the navigation that occurred in addition to the HTML element manipulation is favorite (collection) Or it can be handled by navigation due to input in the address input display.

(2-1) Judgment on creating a navigation document (S210)

In the Internet information processing automation according to the present invention, the navigation document generation operation and the navigation document execution operation are two asynchronous separate tasks, which can be created and processed in two programs, or executed by differently executing parameters in one program. When processing with two programs, it is unnecessary to determine the operation of generating the navigation document (S210), and the navigation document execution operation program is a menu on the menu bar 130 of the program main screen 100 of FIG. 3 or sometimes the main screen 100 ) There is no need for itself, and since there is no need for Shield screen, there is no need for Shield screen creation processing logic and many Event processing logic to obtain UI information.

(3) IE control / operation navigation step (S300)

As a part of most processing for generating a navigation document, navigation occurrence provides HTML document information without distinction from the navigation information collection and search setting step (S400) centering around the repetition loop point (S310) of FIG. 2, Since the collected element information provides the cause information of the navigation, the description will be mainly focused on the individual codes in the flowchart of the navigation document generation process of FIG. 2.

 (3-1) Shield screen request determination (S311)

In the present invention, as described in the solution of the problem, in the process of using IE without a shield screen, the user determines that the required information is an element to automatically search or an element to generate navigation, or if the element needs to set search conditions, the user can use the Shield When the screen is used, and the function key or “SHIELD” menu of the menu bar 130 of the program main screen 100 of FIG. 3 is selected, the application program generates the Shield screen 300 as shown in FIG. 3 (S312) and “SHIELD The "menu" is changed to the "UNSHIELD" menu, and when the user input is restricted to the shield screen, the program is determined, and the shield screen is removed by the user using the "UNSHIELD" menu or a designated key such as Esc (S312).

When creating the Shield screen, the program calculates the horizontal and vertical start position of the HTML screen to generate the Shield screen (S312) and applies the IE's Zoom Rate to convert the coordinates of the Shield screen to the coordinates of the HTML screen. In addition, in case of mouse input to the element, it is assumed that there is a horizontal / vertical scroll bar 230, and the horizontal / vertical scroll offset value and screen view ratio information of the current HTML are always collected.

When the element is selected in the shield screen state as shown in FIG. 3 (221), the HTML document information 110 and the element information 120 are collected, and all or part of the content is selected on the screen 100 capable of being controlled by the program as shown in FIG. 3 If the text color is changed by including the parent parent element information, the information is displayed (123), and the style or attribute of the selected element is changed to change the background or text color (221). 110) and the element information 120 do not have to be displayed on the screen when used by the general public without computer expertise, and when writing a professional Internet automatic search program such as scraping, copy the displayed string or HTML document information or element Information is transferred to the memory required to write a scraping program using socket communication or shared memory, etc. It is used to create Internet information processing automation programs such as wrapping.

(3-2) Judging the occurrence of navigation (S313)

In the present invention, the navigation changes the HTML document, and the screen information (Window Handle) changes. It monitors navigation, DOCUMENT, and WINDOW related events among various events of IE, or (2) the HTML screen of the IE object creation / connection step and It uses a method of obtaining an HTML object, and various HTML Event monitoring such as mouse down, click, key down, and SUBMIT is required to collect the cause of navigation by the element.

The factors that cause navigation are various, such as program control, key or mouse input, such as menus or link elements in HTML, IE toolbars, address input, and favorites. If navigation occurs, the HTML that monitors the HTML event before navigation occurs If there is an active HTML object by determining whether there is an object (S314), the cause of the navigation occurrence is identified from among the above factors, such as (S410), so that the same navigation can be generated in the navigation document automatic execution step (S700), and navigation by element In the case of, the corresponding element information is collected, and when the necessary information is collected, the active HTML Event and object are invalidated.

After the information collection and invalidation process is completed, an HTML object is generated using new HTML screen information, and an event is generated in the new HTML to be activated and collected and new HTML document information is collected (S411).

In the present invention, the HTML document information is not all information of the HTML source source, but the HTML area (RECT) and scroll Left, Top other than the URL (111), Title, Cookie (112) of FIG. 3 required for the automatic execution step (S700) of the navigation document. By extracting items such as the number of elements included in the source of offset and URL, and making it into a structured format, the HTML document-related functions in the program use the same layout, and the information reference of selected and search conditions in the HTML document is set. It is possible information.

(3-3) Element setting (S315)

Element setting is to set the search condition of the selected element 222 of FIG. 4, but it is possible to combine the menu or function key or control key with mouse button click after selecting the element, but without the program popup menu, but in the embodiment of the present invention, the shield screen state Right-click on the pop-up menu instead of the IE PopUp menu to select the "element setting" menu item 410 of the menu 400, the content of the currently selected element 222 is displayed on the element search condition setting screen 500 in FIG. (510), the search term refers to the keyword or program coding method of kipris, OUTPUT FIELD A1 ~ A999 (540) for outputting the search results is the "PROJECT" menu of the menu 130 of the program main screen 100 of Figure 3 When setting a project by selecting an item, as with most information processing automation programs, the project name, whether INPUT / OUTPUT files are used, and the file name and file Set the type, automatic search end time, search interval and message notification contact, etc. Most of the cells are used in the case of Excel files, and in the case of Word Table or TEXT files, the field name is pre-filled in the beginning, A1 ~ A999 Means that data is stored within 1000 from the first 1 of the Excel file A Cell, and in the case of the setting method such as the sum of product price and courier cost in the search condition is more than 10,000 won, when setting the product price element search condition as above When setting the search conditions of the OUTPUT FIELD and the delivery cost element, OUTPUT FILE is summed and displayed as a minus / equal sign.

However, when the product registration date needs to be searched after a specific date, the date is calculated according to the displayed date, so the date 550 is specifically displayed.

In addition, the use of the external function 560 in FIG. 5 cannot select a desired element by the simple conditional expression or search term, cannot reproduce the navigation in a desired manner, or selects an option, and the option selection list window requiring sub-option selection is not opened or In order to solve the limitations of the information processing automation program, such as that no special JavaScript execution is reproduced, when writing the information processing automation program, the parameters according to the dictionary external function name and function name are declared and declared as external functions. Create and link a dummy function to return the function as an Objct type or DLL type normally. If an external function is set (560), create and link only that function. Can complement and struct Using a format is convenient for writing external functions.

In addition, in the sibling selection 520 of FIG. 5, when all sibling elements 223 equal to the first selected element 222 are searched, it is unrealistic to select all the elements one by one, and the number and contents of sibling elements are not fixed. In the present invention, when the HTML document structure virtually forms the parent element as an inverted tree having a chain form as a parent, when the sibling element 223 of the first selected element 222 is selected, the newly selected element is displayed 520 and the inverted When the first selected element 222 and sibling element 223 in the structured tree structure successively search for the parent element, and the same parent element is searched, the sibling element repeatedly searches for the child elements below the same parent element search. You can search all elements in one page of HTML, and information When it spans a page, the page selection 530 and the sibling page element 224 of the existing page are selected, and the elements of all pages are repeatedly cycled in the same manner as above.

As described above, for interactive manipulation within a dialog, it is necessary to consider a method of processing the dialog as a thread, and the element information in the present invention does not mean only the basic components in the HTML source document, and automatically executes the navigation document. In the process of selecting elements or ID or ID (121), Source Index, Class, Tag (122), Title, Text, Value, Src, Href, etc. The information (S412) including the element area and mouse coordinate information, the scroll offset of the HTML document information, and the element search setting information and sibling page information is made into a structure format, and the element information related functions use the same layout and the elements are It is possible to refer to the HTML document information.

(3-4) Media setting (S316)

In the present invention, there are two types of media settings: image settings and audio settings.

(3-4-1) Image settings

When using the function key or control key and mouse button click as described in the above element setting (3-3), or selecting the “media setting” menu item 420 of the program PopUp menu 400, the user is shown in FIG. 7. Tag (124), ID (125), Type (126) and Src (127) information of the selected element (225) is displayed, and the contents of Src are also displayed (610) on the AUDIO / IMAGE setting / registration screen (600). In the case of an image, it is location information (URI) with an image resource, and there is Src information other than rendering a separate image with ActiveX, etc. In most cases, it is possible to download the image through separate communication in the program using the Src information. If it is difficult to download due to security issues, the image displayed on the browser screen can be saved as a file in the same way as image capture. By changing the Src information, the image of the raw HTML can be changed to another image. One, if you specify the image file of the URI Local PC rather than the server may or may not have a web browser, support for security reasons in recent years.

If you want to designate and display the image of the Local PC, in the step of setting the selected element as the image selection 620, such as the image 225 displayed on the screen of FIG. 7, select 630 the INPUT FIELD and enter the FIELD name ( 640) (3-3) As described in the element setting, if the INPUT file is set to use in the project setting, and the entered field name is in the INPUT file set in the project, the name registered in the corresponding field of the INPUT file is Local PC image file name or server URI, and if there is no same field name as when INPUT file is not set, input name in INPUT FIELD is image file name in the project setting folder.

Also, if you select OUTPUT FIELD and enter a name, it is named in the same way as the above INPUT FIELD name. When specifying the same name with a serial number, download the image or capture the image using the method of file name 1 to file name 99. The image file is saved in the way described, and in the image setting step, the selected element area information is displayed on the screen or the selected element 225 is right-clicked without a shield screen to check the horizontal and vertical size of the image in the properties of the PopUp menu provided by IE. This is possible.

The next step is to create an image that you want to display. It is not necessarily the same as the area size of the image element or the aspect ratio is the same, but for the distortion-free image, the same ratio is desirable, and the created image is entered into the INPUT FIELD in settings. After completing the step of pasting the image file type pre-specified by the program to the name or the image file type such as JPG, PNG, and saving it in the designated folder, the program finds image display setting information in the navigation document, and transparency corresponding to the image setting element area Using the steps of creating an opaque screen such as a 0% layered window, specifying an element for displaying an image, and loading an image stored on a local PC and drawing on the newly created opaque screen, There is a way to display the image in IE.

In addition, before registering the setting information in the navigation document with the OK button in the above element image setting step, if MEDIA verification 650 is selected, it can be confirmed that the desired image is displayed on the screen 225 as shown in FIG. 7, and a new image is displayed. It is also possible to select the desired element in the Shield screen state, select the desired image file in the file selection dialog, display the image and save it in the navigation document.

 (3-4-2) Audio settings

The audio setting is for storing music, speech, or English sentences as a file. As described in (3-4-1) Image Settings, use the function key or control key and mouse button click or program PopUp menu (400) By selecting the “Media Settings” menu item 420, the same AUDIO / IMAGE setting / registration screen 600 is used, and in the case of the audio element 226 selected by the user, it usually has PlayList information and thus separate communication within the program As an audio file can be downloaded, if the element 226 selected in FIG. 8 shows the Href (129) and Text (128) information among the information contents displayed on the program main screen, it executes Java Script and "hear example pronunciation". It can be understood that human cognitive ability is an element that processes audio data, but elements must be automatically analyzed to automate Internet information processing. Since the analysis is wrong from the standpoint of the program, it is possible to collect and output English sentence data instead of audio data in the navigation document automatic execution step (S700), so in this case, if you set the audio elegant (660), collect sentence data instead of audio data Error can be prevented.

In addition, when an audio file is downloaded through the PlayList information, it can be quickly transmitted, so it is possible to create a file before audio playback, but in special cases such as encryption of audio data or changing the audio reproduction method through ActiveX, all audio There is a disadvantage in that audio files can be generated only when playback is completed, but an audio file generation method using a LoopBack recording method capable of generating audio files in almost all cases is described.

The steps for selecting the audio element and setting the audio for the selected element and specifying the audio file are similar to the element image setting, so detailed description is omitted, and in the audio setting step, the image setting and the file name to save the audio file are OUTPUT FIELD The input 680 can be specified after all audio playback is finished, and the silence time 670 in the audio setup step is to loop back the audio data output from the speaker to the microphone input to the microphone input to capture the input strength of the microphone using the PCM method. For the purpose of generating audio files by determining that there is no more audio data when the set silence time has elapsed because the input intensity is less than the reference intensity for a certain period of time, the silence time between clauses and clauses is recorded when recording music in multiple clauses. 10 seconds if more than 9 seconds or less Set.

After setting the element audio as above, the program executes the steps to play the audio by generating a mouse click or mouse down and mouse up messages according to the element's area or mouse coordinate information, and selects the speaker to play and creates an object. And setting and initializing the sample bit number and sample rate in PCM format, setting the speaker to the capture service state, saving the PCM data within the silence duration, and when the silence time has elapsed. In the step of releasing the object, it is also possible to play the audio after the step of setting the speaker to the capture service state in the above, and converting PCM data to MP3 data by using a library provided by LAME or by writing the conversion logic yourself You can add

In the case of audio, instead of playback by program control in the absence of Shield screen without the need for audio setting, the file name and silence time to save the played audio data are fixed in the program or set in the project setting process, so that the user can select the audio-related elements. When using the control key and left mouse button to induce audio playback, the application program predicts that audio will be played when the specified control key is selected, and the audio data is downloaded, played, and played back. The designated output file or output file after playback is designated and stored, and the MEDIA verification 650 verifies whether or not an audio file is normally generated during the navigation document creation operation.

(4) Navigation document generation and storage (S500)

When the navigation document creation and storage step is completed by selecting all elements for collecting or registering Internet information required in the IE control / operation and navigation step (S300) and navigation information collection and search setting step (S400), the navigation document is generated. In order to automatically and automatically process Internet information in the automatic execution step (S700), the step of storing the information already collected in the IE control / operation and navigation step and the navigation information collection and search setting step. Is the step of selecting / manipulating and manipulating HTML document information by index and collecting information in the order of occurrence of the event and search set elements in a structure format, compressing and storing many parts without information.

(5) Navigation Document LOAD step (S600)

As described in (2-1) Navigation Document Generation Job Judgment, Internet information processing automation according to the present invention is a separate operation that repeatedly automatically executes Internet information processing based on the task of generating a navigation document and the generated navigation document. The navigation Document LOAD step in two operations is the third step of automatically executing navigation Documnet Internet information processing. (4) The navigation document file saved in the navigation document creation and storage step is read. It is placed in the memory as it was before saving the document creation.

(6) Navigation document automatic execution step (S700)

The navigation document auto-execution step is an IE control / operation in IE driven by the control of an application program. HTML document information and element information and search of navigation document information collected in a navigation step (S300) and a navigation information collection and search setting step (S400). The application program directly controls IE without user interaction with reference to condition setting information, etc., and controls navigation with the URL in the HTML document information according to the cause of the navigation, and specifies the corresponding element within the HTML according to the key input information, and the value value Set, control IE's screen view ratio and scroll position of HTML according to the element's area or mouse input coordinate information, and control IE to be automatically executed by clicking on the element or sending a mouse down / up message to the HTML screen. The set elements for search are set information and Saekeo, and conduct information and settings are stored based on the search terms, the search repetition cycle, depending on the presence or absence of such information, brothers, and processing the information to the Internet automatically repeated until the search end time and exit conditions, etc. specified project settings.

100: Program main screen
110: HTML basic information display unit
120: selection element information display unit
130: program main menu
200: IE web browser screen
210: Toolbar forward / backward button
220: HTML screen that is an IE client area
230: IE vertical scroll bar
300: Shield screen
400: Program PopUp menu
500: Element search condition setting screen
600: AUDIO / IMAGE setting screen

Claims (12)

Application program driving step (S100), the application program to create an Internet Explorer object and obtain and control screen information, obtain and control HTML screen information and HTML object as the screen information (S200), and control the application program The step of creating a shield screen 300 such as a layered window in which the application program can adjust the transparency of 0 to 100% and be positioned on the top of the HTML screen by the user's selection in the driven Internet Explorer, and the user places the Shield on the top of the HTML screen. How to prevent the occurrence of IE's PopUp menu or navigation by clicking the right mouse button on the screen or clicking the element that generates navigation, and the Shield screen receives input. Application program driving step (S100), the application program to create an Internet Explorer object and obtain and control screen information, obtain and control HTML screen information and HTML object as the screen information (S200), and control the application program When the user selects the driven Internet Explorer, the application program generates the Shield screen 300 and places it on the top of the HTML screen, the user selects an element with the mouse through the Shield screen, and the user selects the element on the Shield screen. Converting mouse coordinates into HTML screen coordinates by an application program, obtaining an object of an element selected by a user from an HTML object using HTML screen coordinates, and obtaining element information from the element object Step of driving an application program, searching for Internet Explorer in which the application program is running using PC screen search, obtaining and controlling an Internet Explorer object with screen information of one of the searched Internet Explorers, and controlling Internet Explorer driven by application program control In the user selection, the application program creates the shield screen and places it on the top of the HTML screen, the application program converts the mouse coordinates on the shield screen where the user selects the element into HTML screen coordinates, and the HTML object using the HTML screen coordinates Obtaining the object of the element selected by the user from, Method for obtaining the element information from the element object The method according to claim 2 or 3, wherein the user selects the element selected by the user by changing the style property of the element object obtained by selecting with the mouse through the shield screen and changing the text or background color to display (221). Claim 2 or claim 3, the step of obtaining the element information of the parent of the parent step of the element with the information of the element selected by the user, the information required for the creation of a web information collection automation program, such as scraping from the element information and parent element information Extracting, displaying the extracted elements information on the screen generated by the application program controlling Internet Explorer (120), selecting a part of the displayed contents and reflecting it in the source code to create scraping, etc. by copying a string How to Transferring the information of the element selected by the user of claim 2 or claim 3 and the parental element information obtained from claim 5 to a memory necessary for writing a web information collection automation program such as scraping using an inter-process communication method such as socket communication or shared memory. How to use it to write programs In Internet Explorer driven by application program control, the application program generates a shield screen and places it on the top of the HTML screen by user selection, selecting an element with a mouse through the shield screen (222), a function key or a right mouse button The popup menu 400 of the application program is generated to select the element setting item 410 to display the element setting screen 500, the step of selecting sibling elements equivalent to the selected element, the first selected element and the later selected element Step of repeatedly searching each parent element of the sibling element to find the same parent element, and recursively recursively searching the child elements of the same parent element to retrieve all of the desired sibling element information In Internet Explorer driven by application program control, the application program generates a shield screen and places it on the top of the HTML screen by user selection, selecting an element with a mouse through the shield screen (222), and sibling equal to the selected element Selecting an element with a control key with a mouse button, repeatedly searching the parent element of each of the first selected element and the later selected sibling element to find the same parent element, and repeatedly searching through the child elements of the same parent element to repeat the desired sibling How to retrieve all of the element information In order to solve the error caused by the limitation of the information processing automation program in the information collection step for generating the navigation document, the step of declaring the predefined external function when writing the information processing automation program is normal without performing the function. Writing in Object or DLL method to return, Linking with the application program, driving Internet Explorer through application program control, setting the error occurrence element search condition (500), selecting the external function type, and selecting the element of the search condition How to supplement without modifying the entire program, including the error analysis step, the step of writing source code for the function according to the selected external function type for error correction, and the step of re-linking the external function written in the Object method with the above application program. In Internet Explorer driven by application program control, the application program creates a shield screen and places it on the top of the HTML screen by user selection, selecting an image element on the HTML screen, and a pre-written image file to replace the selected element image Using the step of selecting, the step of creating an opaque screen such as a shield screen suitable for the selected element area and placing it on the top of the HTML screen, and drawing the selected image file on the opaque screen to display the desired image on the Internet Explorer screen. How to display Steps to preset the file name and silence time to save the audio data to be played by project settings, etc. In Internet Explorer driven by application program control, click the audio generation element with the left mouse button with the control key without the shield screen to play audio. Step of inducing, step of predicting the audio playback by the application program, selecting and activating the speaker to be played, creating and activating the object, setting and initializing the sample bit number and sample rate in the LoopBack PCM format, and capturing the speaker in the capture service state How to save audio data using the step of setting, saving the PCM data within silence duration The method of claim 11, wherein the step of converting the PCM data into MP3 data and the step of storing the converted MP3 file before the step of storing the PCM data are stored in the Internet Explorer.

Images