CN116934256A - Evaluation method, supervision method and system based on marking points of frame selection marking technology - Google Patents
Evaluation method, supervision method and system based on marking points of frame selection marking technology Download PDFInfo
- Publication number
- CN116934256A CN116934256A CN202310883156.8A CN202310883156A CN116934256A CN 116934256 A CN116934256 A CN 116934256A CN 202310883156 A CN202310883156 A CN 202310883156A CN 116934256 A CN116934256 A CN 116934256A
- Authority
- CN
- China
- Prior art keywords
- content
- scoring
- expert
- document
- review
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000011156 evaluation Methods 0.000 title claims abstract description 76
- 238000000034 method Methods 0.000 title claims abstract description 45
- 238000012552 review Methods 0.000 claims description 88
- 230000008569 process Effects 0.000 claims description 9
- 238000010234 longitudinal analysis Methods 0.000 claims description 8
- 238000012015 optical character recognition Methods 0.000 claims description 7
- 230000004044 response Effects 0.000 claims description 6
- 238000009825 accumulation Methods 0.000 claims description 4
- 238000013077 scoring method Methods 0.000 claims 3
- 238000007405 data analysis Methods 0.000 abstract description 6
- 230000006870 function Effects 0.000 abstract description 6
- 210000004556 brain Anatomy 0.000 abstract description 5
- 238000000605 extraction Methods 0.000 abstract description 4
- 230000007246 mechanism Effects 0.000 abstract description 4
- 239000003550 marker Substances 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 238000004458 analytical method Methods 0.000 description 6
- 230000006399 behavior Effects 0.000 description 5
- 238000009432 framing Methods 0.000 description 4
- 238000012854 evaluation process Methods 0.000 description 3
- 238000011835 investigation Methods 0.000 description 3
- 230000009471 action Effects 0.000 description 2
- 238000012550 audit Methods 0.000 description 1
- 230000001149 cognitive effect Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/10—Office automation; Time management
- G06Q10/103—Workflow collaboration or project management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q30/00—Commerce
- G06Q30/06—Buying, selling or leasing transactions
- G06Q30/08—Auctions
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Business, Economics & Management (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Strategic Management (AREA)
- General Physics & Mathematics (AREA)
- Human Resources & Organizations (AREA)
- General Engineering & Computer Science (AREA)
- Entrepreneurship & Innovation (AREA)
- General Business, Economics & Management (AREA)
- Economics (AREA)
- Finance (AREA)
- Accounting & Taxation (AREA)
- Marketing (AREA)
- Health & Medical Sciences (AREA)
- Operations Research (AREA)
- Quality & Reliability (AREA)
- Development Economics (AREA)
- Tourism & Hospitality (AREA)
- Artificial Intelligence (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Data Mining & Analysis (AREA)
- Human Computer Interaction (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The application discloses a bid evaluation method, a supervision method and a system based on a bid selection marking technology point, which relate to the technical field of intelligent bid evaluation, wherein the function of the bid selection marking technology point is added in the existing bid evaluation system, a comment expert can carry out the bid selection marking on a to-be-evaluated file in the bid evaluation system, the scoring basis of the subjective comment item by the expert can be definitely clarified, meanwhile, the expert is supported to quickly jump, edit and delete between the scoring points selected by different frames, the text extraction and comparison are carried out on the scoring point parts selected by the expert frame, the scoring point contents of different units are transversely compared, a learning mechanism is established, and a data basis is made for the bid evaluation of the big data analysis expert. The marking points of the frame selection marking technology fundamentally prevent the specialists from scoring without looking at the file or reasonably avoiding the acquiescing scoring situation among the specialists by virtue of subjective impression or scoring mode of 'clapping the brain gate', and provide basis for the supervision department to analyze the professional level of the specialists and whether illegal evaluation exists.
Description
Technical Field
The application relates to the technical field of intelligent label evaluation, in particular to a label evaluation method, a supervision method and a supervision system based on a label marking point of a frame selection marking technology.
Background
In the existing bid evaluation process, an expert submits a bid file according to a bidder or a supplier for comparison and verification, and evaluation and scoring are carried out. After the evaluation is finished, the supervision department can analyze the expert scoring behavior and supervise the expert level and whether the violation evaluation exists.
However, when the expert evaluates the mark, the evaluated mark standard is inconsistent due to inconsistent understanding of the standard in the scoring rule by each person or deviation of subjective cognitive factors in the scoring rule. Therefore, when the expert scoring behavior is analyzed, no objective basis is available, and the situation that the expert scores by means of subjective impression or a scoring mode of 'clapping a brain gate', scoring without looking at a file by the expert, or reasonable prizing between the experts and the like cannot be avoided.
Disclosure of Invention
In view of the above, the application provides a bid evaluation method, a supervision method and a supervision system based on frame selection marking technology marking points, wherein the function of the frame selection marking technology marking points is added in a comment system, so that an expert can directly frame selection marking technology marking points in bidding documents in the bid evaluation process, the technology marking points are scoring bases, scoring of the expert without looking at the documents is avoided, and analysis bases are provided for expert supervision.
For this purpose, the application provides the following technical scheme:
in one aspect, the application provides a label evaluation method based on a frame selection marking technology scoring point, which comprises the following steps:
receiving an instruction for opening a bid file, and displaying a quasi-review document and an editable layer positioned on the upper layer of the quasi-review document; the to-be-reviewed document is a non-editable read-only document;
receiving each frame selection mark on the editable layer and determining the coordinate position of each frame selection mark on the editable layer; the box selection mark is a box selection mark of the review expert on the content meeting the review standard of the review item in the quasi-review document in the review process;
determining contents corresponding to each box selection mark in the to-be-reviewed document based on the coordinate positions of the box selection marks on the editable layer;
receiving the scores of the review experts on each review item;
and storing the box selection marks in a list, and carrying out association storage with the corresponding contents and scores.
Further, the box selection marker includes: shape box selection notes, text selection notes, document underline notes and remark text filling.
Further, the method further comprises the following steps: and each box selection mark of the to-be-reviewed document and corresponding content thereof are called from the list.
Further, the method further comprises the following steps: jump, edit or delete between different box selection markers.
Further, the method further comprises the following steps: and converting the content corresponding to the box selection mark in the document to be reviewed into an image, performing character recognition through an optical character recognition technology, and converting the content corresponding to the box selection mark into an editable character format backup.
In still another aspect, the present application further provides a marking system based on marking points by a frame selection marking technology, including:
the display module is used for receiving an instruction for opening the bidding document, displaying the to-be-reviewed document and an editable layer positioned on the upper layer of the to-be-reviewed document; the to-be-reviewed document is a non-editable read-only document;
a box selection mark module for receiving each box selection mark on the editable layer and determining the coordinate position of each box selection mark on the editable layer; the box selection mark is a box selection mark of the review expert on the content meeting the review standard of the review item in the quasi-review document in the review process;
the scoring module is used for scoring each review item in the to-be-reviewed document;
the annotating content module is used for determining the content corresponding to each box selection mark in the to-be-reviewed document based on the coordinate position of each box selection mark on the editable layer;
the first storage module is used for storing each frame selection mark in a list and carrying out association storage with the corresponding content;
and the calling module is used for calling the bidding document with the frame selection mark.
Further, the method further comprises the following steps: and the second storage module is used for converting the content corresponding to the frame selection mark in the to-be-reviewed document into an image, performing character recognition through an optical character recognition technology, and converting the content corresponding to the frame selection mark into an editable character format for backup.
In yet another aspect, the present application further provides a supervision method based on the marking points of the frame selection marking technology, including:
invoking a bid file scored by an expert to be supervised; the expert to be supervised scores the bidding documents by using the bid evaluation system based on the marking points of the frame selection marking technology;
judging whether a frame selection mark exists in the bidding document; if yes, extracting content corresponding to the box selection mark, judging whether the content is related to the scoring item, and determining the evaluation quality of the expert to be supervised based on the relativity of the content and the scoring item; if not, determining that the review quality of the expert to be supervised is low;
and carrying out image drawing on the comment habit or style of the expert to be supervised based on the comment quality and the comment content of the expert to be supervised, wherein the comment content comprises a scoring item, a frame selection marked content and a score.
Further, determining whether the content is related to a scoring item includes: analyzing whether the content of the frame selection mark has relevance with the scoring item or not by adopting a transverse comparison and longitudinal analysis method;
the lateral alignment includes: transversely comparing bidding documents of multiple bidders, and judging whether contents corresponding to the box selection marks are similar or not; the longitudinal analysis includes: and performing type learning on the content of the same bid evaluation method through data accumulation, learning response characters corresponding to the score points in the historical bid documents, comparing the content of the selection mark with the historical response characters, and judging whether the content meets the requirement of the bid evaluation content.
In yet another aspect, the present application further provides a supervision system based on the marking points of the box selection marking technology, including:
the file acquisition module is used for retrieving the bid files scored by the expert to be supervised; the expert to be supervised scores the bidding documents by using the marking system based on the marking points of the frame selection marking technology;
the frame selection mark judging module is used for judging whether the bidding document has a frame selection mark or not; if yes, calling a box selection content judging module, and if not, calling a review quality judging module;
the frame selection content judging module is used for extracting the content of the frame selection mark and judging whether the content is related to the scoring item when the frame selection mark judging module judges that the frame selection mark exists in the bidding document;
the evaluation quality judging module is used for determining the evaluation quality of the expert to be supervised based on the correlation degree of the content and the scoring item; or when the box selection mark judging module judges that the box selection mark does not exist in the bidding document, determining that the review quality of the expert to be supervised is low;
and the portrait module is used for carrying out portrait on the comment habit or style of the expert to be supervised based on the comment quality and the comment content of the expert to be supervised, which are determined by the comment quality judging module, wherein the comment content comprises a comment item, the content of a frame selection mark and a score.
The application has the advantages and positive effects that:
according to the evaluation method and system based on the frame selection marking technology marking points, the function of the frame selection marking technology marking points is added to an existing evaluation system, an evaluation expert can perform frame selection marking on a to-be-evaluated file in the evaluation system, the evaluation basis of the subjective evaluation item by the expert can be definitely clarified, meanwhile, the expert is supported to rapidly jump, edit and delete among different frame selection marking points, text extraction and comparison can be performed on the marking point parts selected by the expert by the frame, the marking point contents of different units are transversely compared, a learning mechanism is established, and a data basis is formed for the evaluation of the big data analysis expert.
In the supervision method and system based on the frame selection marking technology scoring points, the frame selection marking technology scoring points radically prevent experts from scoring the files without looking at the files or reasonably avoiding the acquiescing scoring conditions among the experts by means of subjective impressions or a scoring mode of 'clapping a brain', and the extracted frame selection content is insubstantial content or is an important investigation point for expert scoring behavior analysis, so that basis is provided for supervision departments to analyze the professional level of the experts and whether illegal reviews exist.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, and it is obvious that the drawings in the following description are some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort to a person skilled in the art.
FIG. 1 is a flowchart of a marking method based on a marking point of a frame selection marking technology in an embodiment of the application;
FIG. 2 is a diagram showing an example of box selection marks in an embodiment of the present application;
FIG. 3 is a diagram of an example of a box selection marker according to an embodiment of the present application;
FIG. 4 is a diagram of yet another exemplary box selection marker according to an embodiment of the present application;
FIG. 5 is a diagram of yet another exemplary box selection marker according to an embodiment of the present application;
FIG. 6 is a block diagram of a marking system based on a marking point of a frame selection marking technique according to an embodiment of the present application;
FIG. 7 is a block diagram of a marking system based on a marking point of a frame selection marking technique according to an embodiment of the present application;
FIG. 8 is a flowchart of a supervision method based on a frame selection marking technique point in an embodiment of the application;
fig. 9 is a block diagram of a supervision system based on a frame selection marking technology point in an embodiment of the application.
Detailed Description
In order that those skilled in the art will better understand the present application, a technical solution in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present application, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present application without making any inventive effort, shall fall within the scope of the present application.
It should be noted that the terms "first," "second," and the like in the description and the claims of the present application and the above figures are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the application described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
As shown in FIG. 1, in the embodiment of the application, the bid evaluation method based on the marking points of the frame selection marking technology is provided, the bid files are generally non-editable read-only files, such as PDF format files, and when the bid files are evaluated, the evaluation expert can only see the bid files for evaluation, but cannot make notes and comments on the bid files, and cannot give the evaluation basis. According to the embodiment of the application, aiming at the read-only bidding documents, a frame selection marking technology point marking function is added in the bid evaluation system, so that a comment expert can carry out frame selection, note or annotation on the technology point marking of the bidding documents during bid evaluation, and a scoring basis is provided. Specifically, the evaluation method is applied to an evaluation system and comprises the following steps:
s11, receiving an instruction for opening a bid file, and displaying a to-be-reviewed document and an editable layer positioned on the upper layer of the to-be-reviewed document;
wherein the document to be reviewed is a PDF file or other non-editable read-only file. The editable layer is a transparent layer and is positioned above the to-be-reviewed document, corresponds to the to-be-reviewed document and does not affect the display of the to-be-reviewed document, and in the specific implementation, the editable layer can be added to the upper layer of the to-be-reviewed document by a JS tool for a review expert to perform box selection marking on the upper layer. In addition, a box selection mark toolbar is added in the bid evaluation system through a JS tool, and the color and the painting brush of the box selection mark can be selected through the toolbar of the bidding system.
S12, receiving each frame selection mark executed on the editable layer, and determining the coordinate position of each frame selection mark on the editable layer;
in the process of the review, the review expert can use the input device to carry out box selection marks on the content (characters or paragraphs) meeting the review standard of the review item in the review document while viewing the quasi-review document, wherein the box selection marks support shape box selection notes, character selection notes, document underline notes and remark text filling, as shown in fig. 2-5. The box selection mark can be regarded as a basis for scoring the review item by a review expert, and the input device can be a mouse or a touch pen. The box marking is performed on an editable layer positioned on the upper layer of the document to be reviewed, and the editable layer is not perceived by a review expert because the editable layer corresponds to the document to be reviewed.
S13, determining contents corresponding to each box selection mark in the to-be-reviewed document based on the coordinate positions of each box selection mark on the editable layer;
s14, receiving the scores of the review experts on each review item;
the step sequence of S12, S13, S14 is determined by the execution sequence of the scoring operation and the frame selection marking operation of the panelist, and for each of the panelists, the scoring may be performed first, then the frame selection marking may be performed, or the frame selection marking may be performed first, then the scoring may be performed; the scoring may also be performed after all of the box-selection labeling is performed for each review item, or after all of the scoring is performed for each review item. The order of execution of the steps is not limited, and various orders of execution are within the scope of the present application.
S15, storing each frame selection mark in a data structure, and carrying out association storage with the corresponding content and score;
wherein the stored content includes the content of the framing mark itself and the coordinate position of the framing mark. The stored frame selection marks are all in a mark list, the mark list is also positioned in a toolbar of the marking system, and the position of the content corresponding to the mark can be quickly and directly selected or jumped through the mark list.
The frame selection mark and the corresponding content are stored in an associated mode, and matching of the content when the comment system calls and views again can be guaranteed. The re-calling can be that the review expert looks up the bid files marked by the boxes again, and the review expert can quickly jump, edit and delete among the scoring points selected by different boxes; the supervision department can also call the historical scoring and the box selection marking of the review expert so as to carry out auditing supervision on the review expert.
To facilitate analysis of content corresponding to the box selection marker, in another embodiment, the method further includes:
s16, converting the content corresponding to the frame selection mark into an image, performing character recognition through OCR (optical character recognition ) technology, and converting the content into an editable character format for backup.
The content after backup can be edited to modify and edit the content of the technical score point and to compare learning.
In the actual evaluation process, an evaluation expert opens a bidding document of a bidder through an evaluation system to evaluate the quasi-evaluation document; the review expert can use the input device to carry out box selection marking on the characters (paragraphs) meeting the review standard of the review item in the review process while viewing the to-be-reviewed document, and a scoring basis is given. The bid evaluation system stores the frame selection mark of the panel expert in a data structure and associates and stores the frame selection mark with the corresponding text (paragraph) of the document to be evaluated. The scoring of the review expert can be carried out while the scoring basis is given, and the quasi-review document with the box selection mark can be checked again after all the box selection marks of the review items are finished, so that the scoring is carried out based on the scoring basis. When the comment expert views the annotated bid file document again, the previous comments can be viewed and edited, and meanwhile, the supervision department can call the history scoring and comment information of the comment expert.
When an expert evaluates a mark (mainly a subjective technical mark part), the marking function of the mark is assigned through a frame selection marking technology, so that the marking basis of the expert on the subjective evaluation item can be definitely determined, meanwhile, the expert is supported to rapidly jump, edit and delete between marking points selected by different frames, meanwhile, the marking points selected by the expert are subjected to text extraction and comparison, the marking point contents of different units are transversely compared, a learning mechanism is established, and a data basis is made for the marking evaluation action of the big data analysis expert.
As shown in fig. 6, the embodiment of the present application further provides a label evaluation system based on a label marking point by a frame selection technique, where the system includes:
the display module 101 is used for receiving an instruction for opening the bidding document, displaying the to-be-reviewed document and an editable layer positioned on the upper layer of the to-be-reviewed document;
wherein the document to be reviewed is a PDF file or other non-editable read-only file. The editable layer is a transparent layer and is positioned above the to-be-reviewed document, corresponds to the to-be-reviewed document and does not affect the display of the to-be-reviewed document, and in the specific implementation, the editable layer can be added to the upper layer of the to-be-reviewed document by a JS tool for a review expert to perform box selection marking on the upper layer. The box selection mark supports shape box selection notes, text selection notes, document underline notes and remark text filling.
A box selection mark module 102, configured to receive each box selection mark executed on the editable layer, and determine a coordinate position of each box selection mark on the editable layer;
during the process of the review, the review expert can view the quasi-review document and simultaneously can use the input device to carry out box selection marking on the content (text or paragraph) meeting the review standard in the quasi-review document, wherein the box selection marking can be regarded as the basis for the review expert to score the review item, and the input device can be a mouse or a touch pen. The box marking is performed on an editable layer positioned on the upper layer of the document to be reviewed, and the editable layer is not perceived by a review expert because the editable layer corresponds to the document to be reviewed.
An annotating content module 103 for determining content corresponding to each box selection mark in the to-be-reviewed document based on the coordinate position of each box selection mark on the editable layer;
a scoring module 104, configured to score each review item in the to-be-reviewed document;
a first storage module 105, configured to store each box selection marker in a data structure, and store the box selection markers in association with corresponding content and scores thereof;
wherein the stored content includes the content of the framing mark itself and the coordinate position of the framing mark. The saved box selection marks are all in the mark list, and the positions of the contents of the corresponding marks can be quickly and directly selected or jumped to through the mark list.
And a calling module 106 for calling the bidding document with the box selection marker.
The frame selection mark and the corresponding content are stored in an associated mode, and matching of the content when the comment system calls and views again can be guaranteed. The re-calling can be that the review expert views the bid file with the box selection mark again, or that the supervision department calls the history scoring and the box selection mark of the review expert so as to carry out audit supervision on the review expert.
To facilitate analysis of content corresponding to the box selection marker, in another embodiment, as shown in fig. 7, the rating marker system further includes:
the second storage module 107 is configured to convert the content corresponding to the frame selection mark into an image, perform text recognition by using OCR technology, and convert the content into an editable text format for backup.
In the embodiment, the function of assigning points by a frame selection marking technology is added in the existing marking system, a marking expert can perform frame selection marking on a to-be-inspected file in the marking system, the marking basis of the expert on the subjective evaluation item can be definitely clarified, meanwhile, the expert is supported to rapidly jump, edit and delete between the marking points selected by different frames, meanwhile, text extraction and comparison are performed on the marking point parts selected by the expert frame, the marking point contents of different units are transversely compared, a learning mechanism is established, and a data basis is made for the large data analysis expert marking action.
As shown in fig. 8, in an embodiment of the present application, a supervision method based on a marking point of a box selection marking technology is provided, where the method includes:
s21, calling the bid files scored by the expert to be supervised; the expert to be supervised scores the bidding documents by using the marking system based on the marking points of the frame selection marking technology in the embodiment;
s22, judging whether a frame selection mark exists in the bidding document; if yes, executing S23, and if not, determining that the review quality of the expert to be supervised is low;
s23, extracting content of the frame selection mark, and judging whether the content is related to the scoring item;
the correlation between the extracted frame selection mark content and the scoring item can be judged manually by a supervisory personnel, and the correlation between the content of the frame selection mark and the scoring item can be analyzed by a transverse comparison and longitudinal analysis method.
The transverse alignment means: in the bid evaluation method, the same scoring item is the same item as a 'question' for bidders, answers are different, but main ideas are more or less the same, and the frame selection content according to the scoring points of the scoring items can have the same standard when the same comment is evaluated, so that the bidding documents of the transverse multiple bidders are compared, whether similar descriptions exist in the content can be judged, meanwhile, the descriptions (whether the descriptions are inconsistent) of the content can be compared, and clues are provided for the bibliographic.
Longitudinal analysis refers to: and (3) performing type learning on the content of the same bid evaluation method through data accumulation, and learning response characters corresponding to scoring points in the historical bid file (the complete file is not the frame content), and comparing the frame content with the historical file to see whether the request of the bid evaluation content is met.
S24, determining the review quality of the expert to be supervised based on the relevance of the content and the scoring item;
specifically, in the evaluation field, when the matching degree of the frame selection content of the evaluation expert and the evaluation score point of the evaluation item is higher, the grading is higher, or when the matching degree of the frame selection content and the evaluation score point of the evaluation item is lower, or the frame selection is not performed, the grading is also lower or 0, and the evaluation quality is considered to be higher. When contrast occurs, the quality of the review is considered lower.
And S25, after the review is finished, the evaluation habit or style of the review person is imaged by combining the expert historical evaluation data with the review content data (comprising the evaluation items, the content of the box selection mark and the scores) through the box selection content, so that data is provided for big data analysis.
According to the supervision method based on the frame selection marking technology scoring points, the frame selection marking technology scoring points radically prevent experts from scoring the documents without looking at the documents or reasonably avoiding the acquiescing scoring conditions among the experts by means of subjective impressions or scoring modes of 'brains', and the extracted frame selection content is insubstantial content or is an important investigation point for expert scoring behavior analysis, so that basis is provided for supervision departments to analyze the professional level of the experts and whether illegal reviews exist.
As shown in fig. 9, in an embodiment of the present application, there is provided a supervision system based on a marking point by a box selection technique, the system including:
a file acquisition module 201, configured to acquire a bid file scored by an expert to be supervised; the expert to be supervised scores the bidding documents by using the marking system based on the marking points of the frame selection marking technology in the embodiment;
a box selection mark judging module 202 for judging whether the bid file has a box selection mark; if yes, the box selection content judging module 203 is called, and if not, the review quality judging module 205 is called;
the frame selection content judging module 203 is configured to extract content of the frame selection mark and judge whether the content is related to the scoring item;
the correlation between the extracted frame selection mark content and the scoring item can be judged manually by a supervisory personnel, and the correlation between the content of the frame selection mark and the scoring item can be analyzed by a transverse comparison and longitudinal analysis method.
The transverse alignment means: in the bid evaluation method, the same scoring item is the same item as a 'question' for bidders, answers are different, but main ideas are more or less the same, and the frame selection content according to the scoring points of the scoring items can have the same standard when the same comment is evaluated, so that the bidding documents of the transverse multiple bidders are compared, whether similar descriptions exist in the content can be judged, meanwhile, the descriptions (whether the descriptions are inconsistent) of the content can be compared, and clues are provided for the bibliographic.
Longitudinal analysis refers to: and (3) performing type learning on the content of the same bid evaluation method through data accumulation, and learning response characters corresponding to scoring points in the historical bid file (the complete file is not the frame content), and comparing the frame content with the historical file to see whether the request of the bid evaluation content is met.
The review quality judging module 204 is configured to determine the review quality of the expert to be supervised based on the relevance between the content and the scoring item determined by the frame selection content judging module; or when the box selection mark judging module judges that the box selection mark does not exist in the bidding document, determining that the review quality of the expert to be supervised is low;
specifically, in the evaluation field, when the matching degree of the frame selection content of the evaluation expert and the evaluation score point of the evaluation item is higher, the grading is higher, or when the matching degree of the frame selection content and the evaluation score point of the evaluation item is lower, or the frame selection is not performed, the grading is also lower or 0, and the evaluation quality is considered to be higher. When contrast occurs, the quality of the review is considered lower.
And the portrait module 205 is used for providing data for big data analysis by carrying out portrait on the comment habit or style of the comment expert by combining the history comment data of the comment expert with the comment content data (comprising the comment item, the content of the frame selection mark and the score) through the frame selection content after the comment is finished.
According to the supervision system based on the frame selection marking technology scoring points, the frame selection marking technology scoring points radically prevent experts from scoring the documents without looking at the documents or reasonably avoiding the acquiescing scoring conditions among the experts by means of subjective impressions or scoring modes of 'brains', and the extracted frame selection content is insubstantial content or is an important investigation point for expert scoring behavior analysis, so that basis is provided for supervision departments to analyze the professional level of the experts and whether illegal reviews exist.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and not for limiting the same; although the application has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some or all of the technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit of the application.
Claims (10)
1. The marking method based on the frame selection marking technology marking point is characterized by comprising the following steps:
receiving an instruction for opening a bid file, and displaying a quasi-review document and an editable layer positioned on the upper layer of the quasi-review document; the to-be-reviewed document is a non-editable read-only document;
receiving each frame selection mark on the editable layer and determining the coordinate position of each frame selection mark on the editable layer; the box selection mark is a box selection mark of the review expert on the content meeting the review standard of the review item in the quasi-review document in the review process;
determining contents corresponding to each box selection mark in the to-be-reviewed document based on the coordinate positions of the box selection marks on the editable layer;
receiving the scores of the review experts on each review item;
and storing the box selection marks in a list, and carrying out association storage with the corresponding contents and scores.
2. The method for scoring points based on the box-selection-marking technique according to claim 1, wherein the box selection marking comprises: shape box selection notes, text selection notes, document underline notes and remark text filling.
3. The scoring method based on the frame selection marking technology scoring point according to claim 1, further comprising: and each box selection mark of the to-be-reviewed document and corresponding content thereof are called from the list.
4. The scoring method based on the frame selection marking technology scoring point of claim 3, further comprising: jump, edit or delete between different box selection markers.
5. The scoring method based on the frame selection marking technology scoring point according to claim 1, further comprising: and converting the content corresponding to the box selection mark in the document to be reviewed into an image, performing character recognition through an optical character recognition technology, and converting the content corresponding to the box selection mark into an editable character format backup.
6. The utility model provides a mark system is criticized to evaluation based on frame selection mark technique point which characterized in that includes:
the display module is used for receiving an instruction for opening the bidding document, displaying the to-be-reviewed document and an editable layer positioned on the upper layer of the to-be-reviewed document; the to-be-reviewed document is a non-editable read-only document;
a box selection mark module for receiving each box selection mark on the editable layer and determining the coordinate position of each box selection mark on the editable layer; the box selection mark is a box selection mark of the review expert on the content meeting the review standard of the review item in the quasi-review document in the review process;
the scoring module is used for scoring each review item in the to-be-reviewed document;
the annotating content module is used for determining the content corresponding to each box selection mark in the to-be-reviewed document based on the coordinate position of each box selection mark on the editable layer;
the first storage module is used for storing each frame selection mark in a list and carrying out association storage with the corresponding content;
and the calling module is used for calling the bidding document with the frame selection mark.
7. The scoring system based on the frame selection marking technique scoring point of claim 6, further comprising: and the second storage module is used for converting the content corresponding to the frame selection mark in the to-be-reviewed document into an image, performing character recognition through an optical character recognition technology, and converting the content corresponding to the frame selection mark into an editable character format for backup.
8. A supervision method of assigning points based on a frame selection marking technology is characterized by comprising the following steps:
invoking a bid file scored by an expert to be supervised; the expert to be supervised scores the bidding documents by using the bid evaluation system based on the marking points of the frame selection marking technology according to any one of claims 6 to 7;
judging whether a frame selection mark exists in the bidding document; if yes, extracting content corresponding to the box selection mark, judging whether the content is related to the scoring item, and determining the evaluation quality of the expert to be supervised based on the relativity of the content and the scoring item; if not, determining that the review quality of the expert to be supervised is low;
and carrying out image drawing on the comment habit or style of the expert to be supervised based on the comment quality and the comment content of the expert to be supervised, wherein the comment content comprises a scoring item, a frame selection marked content and a score.
9. The method of claim 1, wherein determining whether the content is associated with a scoring item comprises: analyzing whether the content of the frame selection mark has relevance with the scoring item or not by adopting a transverse comparison and longitudinal analysis method;
the lateral alignment includes: transversely comparing bidding documents of multiple bidders, and judging whether contents corresponding to the box selection marks are similar or not; the longitudinal analysis includes: and performing type learning on the content of the same bid evaluation method through data accumulation, learning response characters corresponding to the score points in the historical bid documents, comparing the content of the selection mark with the historical response characters, and judging whether the content meets the requirement of the bid evaluation content.
10. A supervisory system for assigning points based on a box selection marking technique, comprising:
the file acquisition module is used for retrieving the bid files scored by the expert to be supervised; the expert to be supervised scores the bidding documents by using the bid evaluation system based on the marking points of the frame selection marking technology according to any one of claims 6 to 7;
the frame selection mark judging module is used for judging whether the bidding document has a frame selection mark or not; if yes, calling a box selection content judging module, and if not, calling a review quality judging module;
the frame selection content judging module is used for extracting the content of the frame selection mark and judging whether the content is related to the scoring item when the frame selection mark judging module judges that the frame selection mark exists in the bidding document;
the evaluation quality judging module is used for determining the evaluation quality of the expert to be supervised based on the correlation degree of the content and the scoring item; or when the box selection mark judging module judges that the box selection mark does not exist in the bidding document, determining that the review quality of the expert to be supervised is low;
and the portrait module is used for carrying out portrait on the comment habit or style of the expert to be supervised based on the comment quality and the comment content of the expert to be supervised, which are determined by the comment quality judging module, wherein the comment content comprises a comment item, the content of a frame selection mark and a score.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310883156.8A CN116934256A (en) | 2023-07-18 | 2023-07-18 | Evaluation method, supervision method and system based on marking points of frame selection marking technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310883156.8A CN116934256A (en) | 2023-07-18 | 2023-07-18 | Evaluation method, supervision method and system based on marking points of frame selection marking technology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116934256A true CN116934256A (en) | 2023-10-24 |
Family
ID=88380004
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310883156.8A Pending CN116934256A (en) | 2023-07-18 | 2023-07-18 | Evaluation method, supervision method and system based on marking points of frame selection marking technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116934256A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117132247A (en) * | 2023-10-27 | 2023-11-28 | 鼎铉商用密码测评技术(深圳)有限公司 | Report auditing method, report auditing device, and readable storage medium |
| CN117808367A (en) * | 2024-01-02 | 2024-04-02 | 汉考国际教育科技(北京)有限公司 | Intelligent examination paper evaluation system for Chinese examination |
-
2023
- 2023-07-18 CN CN202310883156.8A patent/CN116934256A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117132247A (en) * | 2023-10-27 | 2023-11-28 | 鼎铉商用密码测评技术(深圳)有限公司 | Report auditing method, report auditing device, and readable storage medium |
| CN117808367A (en) * | 2024-01-02 | 2024-04-02 | 汉考国际教育科技(北京)有限公司 | Intelligent examination paper evaluation system for Chinese examination |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116934256A (en) | Evaluation method, supervision method and system based on marking points of frame selection marking technology | |
| CN111046784A (en) | Document layout analysis and identification method and device, electronic equipment and storage medium | |
| CN109543925B (en) | Risk prediction method and device based on machine learning, computer equipment and storage medium | |
| CN112307718B (en) | PDF full-automatic indexing system and method based on text features and grammar rules | |
| CN114168716A (en) | Deep learning-based automatic engineering cost extraction and analysis method and device | |
| WO2020151340A1 (en) | Target cell marking method and device, storage medium and terminal device | |
| AU2019325322B2 (en) | Text line image splitting with different font sizes | |
| CN116433052B (en) | Intelligent bidding information acquisition data analysis and evaluation system | |
| US20210334573A1 (en) | Text line normalization systems and methods | |
| CN113051404A (en) | Knowledge reasoning method, device and equipment based on tensor decomposition | |
| CN111369294B (en) | Software cost estimation method and device | |
| CN111444718A (en) | Insurance product demand document processing method and device and electronic equipment | |
| CN114239588A (en) | Article processing method and device, electronic equipment and medium | |
| CN112232564A (en) | Label processing device and method | |
| US8341170B2 (en) | Apparatus and method for visualizing technology change | |
| JP2004252881A (en) | Text data correction method | |
| CN117745398A (en) | Automatic formula calculation method and system for electronic bidding scoring standard | |
| CN111274156B (en) | Automatic identification method and device compatible with multi-frame pages | |
| Antonacopoulos et al. | The lifecycle of a digital historical document: structure and content | |
| CN115238662A (en) | Bidding file rapid editing method and system | |
| CN115602277A (en) | Medical electronic medical record labeling method, device, system and storage medium | |
| CN113111635A (en) | Report form comparison method and device | |
| CN111523615A (en) | Assembly line closed loop flow method for realizing vehicle appearance professional damage labeling | |
| CN113643408B (en) | Image generation method and device, computer readable storage medium and electronic device | |
| CN112541085B (en) | Method for structuring questionnaire, apparatus for structuring questionnaire, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |