Remembering
the Sentence.

Retrieval Augmented Generation for Historical Newspapers ↗ (2024) · finding

Section: Quantitative Evaluation. PDF page unverified.

The stored page-and-section lookup returns 3 candidate paragraph(s). The anchor identifies one paragraph directly.

where N is the number of features in the vector representation, P is all the retrieved paragraphs, r is the answer and 𝑝 𝑖 is the value of the vector representation of 𝑝 at index 𝑖. 1 shows all the results from the different methods: sparse retrieval (BM25), dense retrieval with cosine similarity (E5), the combination of dense retrieval with a cross-encoder model (E5+Cohere), and the reranker after injecting information (E5+Cohere+NER). As expected, dense retrieval produces a superior result compared to sparse retrieval thanks to its ability to capture the semantic and context meaning of a sentence. On the other hand, BM25 is highly dependent on vocabulary matching. In addition, we can see that with the normal reranking method (E5+Cohere), the system can increase its performance by nearly 0.1 points for NDCG. However, injecting NER seems to worsen the final result. This could be due in large part to the fact that using NER could cause misleading information in certain situations where named entities do not directly contribute to the key features of a sentence. Another factor might come from the errors of the NER extraction model itself. Regarding the results of the answer generation module with E5 retrieval shown in Table 2 , in the three languages, Finnish shows the lowest results among the three languages. This might show the inability of LLaMA3 to handle some languages with fewer resources. Meanwhile, the difference created by the BERTscore between English and French is significantly higher than that of other metrics where the gaps are unnoticeable. We computed the linear correlations between these scores to analyze how well each metric relates to each other across languages. All three languages share a similar result, where the BERTscore and cosine similarity show the highest correlation with each other, greater than 0.7 between each pair. The reason behind this might involve the fact that both scores work on dense embedding vectors, making them similar. In addition to this, the quality index is slightly more correlated with other scores compared to the LLM score in the English data (Table 3 ). However, the reverse trend can be observed for French and Finnish, in Tables 4 and 5 , respectively. The quality index can be seen to have a very low correlation in these languages. In contrast, the LLM score has a much higher correlation, with roughly 0.8 in relation to the BERTscore and cosine similarity in the French data.

Stored parser text; segmentation and table errors are preserved.

Claim: tran2024_104b_claim_5011d54ce58b
Paragraph: tran2024_104b_5.2
Vault note: literature/Retrieval Augmented Generation for Historical Newspapers.md

Retrieval Augmented Generation for Historical Newspapers ↗ (2024) · methodology

Section: Quantitative Evaluation. PDF page unverified.

The stored page-and-section lookup returns 3 candidate paragraph(s). The anchor identifies one paragraph directly.

4.1.1 Setup. For this evaluation, we used the French and Finnish subsets of Miracl [20] , a multilingual dataset used for the evaluation of information retrieval. In addition to French and Finnish, we added English to evaluate the answer generation module. Since this dataset is already split into passages, we directly indexed each chunk in the database. In addition, Miracl provides a title as metadata for each of these passages, which we indexed and used for title retrieval. Lastly, we set the temperature of LLaMA3 to 0.3 for the final answer generation module.

Stored parser text; segmentation and table errors are preserved.

Claim: tran2024_104b_claim_6f0803d4003b
Paragraph: tran2024_104b_5.0
Vault note: literature/Retrieval Augmented Generation for Historical Newspapers.md

Retrieval Augmented Generation for Historical Newspapers ↗ (2024) · methodology

Section: Methodology. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

Four our RAG system pipeline we first create a database using the small multilingual model E5 4 to embed all documents into vector representations. We also add metadata to the text during this step, such as the article's title. We then index the E5 embeddings in the open-source vector database Chroma foot_4 . The document's similarity calculation is calculated using cosine similarity and the retrieval process will employ maximal marginal relevance (MMR) [2] . We use the same procedure on the title or summary of each article to create a second database which is in the role of a semantic router in our pipeline. After the creation of the necessary databases, the main system make use of these and comprises of four main components: a query router where we redirect the behavior of the system based on the user's question, a base retrieval model where some of the best documents related to the question are retrieved and they will be ranked using a rerank module to filter out some irrelevant ones. And lastly, all of these retrieved documents are aggregated and forward to the prompt of an LLM to generate the final answer given a user's query. The base system uses a query routing procedure to adapt the system to whether or not to go directly to a web search if none of the retrieved documents is relevant to the query. We provide an article-level retrieval mode based on the article database created earlier that can assess whether the system can continue the normal retrieving path. In other words, we first use a retrieval module to retrieve text at the title or summary level, which helps to determine the path to take. The system goes directly to the web search module if we cannot retrieve an article with a similarity score that exceeds a certain threshold. Otherwise, it continues on the traditional path. Then, we retrieve the documents based on the article title we have retrieved before employing reranking to select some of the best candidates to be further forwarded to the prompt. In this step, we set a threshold to remove irrelevant information. The reranker consists of two different paths presenting two ways of calculating the similarity score for a query-document pair; one uses Cohere Multilingual Reranker 6 which outputs a Cohere score; the other works on the named entities and generates a NER score, which is not affected by the OCR errors, thus injecting more stable information into the module. First, we extract the named entities with hmBERT foot_6 [14] and inject this information into our module. The task involves first creating a string that encompasses the named entities:

Stored parser text; segmentation and table errors are preserved.

Claim: tran2024_104b_claim_a0d3a32d50de
Paragraph: tran2024_104b_3.0
Vault note: literature/Retrieval Augmented Generation for Historical Newspapers.md

Retrieval Augmented Generation for Historical Newspapers ↗ (2024) · limitation

Section: Qualitative Evaluation. PDF page unverified.

The stored page-and-section lookup returns 2 candidate paragraph(s). The anchor identifies one paragraph directly.

In Figure 1 , we show an example in which we deliberately damage the query by modifying the name of the exposition from Caros Sandoval to Carlos Sandov. This process is to experiment with whether the system can produce robust results despite OCR errors or user misspellings when querying. It can be seen that, though the Cohere model is obtaining a good answer, its score is relatively low for Figure 1 . However, with the injection of NER into the model, the score is much higher, indicating that more precise information can boost the system's final overall performance. In other cases, Cohere model could still show some robust performances despite the noises we have added and provide high scores without the help of NER information. The final result (Answer) of LLaMA3 in Figure 2 can be reasonable given the data. However, it can be seen that the answer is returned in different languages, whereas the queries are made only in French. This could create inconsistent quality and should be addressed in the future.

Stored parser text; segmentation and table errors are preserved.

Claim: tran2024_104b_claim_e80d9200c132
Paragraph: tran2024_104b_6.2
Vault note: literature/Retrieval Augmented Generation for Historical Newspapers.md

A Library Perspective on Supervised Text Processing in Digital Libraries: An Investigation in the Biomedical Domain ↗ (2024) · finding

Section: RQ3: Data Labeling. PDF page unverified.

The stored page-and-section lookup returns 11 candidate paragraph(s). The anchor identifies one paragraph directly.

conduct a new search here). The results are shown in Table 7 . For CDR, the different labeling methods seem to perform quite well. The distantly supervised labeling method achieved a similar, but slightly decreased, F1 score compared to the expert labeling. This suggests that the Comparative Toxigenomics Database [7] knowledge base used for distant labeling is reliable and provides high-quality data for this task. On ChemProtC, the difference between the labeling methods and expert labeling was more noticeable. The models' performance dropped significantly when trained on noisily generated data, which could also be a cause of our re-grouping of the data. Overall, GPT-4o mode performed the best across most tasks and labeling methods. It consistently outperformed other models with training on BERT models, demonstrating its robustness and effectiveness even when trained on noisily labeled data. This highlights the potential of advanced language models to handle noisy data and achieve high performance without the need for perfect labeling. In brief, LLMs labeled the training data sufficiently well for our purposes and came with acceptable costs in the end.

Stored parser text; segmentation and table errors are preserved.

Claim: kroll2024_3ce7_claim_bfd1622117de
Paragraph: kroll2024_3ce7_10.13
Vault note: literature/A Library Perspective on Supervised Text Processing in Digital Libraries An Investigation in the Biomedical Domain.md

A Library Perspective on Supervised Text Processing in Digital Libraries: An Investigation in the Biomedical Domain ↗ (2024) · finding

Section: Introduction. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

From a natural language processing perspective, several works exist that propose advanced methods for extracting named entities and their semantic relationships or classifying texts in general; see [8, 36, 42] to name just a few. When implementing extraction workflows in a digital library, questions beyond a benchmarkcentric evaluation arise, e.g., about trade-offs between costs and quality. Regarding training and application costs, a cheaper model might be favored over a complex model, achieving higher accuracy. In brief, this work is written from the perspective of a digital library. It differs from existing work in that we 1) compare the trade-off between extraction quality and costs, 2) dive into designing complete end-to-end systems in contrast to benchmark-centric evaluations, and 3) approach how we can generate/retrieve training data.

Stored parser text; segmentation and table errors are preserved.

Claim: kroll2024_3ce7_claim_82620d172f4a
Paragraph: kroll2024_3ce7_0.2
Vault note: literature/A Library Perspective on Supervised Text Processing in Digital Libraries An Investigation in the Biomedical Domain.md

A Library Perspective on Supervised Text Processing in Digital Libraries: An Investigation in the Biomedical Domain ↗ (2024) · definition

Section: RQ3: Data Labeling. PDF page unverified.

The stored page-and-section lookup returns 11 candidate paragraph(s). The anchor identifies one paragraph directly.

Distantly-Supervised Labeling. The related work section describes weak supervision as a possible remedy [34] . The central idea is that external knowledge is used to label sentences. If some sentence includes two entities and the entities have a relationship within the given knowledge base, then we implicitly assume that the sentence also expresses this relationship. In brief, distant supervision allows fast and large data set generation. That is why we investigate it here. However, it requires external knowledge bases that include the relations someone is interested in, and it might also be limited in precision, as sentences may be labeled in a noisy fashion.

Stored parser text; segmentation and table errors are preserved.

Claim: kroll2024_3ce7_claim_598a7597351d
Paragraph: kroll2024_3ce7_10.1
Vault note: literature/A Library Perspective on Supervised Text Processing in Digital Libraries An Investigation in the Biomedical Domain.md

A Library Perspective on Supervised Text Processing in Digital Libraries: An Investigation in the Biomedical Domain ↗ (2024) · definition

Section: Related Work. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

Named Entity Recognition and Disambiguation. The first step in extracting semantic relationships between named entities is to identify these entities in the text. Usually, recognition tools recognize entities within texts, and subsequent disambiguation tools assign those text spans to precise identifiers to disambiguate them. A comprehensive overview of possible detection methods is given in [42] . A plethora of different tools exist to identify biomedical entities in texts, e.g., PubTator [40] , GNormPlus [41] , GNorm2 [38] , TaggerOne [27] , and many more. While entity detection is a relevant topic in digital libraries, our work focuses on relation extraction between them and thus assumes that the entities are given.

Stored parser text; segmentation and table errors are preserved.

Claim: kroll2024_3ce7_claim_f9b789fb7fce
Paragraph: kroll2024_3ce7_1.0
Vault note: literature/A Library Perspective on Supervised Text Processing in Digital Libraries An Investigation in the Biomedical Domain.md

Scaffolding Inquiry-Oriented Web Search using LLM-based Question Generation ↗ (2025) · finding

Section: F. Results. PDF page unverified.

The stored page-and-section lookup returns 9 candidate paragraph(s). The anchor identifies one paragraph directly.

TABLE III: Means and standard deviations of each metric for the online user study. Asterisks (*) indicate metrics for which significant differences across UI conditions were found in the variance analysis. Superscripts P, A, E, and S indicate statistically significant differences in pairwise comparisons with PLAIN, ADJUNCT, EXPECTED, and SCAFFOLDING conditions, respectively. UI Condition Metric PLAIN ADJUNCT EXPECTED SCAFFOLDING Search behavior metrics Task duration (s) 745.8 (476.8) 833.4 (586.1) 869.0 (636.9) 799.0 (480.8) Total SERP duration (s)* 145.4 (120.8) E 264.4 (240.1) 291.3 (256.7) P 285.3 (297.4) Avg. SERP duration per query (s)* 59.3 (95.4) AS 95.3 (66.8) P 96.0 (95.4) 104.8 (86.5) P Number of queries 3.07 (2.42) 3.09 (2.49) 3.81 (2.42) 3.08 (2.43) Number of clickthroughs 6.17 (4.79) 5.94 (3.96) 5.92 (3.33) 6.67 (4.07) Task outcome Semantic relevance between prompts and opinion* -0.677 (0.060) ES 0.710 (0.038) A 0.723 (0.052) A Evaluation of prompts Relevance to theme* -2.59 (2.05) ES 3.04 (2.16) A 3.15 (2.07) A Importance for the theme* -2.46 (2.02) ES 2.96 (2.12) A 3.09 (2.02) A Interestingness* -2.40 (1.99) S 2.88 (2.06) 2.98 (1.95) A Ease of answering -2.59 (2.06) 2.68 (2.03) 2.80 (1.88) Usefulness for deepening understanding* -2.39 (1.96) 2.84 (2.03) 2.94 (1.90) Usefulness for gaining new perspectives* -2.22 (1.85) S 2.65 (1.95) 2.83 (1.90) A Usefulness in organizing information/opinions* -2.31 (1.90) ES 2.81 (2.05) A 2.82 (1.85) A

Stored parser text; segmentation and table errors are preserved.

Claim: yamamoto2025_1935_claim_cfbab3e58c02
Paragraph: yamamoto2025_1935_21.6
Vault note: literature/Scaffolding Inquiry-Oriented Web Search using LLM-based Question Generation.md

Scaffolding Inquiry-Oriented Web Search using LLM-based Question Generation ↗ (2025) · methodology

Section: Limitations and future work. PDF page unverified.

The stored page-and-section lookup returns 3 candidate paragraph(s). The anchor identifies one paragraph directly.

The third limitation involves prompt generation. To address the challenge of not being able to directly observe users' thoughts during a web search, the proposed method employs an LLM to predict opinions that a hypothetical learner might form when reading a page and then generates prompts accordingly. However, because the opinions users form may vary depending on their demographic attributes and prior knowledge, the relevance and effectiveness of the prompts may also differ among users. Thus, the personalization of prompt generation should be considered in future studies.

Stored parser text; segmentation and table errors are preserved.

Claim: yamamoto2025_1935_claim_041b1c3e9e39
Paragraph: yamamoto2025_1935_27.2
Vault note: literature/Scaffolding Inquiry-Oriented Web Search using LLM-based Question Generation.md

Scaffolding Inquiry-Oriented Web Search using LLM-based Question Generation ↗ (2025) · finding

Section: VI. DISCUSSION. PDF page unverified.

The stored page-and-section lookup returns 5 candidate paragraph(s). The anchor identifies one paragraph directly.

For RQ1, the results of the online study indicate that participants under the SCAFFOLDING condition spent more time per query on the SERP than those under the PLAIN condition. A similar tendency was observed for the ADJUNCT condition. In contrast, although no significant difference was observed in the SERP duration per query under the EXPECTED condition, the total SERP duration was longer than that under the PLAIN condition. These results suggest that under the SCAFFOLDING, EXPECTED, and ADJUNCT conditions, participants may have spent more time on the SERP screen because they reflected on the asked questions or scrutinized the search results prompted by the displayed questions.

Stored parser text; segmentation and table errors are preserved.

Claim: yamamoto2025_1935_claim_06e454365208
Paragraph: yamamoto2025_1935_25.0
Vault note: literature/Scaffolding Inquiry-Oriented Web Search using LLM-based Question Generation.md

OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors ↗ (2022) · finding

Section: Data Records. Stored geometric page 8 (docling).

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

• room_no: room ID specified as "1" (left room in Fig. 1 ) or "2" (right room in Fig. 1 ).

Stored parser text; segmentation and table errors are preserved.

Claim: bocus2022_ce7f_claim_069388bcbc50
Paragraph: bocus2022_ce7f_10.23
Vault note: literature/OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors.md

OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors ↗ (2022) · methodology

Section: exp044-exp048. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

Device-free dynamic localization. CSI transmitter (NUC3) and CSI receiver (NUC2) are placed side by side and the target moves along a short straight path for each experiment number.

Stored parser text; segmentation and table errors are preserved.

Claim: bocus2022_ce7f_claim_04944cb19a12
Paragraph: bocus2022_ce7f_6.0
Vault note: literature/OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors.md

OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors ↗ (2022) · methodology

Section: Data Records. Stored geometric page 2 (docling).

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

Considering the crowd counting experiment, Fig. 6 shows the first path power level (in dBm) for the two UWB systems between a given pair of nodes in each case. The first path power level (fp_pow_dbm) has been computed using the formula given in the DW1000 manual 44 . As can be observed, the first path power level increases gradually as each person was moving out of the monitoring area. This is an expected behaviour since the LoS signal becomes less and less obstructed. By using the fp_pow_dbm parameter together with other parameters such as overall received UWB signal power level (rx_pow_dbm), UWB CIR data and WiFi CSI data, the number of people in a given environment can be inferred through the use of artificial intelligence algorithms.

Stored parser text; segmentation and table errors are preserved.

Claim: bocus2022_ce7f_claim_4ea99acc4119
Paragraph: bocus2022_ce7f_10.33
Vault note: literature/OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors.md

Device-Free Crowd Size Estimation Using Wireless Sensing on Subway Platforms ↗ (2024) · methodology

Section: •. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

We demonstrate the use of two separately trained polynomial regression models, when a rail vehicle is present and absent, in order to perform crowd size estimation using the change in the RSSI between wireless sensor nodes in the different environment states.

Stored parser text; segmentation and table errors are preserved.

Claim: janssens2024_2f53_claim_054a1907ff89
Paragraph: janssens2024_2f53_1.0
Vault note: literature/Device-Free Crowd Size Estimation Using Wireless Sensing on Subway Platforms.md

Device-Free Crowd Size Estimation Using Wireless Sensing on Subway Platforms ↗ (2024) · finding

Section: Crowd Size Estimation. PDF page unverified.

The stored page-and-section lookup returns 15 candidate paragraph(s). The anchor identifies one paragraph directly.

When evaluating the crowd size estimation with no vehicle present, a median error of 2.769 people is observed, along with an MAE of 3.342 people and an RMSE of 4.211. The CDF plot is shown in Figure 8 as a green dash-dotted line. When evaluating the crowd size estimation when a vehicle is present, a median error of 3.304 people is observed, along with an MAE of 4.478 people and an RMSE of 6.326. The CDF plot is shown in Figure 8 as an orange dashed line.

Stored parser text; segmentation and table errors are preserved.

Claim: janssens2024_2f53_claim_07adaab5862f
Paragraph: janssens2024_2f53_9.7
Vault note: literature/Device-Free Crowd Size Estimation Using Wireless Sensing on Subway Platforms.md

Device-Free Crowd Size Estimation Using Wireless Sensing on Subway Platforms ↗ (2024) · finding

Section: Combined Model. PDF page unverified.

The stored page-and-section lookup returns 6 candidate paragraph(s). The anchor identifies one paragraph directly.

When we look at more statistical metrics, we obtain a median error of 2.856 people, an MAE of 3.567 people, and an RMSE of 4.706. These numbers are a clear improvement over the use of a single polynomial regression model trained on both states, which results in a median error of 4.617 people, an MAE of 6.192 people, and an RMSE of 8.250.

Stored parser text; segmentation and table errors are preserved.

Claim: janssens2024_2f53_claim_09a6e0345c6f
Paragraph: janssens2024_2f53_10.1
Vault note: literature/Device-Free Crowd Size Estimation Using Wireless Sensing on Subway Platforms.md

Integrating AI into Library Systems: A Perspective on Applications and Challenges ↗ (2024) · finding

Section: Optical Character Recognition. PDF page unverified.

The stored page-and-section lookup returns 13 candidate paragraph(s). The anchor identifies one paragraph directly.

Jiang et al. [19] created large-scale parallel datasets of OCR'd text and human proofread counterparts sourced from Project Gutenberg and Hathitrust Digital Library, containing over 19,000 works in six domains: fiction, social science, agriculture, world war history, medicine, and business. With this benchmark dataset, Jiang [18] evaluated domain classification of text with OCR errors. The BERT models fine-tuned with their dataset show better encoding stability and resilience to OCR noise in classification tasks than models pretrained on born-digital texts. To evaluate the impact of OCR noise, Jiang et al. [20] encoded OCR'd and human-corrected versions of book chapters with pre-trained and fine-tuned BERT models and then compared them for similarities and quality of encodings. The authors' evaluation showed that BERT embeddings can be resilient to OCR errors when encoding chapter-level content with high NDCG scores. When encoding word and sentence level content, OCR errors can introduce erroneous tokens and disrupt the coherence of sentences.

Stored parser text; segmentation and table errors are preserved.

Claim: tai2024_5a71_claim_151f5dfa0b31
Paragraph: tai2024_5a71_8.8
Vault note: literature/Integrating AI into Library Systems A Perspective on Applications and Challenges.md

Integrating AI into Library Systems: A Perspective on Applications and Challenges ↗ (2024) · finding

Section: Applications and Research Directions. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

In this section, we explore RQ2 ("How is AI currently being researched and applied in libraries?") and RQ3 ("What are the limitations and future research directions of different AI technologies as they relate to libraries?"). This section consists of a detailed investigation of the latest research on practical applications of artificial intelligence in libraries. Through our review, we identified three major areas of interest: recommendation systems, information and resource retrieval, and optical character recognition. In each subsection, we outline different papers, emphasizing the technologies utilized, the limitations of the research, and directions for future studies. Table 1 shows the papers we reviewed for this section, categorized by the three major areas of interest and sub-areas of research.

Stored parser text; segmentation and table errors are preserved.

Claim: tai2024_5a71_claim_0906493ec965
Paragraph: tai2024_5a71_3.0
Vault note: literature/Integrating AI into Library Systems A Perspective on Applications and Challenges.md

Integrating AI into Library Systems: A Perspective on Applications and Challenges ↗ (2024) · finding

Section: Chatbots.. PDF page unverified.

The stored page-and-section lookup returns 2 candidate paragraph(s). The anchor identifies one paragraph directly.

Traditionally, reference services are provided through face-toface conversations. However, during times of congestion, nonoperational hours, and for users who want to access resources remotely [2] , AI-operated reference services could be particularly useful. Chatbots offer readily accessible information and personalized user assistance, ideally suited for library reference services. Many researchers have found ChatGPT particularly useful to help patrons navigate through the library's reserve of information and resources. Hall and McKee [13] and Lo [25] also observed extensive opportunities for libraries to use prompt engineering with Chat-GPT in tasks like summarizing content and developing curricula and rubrics. With effective prompting of novel language models, librarians can offer personalized assistance to patrons, helping with their research efforts and improving users' information literacy skills. Adetayo [2] argued for the integration of BingChat into library websites and catalogs to reshape existing digital reference services. Although BingChat and other industry chatbots are extremely resourceful, they are not trained specifically for libraries, so they may not be able to answer questions pertinent to specific library information and their unique resources.

Stored parser text; segmentation and table errors are preserved.

Claim: tai2024_5a71_claim_08b63849aa90
Paragraph: tai2024_5a71_6.1
Vault note: literature/Integrating AI into Library Systems A Perspective on Applications and Challenges.md

Learning from LLM Disagreement in Retrieval Evaluation ↗ (2025) · methodology

Section: D. LLM Filtering. PDF page unverified.

The stored page-and-section lookup returns 1 candidate paragraph(s). The anchor identifies one paragraph directly.

We use two locally hosted LLMs to assign a binary relevance label (Relevant or Non-Relevant) to each abstract based on whether or not it describes a meaningful contribution to the given SDG targets. The LLMs were given identical prompts that include instructions to return a binary label along with a brief justification. The process is described in Section IV-A.

Stored parser text; segmentation and table errors are preserved.

Claim: ingram2025_642d_claim_00d7b853d1ec
Paragraph: ingram2025_642d_9.0
Vault note: literature/Learning from LLM Disagreement in Retrieval Evaluation.md

Learning from LLM Disagreement in Retrieval Evaluation ↗ (2025) · methodology

Section: D. Learnability of Filtering Behavior. PDF page unverified.

The stored page-and-section lookup returns 2 candidate paragraph(s). The anchor identifies one paragraph directly.

Fig. 5 confirms that all three classifiers outperform chance. The above-baseline performance indicates that disagreement is non-random and tied to consistent lexical differences, though it does not imply that either model applies a single coherent or interpretable criterion. Further analysis, such as feature inspection or semantic embedding comparisons, would be required to characterize the specific lexical or conceptual criteria each model implicitly applies.

Stored parser text; segmentation and table errors are preserved.

Claim: ingram2025_642d_claim_0e0a9a4cd3bf
Paragraph: ingram2025_642d_19.2
Vault note: literature/Learning from LLM Disagreement in Retrieval Evaluation.md

Learning from LLM Disagreement in Retrieval Evaluation ↗ (2025) · methodology

Section: D. Learnability of Filtering Differences. PDF page unverified.

The stored page-and-section lookup returns 2 candidate paragraph(s). The anchor identifies one paragraph directly.

Using TF-IDF vectors as input features, we train a logistic regression classifier to predict which model labeled each document as relevant. We evaluate performance using five-fold cross-validation and report the area under the ROC curve (AUC) as the evaluation metric. All models are trained separately for each SDG to isolate domain-specific patterns and prevent topic leakage across goals.

Stored parser text; segmentation and table errors are preserved.

Claim: ingram2025_642d_claim_0631056d69b7
Paragraph: ingram2025_642d_14.1
Vault note: literature/Learning from LLM Disagreement in Retrieval Evaluation.md

Exploring Efficient Optimization Techniques in Online Retrieval-Augmented Generation Application ↗ (2024) · finding

Section: Generation. PDF page unverified.

The stored page-and-section lookup returns 6 candidate paragraph(s). The anchor identifies one paragraph directly.

After filtering, as all of the remaining documents are highly related to the query, the answer generation task is simplified to summarizing all given documents, which in our case should produce short introduction for each researcher. In this stage, we observed that LLMs are prone to confuse information among different documents, e.g. attributing researcher A's outcome to researcher B in the answer, which is totally unacceptable. This is also a typical kind of hallucination observed in previous work [12] . Since all documents have the corresponding researcher name, we first tried to add a preprocessing that sorts and integrates documents according to researcher identity before generation. The hallucination reduces but still exists. Considering that each integrated context now contains full information for one researcher, similar to filtering, we converted one LLM call that sequentially generates introduction for all 𝑁 researchers to 𝑁 parallel calls that each generates introduction for one researcher and merged the outputs at the end. This not only makes each task easier and eliminates the "confuse context" hallucination since they are isolated, but also accelerates the generation as explained before.

Stored parser text; segmentation and table errors are preserved.

Claim: zhang2024_aea3_claim_10eb06633adf
Paragraph: zhang2024_aea3_6.0
Vault note: literature/Exploring Efficient Optimization Techniques in Online Retrieval-Augmented Generation Application.md

Exploring Efficient Optimization Techniques in Online Retrieval-Augmented Generation Application ↗ (2024) · hypothesis

Section: Retrieval and Filter. PDF page unverified.

The stored page-and-section lookup returns 12 candidate paragraph(s). The anchor identifies one paragraph directly.

The advantage is that it requires no or seldom handcrafted supervised data, which saves a lot of human labor. The premise is that there must exist a LLM that can stably generate outputs of expected quality, otherwise the "student" won't have a good "textbook" to learn from. That's why GPT-4, one of the best-performing LLM, is widely used for this task. In our case, we experimented with the largest LLM among open-source families to choose the best teacher model, and also tried prompt engineering techniques including few-shot [3] and Chain-of-Thought (CoT) [24] prompting to further improve data quality. Few-shot prompting, also called in-context learning, provides several input and output examples of the task before asking the model to solve a new problem. This gives the model better understanding of a specific task. CoT prompts the LLM to break down question and output intermediate thinking step, which can make the final answer more accurate. In the case of relevance judgement, we have found that instructing LLM to additionally explain the reason instead of just giving the conclusion leads to better accuracy.

Stored parser text; segmentation and table errors are preserved.

Claim: zhang2024_aea3_claim_23d4516e921c
Paragraph: zhang2024_aea3_5.7
Vault note: literature/Exploring Efficient Optimization Techniques in Online Retrieval-Augmented Generation Application.md

Exploring Efficient Optimization Techniques in Online Retrieval-Augmented Generation Application ↗ (2024) · methodology

Section: Indexing. PDF page unverified.

The stored page-and-section lookup returns 2 candidate paragraph(s). The anchor identifies one paragraph directly.

These keywords covered most of the academic information from the proposal or abstract.

Stored parser text; segmentation and table errors are preserved.

Claim: zhang2024_aea3_claim_03030103ce77
Paragraph: zhang2024_aea3_4.2
Vault note: literature/Exploring Efficient Optimization Techniques in Online Retrieval-Augmented Generation Application.md

OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors ↗ (2022) · reading result

Paper-level provenance. This reading-pipeline claim has no paragraph anchor. Analyst locator: Background & Summary / Contributions. Open the publication to check it.

Claim: claim-0a293d9d
Vault note: literature/OPERAnet, a multimodal activity recognition dataset acquired from radio frequency and vision-based sensors.md