Samaya Research — AI Research for Finance
Samaya Research publishes AI research focused on financial reasoning, retrieval, and language model evaluation. We build open benchmarks and share findings with the research community.
Publications
- OpaqueToolsBench: Learning Nuances of Tool Behavior Through InteractionACM CAIS 2026 · Feb 16, 2026
A benchmark where agents must learn underspecified, opaque tools through interaction. ToolObserver iteratively rewrites tool documentation from execution feedback, outperforming costly auto-documentation methods.
- Lost in the Maze: Overcoming Context Limitations in Long-Horizon Agentic SearcharXiv 2025 · Oct 21, 2025
Long-horizon agentic search breaks down as context fills with noise. Slim separates search and browse tools and periodically summarizes the trajectory, matching strong baselines at far lower cost and fewer tool calls.
- Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language ModelsICLR 2025 · Sep 17, 2024
The first dense retriever that can be prompted like a language model. Trained on ~500k instance-level instructions from MS MARCO, it follows free-form relevance instructions and is far more robust to query phrasing.
- Dancing in Chains: Reconciling Instruction Following and Faithfulness in Language ModelsEMNLP 2024 · Jul 31, 2024
Language models face a tension between following user instructions and staying faithful to source text. The paper characterizes this tradeoff and proposes training methods to reconcile the two.
- RAG-QA Arena: Evaluating Domain Robustness for Long-Form Retrieval-Augmented Question AnsweringEMNLP 2024 · Jul 19, 2024
A benchmark for long-form retrieval-augmented QA across seven domains, pairing human-written long-form answers with an LLM evaluator to measure cross-domain robustness.
- Selective ‘Selective Prediction’: Reducing Unnecessary Abstention in Vision-Language ReasoningACL Findings · Jun 12, 2024
Selective prediction makes vision-language models abstain too often. ReCoVERR gathers extra visual evidence at inference time so models answer up to 20% more questions without raising the error rate.
- When Does Monolingual Data Help Multilingual Translation: The Role of Domain and Model ScaleNAACL · Jun 1, 2024
A systematic study of when adding monolingual data improves multilingual translation, isolating the effects of domain match and model scale.
- Lost in the Middle: How Language Models Use Long ContextsTACL · Feb 23, 2024
Language models use information best when it sits at the start or end of the input, with accuracy dropping sharply for facts buried in the middle of long contexts.
- Deal, or No Deal (or Who Knows)? Forecasting Uncertainty in Conversations Using Large Language ModelsACL Findings · Feb 5, 2024
FortUne Dial reframes conversation forecasting with uncertainty-aware metrics. Fine-tuning for calibration lets small open models forecast negotiation outcomes on par with models 10x their size.
- Improving Wikipedia Verifiability With AINature Machine Intelligence · Oct 19, 2023
SIDE checks Wikipedia claims against their cited sources, flags citations that fail to support a claim, and recommends better references. Editors preferred its suggestions in real trials.
- Cross-Domain Image Captioning With Discriminative FinetuningCVPR · Jun 18, 2023
Finetuning a captioner with a discriminative, self-supervised reward produces more specific, less generic captions that transfer better across visual domains.
- Can Discrete Information Extraction Prompts Generalize Across Language Models?ICLR · Feb 20, 2023
An investigation of whether automatically discovered discrete prompts for information extraction transfer across different language models.
Benchmark: FrontierFinance
An open benchmark of open-ended financial-research queries, scored against expert rubric items. Open-ended financial-research queries are scored against expert rubric items to measure frontier AI intelligence in finance.
Explore the benchmark → · Dataset on Hugging Face · Code on GitHub