Skip to content
Varish Gahlot

SMART-Eq

Structured and Unstructured Market Analysis Using Retrieval Techniques and LLMs for Equities

International IEEE conference

2025 World Skills Conference on Universal Data Analytics and Sciences (WorldSUAS)

Authors

Aditya S. Chandra, Anisha S. Chandra, Varish Gahlot

DOI

10.1109/WorldSUAS66815.2025.11199272

Varish's contribution

Varish built the retrieval layer: the techniques that identify relevant information across structured financial data and unstructured text and connect it, so that the language model works from linked evidence rather than a single source.

Pipeline, as described in the paper

  1. 01Data ingestion
  2. 02Domain-specific text chunking
  3. 03Dual indexing: lexical and vector
  4. 04Hybrid retrieval and re-ranking
  5. 05LLM-generated analysis
Highlighted: where the retrieval layer, Varish's documented contribution, sits in the pipeline. Simplified from the published abstract.

01The research question

Can a language model give more precise, better-grounded answers about equities when it draws on both the numbers and the narrative around them?

Equity forecasting has traditionally relied on structured data: historical prices, company fundamentals and macroeconomic indicators. Much of what analysts actually weigh, though, lives in unstructured text such as analyst reports, earnings-call transcripts and financial news. SMART-Eq asks how those qualitative sources can be retrieved and combined with structured data so that a large language model can reason across both.

02Method

The paper describes a multi-stage pipeline: data ingestion, domain-specific chunking of financial text, and dual indexing (lexical and vector-based) to support hybrid retrieval. Retrieved material is then supplied to a large language model to generate its analysis.

Four retrieval paradigms were evaluated side by side: traditional text search, vector-based retrieval, hybrid retrieval, and vector-based retrieval with re-ranking. Testing used a custom dataset of 315 financial documents with 20 control questions, validated against the FinQA dataset and assessed with ROUGE metrics and expert evaluation.

  • Four retrieval paradigms compared: traditional text search, vector-based retrieval, hybrid retrieval, and vector-based retrieval with re-ranking
  • A custom dataset of 315 financial documents, augmented by 20 control questions
  • Validation against the FinQA dataset
  • Evaluation with ROUGE metrics and expert review

03Varish's contribution

SMART-Eq is collaborative work by three co-authors. Varish built the retrieval layer: the techniques that identify relevant information across structured financial data and unstructured text and connect it, so the model reasons from linked evidence rather than from a single source.

04Findings

According to the published abstract, incorporating vectorization and re-ranking improved both the precision of the retrieved information and the contextual accuracy of the model's responses. The authors conclude that the approach bridges qualitative insight with quantitative data for equity market analysis.

05Limitations and future directions

The paper closes with integration challenges and future research directions, including fine-tuning language models for the financial domain and processing data in real time. The evaluation is on a 315-document dataset; the findings describe that experimental setting, not live trading performance.