Skip to content

Instantly share code, notes, and snippets.

@ehzawad
Created April 8, 2026 00:27
Show Gist options
  • Select an option

  • Save ehzawad/3907ff5d791ff30ea5ab4e3bef019f35 to your computer and use it in GitHub Desktop.

Select an option

Save ehzawad/3907ff5d791ff30ea5ab4e3bef019f35 to your computer and use it in GitHub Desktop.
User question:                                                                                        
{truncated user question}                                                                             
                                                                                                      
Candidates:                                                                                           
Candidate 1:                                                                                          
Tag: {tag}                                                                                            
Matched question: {truncated question}                                                              
Answer: {truncated answer}                                                                            
Retrieval score: {score:.6f}                                                                          
Candidate 2:                                                                                          
Tag: {tag}                                                                                            
Matched question: {truncated question}
Answer: {truncated answer}
Retrieval score: {score:.6f}
Candidate 3:
Tag: {tag}
Matched question: {truncated question}
Answer: {truncated answer}
Retrieval score: {score:.6f}

Return JSON only with this exact schema:
{
  "best_index": 1,
  "confidence": 0.0,
  "abstain": false
}

Rules:
- best_index must be an integer between 1 and the number of candidates.
- confidence must be a number between 0 and 1.
- Candidate 1 is the default because the retrieval system already ranked it highest.
- Choose Candidate 2 or Candidate 3 only if it is clearly better than Candidate 1 on meaning and
intent.
- Clearly better means Candidate 1 misses a key constraint, answers a different intent, or is
materially less specific.
- Do not move away from Candidate 1 for minor wording differences, stylistic preference,
near-duplicate answers, or small score gaps.
- If Candidate 1 is acceptable and the other candidates are only slightly better or are paraphrases,
return best_index 1.
- Use abstain=true only when none of the candidates clearly answer the user's question.
- Use retrieval rank and retrieval score as tie-breakers, favoring the higher-ranked candidate.
- Judge Bengali meaning and user intent, not surface word overlap.
- Do not include markdown, explanations, or extra text.

The "Recent conversation context" block is omitted entirely when there's no prior history (i.e., first
 message in a conversation). The {...} placeholders fill dynamically per request from FAISS results,
user input, and message history. The Ollama call sends this as "prompt" with "format": "json" and
"stream": false.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment