Software Architecture for AI

By O’Reilly: Software Architecture Superstream: Architecture Patterns and Antipatterns for AI - O’Reilly Live Events

Emerging Patterns in Agentic AI Link to heading

by Bharani Subramaniam

Append only context Link to heading

• If you change the context after sending to the LLM, it will miss the cache and be slow and expensive.

Constrain tool selection Link to heading

• Too many tools can be overwhelming for the LLM.
• Constraint tool selection to the specific subset by using a prefix in each tool name to help the LLM; e.g. search_, code_, etc.

Mind Your Language Models: An Approach to Architecting Intelligent Systems Link to heading

by Nischal HP

Embedding Models Link to heading

• MTEB Leaderboard - a Hugging Face Space by mteb

Text Processing & Chunking Link to heading

• Why chunk text?
  • Limit on context windows
  • Longer the text, lesser the relevance, vaguer the answer
• How to chunk text?
  • Content and Context are the drivers
    • Semantic
    • Sentence
    • Sliding window
    • Hierarchical
• How to evaluate chunking?
  • Real world queries
  • Evaluation of several models
  • Ranked segments and documents
  • Precision, Recall, nDCG

Retrieval Strategy Link to heading

• Improving retrieval
  • Query augmentation
  • Keyword search
  • Vector Search
  • Fusion and Reranking

Selecting LLM & Grounding Link to heading