Fork: 0
yangyangxie / cortex-hub
Download ZIP 487 commits
Transfer to URL with SHA
cortex-hub / test_venv / lib / python3.9 / site-packages / litellm / caching /
latest commit 4edcfeb6ca
@Antigravity AI Antigravity AI authored 9 hours ago
..
Readme.md Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
__init__.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
_internal_lru_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
azure_blob_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
base_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
caching.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
caching_handler.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
disk_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
dual_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
gcs_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
in_memory_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
llm_caching_handler.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
qdrant_semantic_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
redis_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
redis_cluster_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
redis_semantic_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
s3_cache.py Hardened Windows Agent Task Termination and Automatic Cleanup 9 hours ago
Readme.md

Caching on LiteLLM

LiteLLM supports multiple caching mechanisms. This allows users to choose the most suitable caching solution for their use case.

The following caching mechanisms are supported:

  1. RedisCache
  2. RedisSemanticCache
  3. QdrantSemanticCache
  4. InMemoryCache
  5. DiskCache
  6. S3Cache
  7. AzureBlobCache
  8. DualCache (updates both Redis and an in-memory cache simultaneously)

Folder Structure

litellm/caching/
├── base_cache.py
├── caching.py
├── caching_handler.py
├── disk_cache.py
├── dual_cache.py
├── in_memory_cache.py
├── qdrant_semantic_cache.py
├── redis_cache.py
├── redis_semantic_cache.py
├── s3_cache.py

Documentation