SLMAndYou

Overview

SLMAndYou is a Flutter application that enables running Small Language Models (SLMs) directly on your Android device — fully offline, with no cloud dependency. It loads GGUF-format models via llama.cpp and streams token-by-token responses in real time.

Features

• On-Device Inference – Run SLMs locally on Android with no internet required
• GGUF Model Support – Load any GGUF-format model file from device storage
• Real-Time Token Streaming – Responses stream token-by-token as they generate
• Configurable Parameters – Fine-tune temperature, top-p, top-k, repeat penalty, context length, and max tokens
• Model File Picker – Browse and select model files or use the default TinyLlama path
• Hardware Stats Dashboard – View device info, memory, storage, and architecture at a glance
• HuggingFace Integration – Direct link to download GGUF models from HuggingFace
• Model Loading Progress – Visual progress indicator during model loading
• Live Logging Panel – Real-time verbose logs for debugging and monitoring
• GPU Acceleration – Supports GPU layer offloading for faster inference

Architecture

• Framework: Flutter (Dart) with Material 3 design
• Inference Engine: llama.cpp via llama_flutter_android package
• Platform: Android (SDK 3.3.0+)
• Default Model: TinyLlama 1.1B Chat v1.0 (Q2_K quantized GGUF)
• Storage: SharedPreferences for persisting model path selection

Screenshots