Danh sách các thư viện, framework và công cụ phổ biến trong hệ sinh thái Python dành cho Artificial Intelligence (AI), Machine Learning (ML), Deep Learning và Generative AI.
| Tool | Description |
|---|
| NumPy | Numerical computing, matrix operations |
| Pandas | DataFrame, data manipulation |
| Polars | High-performance DataFrame |
| PyArrow | Columnar data processing, Apache Arrow |
| Dask | Parallel and distributed computing |
| DuckDB | Analytical SQL database |
| Tool | Description |
|---|
| Matplotlib | Basic plotting and visualization |
| Seaborn | Statistical data visualization |
| Plotly | Interactive charts |
| Bokeh | Interactive visualization |
| Altair | Declarative statistical visualization |
| Tool | Description |
|---|
| Scikit-learn | Classification, regression, clustering |
| XGBoost | Gradient boosting framework |
| LightGBM | High-performance gradient boosting |
| CatBoost | Gradient boosting for categorical data |
| Statsmodels | Statistical modeling and analysis |
| Tool | Description |
|---|
| PyTorch | Deep learning framework |
| TensorFlow | Deep learning and model deployment |
| Keras | High-level neural network API |
| JAX | High-performance numerical computing and automatic differentiation |
| Tool | Description |
|---|
| OpenCV | Image and video processing |
| Pillow | Image manipulation |
| Ultralytics YOLO | Object detection |
| Detectron2 | Object detection and segmentation |
| MMDetection | Object detection toolbox |
| Torchvision | Computer vision models and datasets |
| Tool | Description |
|---|
| NLTK | Fundamental NLP operations |
| spaCy | Tokenization, NER and NLP pipelines |
| Gensim | Topic modeling and Word2Vec |
| Stanza | Multilingual NLP |
| Sentence Transformers | Text embeddings and semantic similarity |
| Tool | Description |
|---|
| Hugging Face Transformers | Pretrained language models |
| Diffusers | Image and video diffusion models |
| PEFT | Parameter-efficient fine-tuning, LoRA |
| TRL | RLHF and model post-training |
| vLLM | High-throughput LLM inference |
| llama.cpp | Local LLM inference |
| Unsloth | Efficient LLM fine-tuning |
| Ollama | Local model management and inference |
| Tool | Description |
|---|
| LangChain | LLM application framework |
| LangGraph | Stateful agent workflows |
| LlamaIndex | Data ingestion and RAG |
| Haystack | Search and RAG pipelines |
| DSPy | Programming and optimization of LM pipelines |
| PydanticAI | Typed AI agent framework |
| CrewAI | Multi-agent orchestration |
| AutoGen | Multi-agent conversation framework |
| Tool | Description |
|---|
| FAISS | Vector similarity search |
| ChromaDB | Vector database |
| Qdrant | Vector search engine |
| Milvus | Scalable vector database |
| Weaviate | Vector database |
| Sentence Transformers | Embedding generation |
| Tool | Description |
|---|
| MLflow | Experiment tracking and model registry |
| Weights & Biases | Training monitoring |
| TensorBoard | Metrics and model visualization |
| Optuna | Hyperparameter optimization |
| Evidently AI | Data and model drift monitoring |
| DeepEval | LLM evaluation |
| Tool | Description |
|---|
| FastAPI | Expose AI models through REST APIs |
| BentoML | Model serving |
| Ray Serve | Distributed model serving |
| ONNX Runtime | Cross-platform model inference |
| Gradio | AI demo interfaces |
| Streamlit | Data and AI dashboards |
| Tool | Description |
|---|
| Apache Airflow | Schedule data and ML pipelines |
| Prefect | Workflow orchestration |
| Dagster | Data orchestration |
| Kubeflow | Kubernetes-based ML workflows |
| DVC | Data and model versioning |
| Feast | Feature store |
- Whisper
- faster-whisper
- SpeechBrain
- Gymnasium
- Stable-Baselines3
- Ray RLlib
- Data Foundation: NumPy → Pandas → Polars → Matplotlib
- Machine Learning: Scikit-learn → XGBoost → Optuna
- Deep Learning: PyTorch → Torchvision
- Modern AI: Hugging Face → Transformers → PEFT
- AI Applications: Embeddings → Qdrant → RAG → LangGraph
- Production: FastAPI → MLflow → Docker
| Task | Recommended Tools |
|---|
| Text classification | Scikit-learn / Transformers |
| Semantic search | Sentence Transformers |
| Sentiment analysis | Transformers |
| Named Entity Recognition | spaCy |
| Local LLM | Ollama / llama.cpp |
| Agent workflows | LangGraph |
| Vector search | Qdrant |
| API development | FastAPI |
| Model evaluation | MLflow / DeepEval |
- Training: Teaching a model using datasets.
- Fine-tuning: Adapting a pretrained model to a specific task.
- Inference: Running a trained model to generate predictions.
- Embedding: Converting data into numerical vector representations.
- RAG: Combining retrieval with LLM generation.
- Orchestration: Connecting models, tools and workflows.
- MLOps: Managing the lifecycle of ML models in production.
Not every AI application requires model training. Most practical AI systems combine pretrained models, inference, classification, retrieval and orchestration.