AI biếtAI biết
AI Knowledge
AI Knowledge
  • Ai biết

    • AI Knowledge
    • 100 AI Concepts – Glossary
  • Tools

    • Cloudflare tools
    • Typescript tool
    • Python AI / ML Ecosystem

Python AI / ML Ecosystem

Danh sách các thư viện, framework và công cụ phổ biến trong hệ sinh thái Python dành cho Artificial Intelligence (AI), Machine Learning (ML), Deep Learning và Generative AI.

1. Data Processing

ToolDescription
NumPyNumerical computing, matrix operations
PandasDataFrame, data manipulation
PolarsHigh-performance DataFrame
PyArrowColumnar data processing, Apache Arrow
DaskParallel and distributed computing
DuckDBAnalytical SQL database

2. Data Visualization

ToolDescription
MatplotlibBasic plotting and visualization
SeabornStatistical data visualization
PlotlyInteractive charts
BokehInteractive visualization
AltairDeclarative statistical visualization

3. Traditional Machine Learning

ToolDescription
Scikit-learnClassification, regression, clustering
XGBoostGradient boosting framework
LightGBMHigh-performance gradient boosting
CatBoostGradient boosting for categorical data
StatsmodelsStatistical modeling and analysis

4. Deep Learning Frameworks

ToolDescription
PyTorchDeep learning framework
TensorFlowDeep learning and model deployment
KerasHigh-level neural network API
JAXHigh-performance numerical computing and automatic differentiation

5. Computer Vision

ToolDescription
OpenCVImage and video processing
PillowImage manipulation
Ultralytics YOLOObject detection
Detectron2Object detection and segmentation
MMDetectionObject detection toolbox
TorchvisionComputer vision models and datasets

6. Natural Language Processing (NLP)

ToolDescription
NLTKFundamental NLP operations
spaCyTokenization, NER and NLP pipelines
GensimTopic modeling and Word2Vec
StanzaMultilingual NLP
Sentence TransformersText embeddings and semantic similarity

7. LLM & Generative AI

ToolDescription
Hugging Face TransformersPretrained language models
DiffusersImage and video diffusion models
PEFTParameter-efficient fine-tuning, LoRA
TRLRLHF and model post-training
vLLMHigh-throughput LLM inference
llama.cppLocal LLM inference
UnslothEfficient LLM fine-tuning
OllamaLocal model management and inference

8. AI Agents & RAG

ToolDescription
LangChainLLM application framework
LangGraphStateful agent workflows
LlamaIndexData ingestion and RAG
HaystackSearch and RAG pipelines
DSPyProgramming and optimization of LM pipelines
PydanticAITyped AI agent framework
CrewAIMulti-agent orchestration
AutoGenMulti-agent conversation framework

9. Vector Database & Search

ToolDescription
FAISSVector similarity search
ChromaDBVector database
QdrantVector search engine
MilvusScalable vector database
WeaviateVector database
Sentence TransformersEmbedding generation

10. Model Evaluation & Experiment Tracking

ToolDescription
MLflowExperiment tracking and model registry
Weights & BiasesTraining monitoring
TensorBoardMetrics and model visualization
OptunaHyperparameter optimization
Evidently AIData and model drift monitoring
DeepEvalLLM evaluation

11. Deployment & API

ToolDescription
FastAPIExpose AI models through REST APIs
BentoMLModel serving
Ray ServeDistributed model serving
ONNX RuntimeCross-platform model inference
GradioAI demo interfaces
StreamlitData and AI dashboards

12. MLOps & Data Pipeline

ToolDescription
Apache AirflowSchedule data and ML pipelines
PrefectWorkflow orchestration
DagsterData orchestration
KubeflowKubernetes-based ML workflows
DVCData and model versioning
FeastFeature store

13. Specialized AI Libraries

Speech-to-Text

  • Whisper
  • faster-whisper
  • SpeechBrain

Text-to-Speech

  • Coqui TTS
  • Kokoro
  • Piper

OCR

  • EasyOCR
  • PaddleOCR
  • docTR

Reinforcement Learning

  • Gymnasium
  • Stable-Baselines3
  • Ray RLlib

Recommendation Systems

  • RecBole
  • LightFM
  • implicit

Anomaly Detection

  • PyOD
  • Isolation Forest

Synthetic Data

  • SDV
  • Faker
  • Gretel

AutoML

  • AutoGluon
  • FLAML
  • H2O AutoML

Recommended Learning Roadmap

  1. Data Foundation: NumPy → Pandas → Polars → Matplotlib
  2. Machine Learning: Scikit-learn → XGBoost → Optuna
  3. Deep Learning: PyTorch → Torchvision
  4. Modern AI: Hugging Face → Transformers → PEFT
  5. AI Applications: Embeddings → Qdrant → RAG → LangGraph
  6. Production: FastAPI → MLflow → Docker

Practical AI Stack

TaskRecommended Tools
Text classificationScikit-learn / Transformers
Semantic searchSentence Transformers
Sentiment analysisTransformers
Named Entity RecognitionspaCy
Local LLMOllama / llama.cpp
Agent workflowsLangGraph
Vector searchQdrant
API developmentFastAPI
Model evaluationMLflow / DeepEval

Important Concepts

  • Training: Teaching a model using datasets.
  • Fine-tuning: Adapting a pretrained model to a specific task.
  • Inference: Running a trained model to generate predictions.
  • Embedding: Converting data into numerical vector representations.
  • RAG: Combining retrieval with LLM generation.
  • Orchestration: Connecting models, tools and workflows.
  • MLOps: Managing the lifecycle of ML models in production.

Not every AI application requires model training. Most practical AI systems combine pretrained models, inference, classification, retrieval and orchestration.

Prev
Typescript tool