# 4.Deep Learning

---

![](../../docs/img/4.Deep_Learning.png)

---

# 4、深度学习(Deep Learning)

## 4.1 论文(Papers)

### (1) Deep Learning Papers Reading Roadmap

### (2) Paper with code

### (3) Paper with code-state of the art

## 4.2神经网络(Neural Networks)

### (1) 理解神经网络(Understanding Neural Networks)

### (2) 损失函数(Loss Functions)

### (3) 激活函数(Activation Functions)

### (4) 权重初始化(Weight Initialization)

### (5) 梯度消失/爆炸问题(Vanishing/Exploding Gradient Problem)

## 4.3 结构(Architectures)

### (1) 前馈神经网络(Feedforward Neural Network)

### (2) 自编码器(Autoencoder)

### (3) 卷积神经网络(CNN,Convolutional Neural Network)

### (4) 循环神经网络(Recurrent Neural Network)

### (5) Transformer

### (6) 孪生神经网络(Siamese Network)

### (7) 对抗生成网络(GAN,Generative Adversarial Network)

### (8) 进化发展结构(Evolving Architectures /NEAT)

### (9) 残差连接(Residual Connections) 

## 4.4 训练(Training)

### (1) 优化器(Optimizers)

### (2) 学习率调度器(Learning Rate Schedule)

### (3) 批量归一化(Batch Normalization)

### (4) 批量大小影响(Batch Size Effects)

### (5) 正则化(Regularization)

### (6) 多任务学习(Multitask Learning)

### (7) 迁移学习(Transfer Learning)

### (8) 课程学习(Curriculum Learning)

## 4.5 工具框架(Tools&Frame)

### (1) Important Libraries

#### a.Awesome Deep Learning

#### b.Huggingface Transformers

### (2) TensorFlow

### (3) Pytorch

### (4) TensorBoard

### (5) MLFlow

## 4.6 高级模型优化(Model optimization advanced)

### (1) 蒸馏(Distillation)

### (2) 量化(Quantization)

### (3) 神经架构搜索(NAS,Neural Architecture Search)