4.Deep Learning#

4、深度学习(Deep Learning)#

4.1 论文(Papers)#

(1) Deep Learning Papers Reading Roadmap#

(2) Paper with code#

(3) Paper with code-state of the art#

4.2神经网络(Neural Networks)#

(1) 理解神经网络(Understanding Neural Networks)#

(2) 损失函数(Loss Functions)#

(3) 激活函数(Activation Functions)#

(4) 权重初始化(Weight Initialization)#

(5) 梯度消失/爆炸问题(Vanishing/Exploding Gradient Problem)#

4.3 结构(Architectures)#

(1) 前馈神经网络(Feedforward Neural Network)#

(2) 自编码器(Autoencoder)#

(3) 卷积神经网络(CNN,Convolutional Neural Network)#

(4) 循环神经网络(Recurrent Neural Network)#

(5) Transformer#

(6) 孪生神经网络(Siamese Network)#

(7) 对抗生成网络(GAN,Generative Adversarial Network)#

(8) 进化发展结构(Evolving Architectures /NEAT)#

(9) 残差连接(Residual Connections)#

4.4 训练(Training)#

(1) 优化器(Optimizers)#

(2) 学习率调度器(Learning Rate Schedule)#

(3) 批量归一化(Batch Normalization)#

(4) 批量大小影响(Batch Size Effects)#

(5) 正则化(Regularization)#

(6) 多任务学习(Multitask Learning)#

(7) 迁移学习(Transfer Learning)#

(8) 课程学习(Curriculum Learning)#

4.5 工具框架(Tools&Frame)#

(1) Important Libraries#

a.Awesome Deep Learning#

b.Huggingface Transformers#

(2) TensorFlow#

(3) Pytorch#

(4) TensorBoard#

(5) MLFlow#

4.6 高级模型优化(Model optimization advanced)#

(1) 蒸馏(Distillation)#

(2) 量化(Quantization)#