深度学习教程¶

概述¶

深度学习是机器学习的一个子领域，使用多层神经网络从数据中学习层次化的特征表示。

神经网络架构¶

graph LR
    subgraph 输入层
        I1[x₁]
        I2[x₂]
        I3[x₃]
    end

    subgraph 隐藏层1
        H11[神经元]
        H12[神经元]
        H13[神经元]
    end

    subgraph 隐藏层2
        H21[神经元]
        H22[神经元]
    end

    subgraph 输出层
        O1[y]
    end

    I1 --> H11
    I1 --> H12
    I1 --> H13
    I2 --> H11
    I2 --> H12
    I2 --> H13
    I3 --> H11
    I3 --> H12
    I3 --> H13

    H11 --> H21
    H11 --> H22
    H12 --> H21
    H12 --> H22
    H13 --> H21
    H13 --> H22

    H21 --> O1
    H22 --> O1

    style I1 fill:#e3f2fd
    style I2 fill:#e3f2fd
    style I3 fill:#e3f2fd
    style O1 fill:#e8f5e9

神经网络基础¶

感知机¶

感知机是最简单的神经网络模型。

flowchart LR
    X[输入 x] --> W[加权求和]
    W --> Z[Σwᵢxᵢ + b]
    Z --> A[激活函数 f]
    A --> Y[输出 y]

    style X fill:#e3f2fd
    style Z fill:#fff3e0
    style Y fill:#e8f5e9

数学表达：

y = f(w₁x₁ + w₂x₂ + ... + wₙxₙ + b)

激活函数¶

函数	公式	特点
Sigmoid	σ(x) = 1/(1+e⁻ˣ)	输出(0,1)，易梯度消失
Tanh	tanh(x)	输出(-1,1)，零中心化
ReLU	max(0,x)	计算简单，缓解梯度消失
Leaky ReLU	max(0.01x,x)	ReLU变体，解决死亡ReLU

import torch
import torch.nn.functional as F

# 常用激活函数
x = torch.randn(1, 10)
print(f"ReLU: {F.relu(x)}")
print(f"Sigmoid: {torch.sigmoid(x)}")
print(f"Tanh: {torch.tanh(x)}")

PyTorch基础¶

张量操作¶

import torch

# 创建张量
x = torch.tensor([1, 2, 3])
y = torch.zeros(3, 3)

# 基本运算
z = x + y
z = torch.matmul(x, y)

# GPU加速
if torch.cuda.is_available():
    x = x.cuda()

构建神经网络¶

import torch.nn as nn

class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(784, 256)
        self.fc2 = nn.Linear(256, 128)
        self.fc3 = nn.Linear(128, 10)
        self.dropout = nn.Dropout(0.2)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        x = self.dropout(x)
        x = torch.relu(self.fc2(x))
        x = self.fc3(x)
        return x

训练循环¶

import torch.optim as optim

# 模型、损失函数、优化器
model = SimpleNet()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# 训练循环
for epoch in range(10):
    for data, target in train_loader:
        optimizer.zero_grad()
        output = model(data)
        loss = criterion(output, target)
        loss.backward()
        optimizer.step()

    print(f"Epoch {epoch+1}, Loss: {loss.item():.4f}")

卷积神经网络（CNN）¶

CNN结构¶

输入图像 → [卷积层 → 激活层 → 池化层] × N → 全连接层 → 输出

卷积层详解¶

class CNN(nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        # 卷积层
        self.conv1 = nn.Conv2d(1, 32, kernel_size=3, padding=1)
        self.conv2 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
        # 池化层
        self.pool = nn.MaxPool2d(2, 2)
        # 全连接层
        self.fc1 = nn.Linear(64 * 7 * 7, 128)
        self.fc2 = nn.Linear(128, 10)

    def forward(self, x):
        x = self.pool(torch.relu(self.conv1(x)))
        x = self.pool(torch.relu(self.conv2(x)))
        x = x.view(-1, 64 * 7 * 7)
        x = torch.relu(self.fc1(x))
        x = self.fc2(x)
        return x

经典CNN架构¶

架构	年份	特点
LeNet-5	1998	CNN开山之作
AlexNet	2012	深度学习突破
VGG	2014	小卷积核堆叠
ResNet	2015	残差连接，解决梯度消失
EfficientNet	2019	复合缩放策略

循环神经网络（RNN）¶

RNN基础¶

class SimpleRNN(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(SimpleRNN, self).__init__()
        self.hidden_size = hidden_size
        self.rnn = nn.RNN(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        h0 = torch.zeros(1, x.size(0), self.hidden_size)
        out, _ = self.rnn(x, h0)
        out = self.fc(out[:, -1, :])
        return out

LSTM（长短期记忆）¶

LSTM解决了RNN的长期依赖问题。

class LSTM(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(LSTM, self).__init__()
        self.hidden_size = hidden_size
        self.lstm = nn.LSTM(input_size, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        h0 = torch.zeros(1, x.size(0), self.hidden_size)
        c0 = torch.zeros(1, x.size(0), self.hidden_size)
        out, _ = self.lstm(x, (h0, c0))
        out = self.fc(out[:, -1, :])
        return out

Transformer架构¶

注意力机制¶

Attention(Q,K,V) = softmax(QKᵀ/√dₖ)V

Transformer结构¶

class Transformer(nn.Module):
    def __init__(self, d_model=512, nhead=8):
        super(Transformer, self).__init__()
        self.embedding = nn.Embedding(vocab_size, d_model)
        self.pos_encoder = PositionalEncoding(d_model)
        self.transformer = nn.Transformer(
            d_model=d_model,
            nhead=nhead,
            num_encoder_layers=6,
            num_decoder_layers=6
        )
        self.fc_out = nn.Linear(d_model, vocab_size)

    def forward(self, src, tgt):
        src = self.pos_encoder(self.embedding(src))
        tgt = self.pos_encoder(self.embedding(tgt))
        output = self.transformer(src, tgt)
        return self.fc_out(output)

模型训练技巧¶

学习率调度¶

# 学习率预热 + 衰减
scheduler = optim.lr_scheduler.OneCycleLR(
    optimizer,
    max_lr=0.01,
    steps_per_epoch=len(train_loader),
    epochs=num_epochs
)

数据增强¶

from torchvision import transforms

transform = transforms.Compose([
    transforms.RandomHorizontalFlip(),
    transforms.RandomRotation(10),
    transforms.ColorJitter(brightness=0.2),
    transforms.ToTensor(),
    transforms.Normalize((0.5,), (0.5,))
])

GPU训练¶

# 检查GPU
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

# 移动模型到GPU
model = model.to(device)

# 训练时移动数据到GPU
for data, target in train_loader:
    data, target = data.to(device), target.to(device)
    # ... 训练代码

下一步学习¶

Python机器学习实战 - 实战项目
自然语言处理 - NLP应用
计算机视觉 - CV应用