【前沿】Pykan风靡GitHub，近万收藏的含金量到底如何

用户11172986

发布于 2024-06-20 19:16:12

1460

发布于 2024-06-20 19:16:12

文章被收录于专栏：气python风雨气python风雨

前言

近期，技术圈被一个名为Pykan的项目彻底点燃，其在GitHub上的关注度火箭般攀升，逼近万星大关，引得无数开发者侧目

像什么xx之心，量子x, 新x元等top公众号都写了相关文章

有人感慨前苏联数理水平之高，有人惊叹作者脑洞大开，更有人称之为未来已来

当然也不乏质疑的声音

本项目中小编将基于官方的示例进行一些简单测试

旨在剥开层层热度的外衣，与大家一窥它的真实面貌与潜力

注意

尽量使用核数较多的资源，例如我目前使用的4核16g 当前镜像为气象分析3.9

点击以下链接可在线运行Fork查看完整程序 🔜🔜若没有成功加载可视化图，点击运行可以查看 ps:隐藏代码在【代码已被隐藏】所在行，点击所在行，可以看到该行的最右角，会出现个三角形，点击查看即可

【前沿】Pykan风靡GitHub，近万收藏的含金量到底如何

其名为KAN

Kolmogorov-Arnold Networks (KANs) 是一类受到数学家 Kolmogorov 和 Arnold 的工作启发的神经网络结构，它们为多层感知器（MLPs）提供了一种有吸引力的替代方案。KANs 的设计基于 Kolmogorov-Arnold 表示定理，这与 MLPs 基于的通用近似定理形成对比。一个显著的区别在于 KANs 将激活函数置于网络的边（连接）上，而传统的 MLPs 则将激活函数应用于节点上。这一结构上的变化使得 KANs 在模型精度和可解释性方面有时能够显著优于 MLPs。

关键特性包括：

更快的规模增长与更高精度：KANs 能够以较少的参数量实现比 MLPs 更高的预测准确性，展现出更优的参数效率。
可解释性与可视化：KANs 提供了 MLPs 难以实现的模型可解释性和交互性，有助于科学规律的发现和理解。
应用场景多样：从拟合符号公式、特殊函数到偏微分方程求解，乃至避免灾难性遗忘，展示了 KANs 在多个领域的潜力。
发现新定律：KANs 可能帮助科研人员直观地探索和发现新的科学和数学定律，如对结的数学规律或安德森局域化的物理规律的发现。

以上为github介绍机翻

在github界面介绍的最后，作者还写了一句话

Practice is the only criterion for testing understanding (实践是检验真理的唯一标准).

环境搭建

pykan需要matplotlib==3.6.2 numpy==1.24.4 scikit_learn==1.1.3 setuptools==65.5.0 sympy==1.11.1 torch==2.2.2 tqdm==4.66.2 当前镜像库比较齐全，我们只需要torch和pykan即可

In [5]:

pip install pykan -i https://pypi.mirrors.ustc.edu.cn/simple/

In [4]:

pip install --index-url https://pypi.mirrors.ustc.edu.cn/simple torch==2.2.2 --user

回归问题

我们首先将该问题视为回归问题（输出维数 = 1，MSE 损失）

建立一个双月数据集，官方示例是一千样本，我改为10000样本测试

In [1]:

from kan import KAN
import matplotlib.pyplot as plt
from sklearn.datasets import make_moons
import torch
import numpy as np

dataset = {}
train_input, train_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)
test_input, test_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)

dataset['train_input'] = torch.from_numpy(train_input)
dataset['test_input'] = torch.from_numpy(test_input)
dataset['train_label'] = torch.from_numpy(train_label[:,None])
dataset['test_label'] = torch.from_numpy(test_label[:,None])

X = dataset['train_input']
y = dataset['train_label']
plt.scatter(X[:,0], X[:,1], c=y[:,0])

支持向量机

In [10]:

from sklearn import svm
from sklearn.model_selection import train_test_split
from sklearn.metrics import classification_report

# 将数据集分为训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(train_input, train_label, test_size=0.2, random_state=42)

# 初始化SVM模型，这里我们使用RBF核函数
clf = svm.SVC(kernel='rbf', C=1, gamma='scale')

# 训练模型
clf.fit(X_train, y_train)

# 预测并评估
y_pred = clf.predict(X_test)
print(classification_report(y_test, y_pred))

              precision    recall  f1-score   support

           0       1.00      0.99      1.00       107
           1       0.99      1.00      0.99        93

    accuracy                           0.99       200
   macro avg       0.99      1.00      0.99       200
weighted avg       1.00      0.99      1.00       200

KAN

model = KAN(width=[2,1], grid=3, k=3)

def train_acc():
    return torch.mean((torch.round(model(dataset['train_input'])[:,0]) == dataset['train_label'][:,0]).float())

def test_acc():
    return torch.mean((torch.round(model(dataset['test_input'])[:,0]) == dataset['test_label'][:,0]).float())

results = model.train(dataset, opt="LBFGS", steps=20, metrics=(train_acc, test_acc));
results['train_acc'][-1], results['test_acc'][-1]

自动符号回归

In [5]:

lib = ['x','x^2','x^3','x^4','exp','log','sqrt','tanh','sin','tan','abs']
model.auto_symbolic(lib=lib)
formula = model.symbolic_formula()[0][0]
formula

fixing (0,0,0) with sin, r2=0.9731504496053569
fixing (0,1,0) with tan, r2=0.978959743323539−

0.39sin⁡(3.1𝑥1−1.53)−0.82tan⁡(0.92𝑥2−3.39)+0.48

这个公式有多准确？

In [6]:

def acc(formula, X, y):
    batch = X.shape[0]
    correct = 0
    for i in range(batch):
        correct += np.round(np.array(formula.subs('x_1', X[i,0]).subs('x_2', X[i,1])).astype(np.float64)) == y[i,0]
    return correct/batch

print('train acc of the formula:', acc(formula, dataset['train_input'], dataset['train_label']))
print('test acc of the formula:', acc(formula, dataset['test_input'], dataset['test_label']))

train acc of the formula: tensor(0.9987)
test acc of the formula: tensor(0.9981)

分类方程

（输出维数 = 2，交叉熵损失）。

In [11]:

from kan import KAN
import matplotlib.pyplot as plt
from sklearn.datasets import make_moons
import torch
import numpy as np

dataset = {}
train_input, train_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)
test_input, test_label = make_moons(n_samples=10000, shuffle=True, noise=0.1, random_state=None)

dataset['train_input'] = torch.from_numpy(train_input)
dataset['test_input'] = torch.from_numpy(test_input)
dataset['train_label'] = torch.from_numpy(train_label)
dataset['test_label'] = torch.from_numpy(test_label)

X = dataset['train_input']
y = dataset['train_label']
plt.scatter(X[:,0], X[:,1], c=y[:])

<matplotlib.collections.PathCollection at 0x7f80a3933b20>

向量机

In [12]:

from sklearn import svm
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score

# 数据预处理：对输入数据进行标准化
scaler = StandardScaler()
train_input_scaled = scaler.fit_transform(train_input)
test_input_scaled = scaler.transform(test_input)

# 初始化SVM模型，选择RBF核函数
svm_model = svm.SVC(kernel='rbf')

# 训练模型
svm_model.fit(train_input_scaled, train_label.ravel())

# 预测测试集
predictions = svm_model.predict(test_input_scaled)

# 计算并打印准确率
accuracy = accuracy_score(test_label, predictions)
print(f"SVM with RBF Kernel Accuracy: {accuracy}")

SVM with RBF Kernel Accuracy: 0.9993

简易神经网络

In [13]:

import torch
import torch.nn as nn
import torch.optim as optim

# 将numpy数组转换为torch张量
train_input_tensor = torch.FloatTensor(train_input)
train_label_tensor = torch.LongTensor(train_label)
test_input_tensor = torch.FloatTensor(test_input)
test_label_tensor = torch.LongTensor(test_label)

# 定义神经网络结构
class SimpleClassifier(nn.Module):
    def __init__(self, input_dim, hidden_dim, output_dim):
        super(SimpleClassifier, self).__init__()
        self.layer1 = nn.Linear(input_dim, hidden_dim)
        self.layer2 = nn.Linear(hidden_dim, output_dim)
        
    def forward(self, x):
        x = torch.relu(self.layer1(x))
        x = self.layer2(x)
        return x

# 实例化模型、损失函数和优化器
input_dim = train_input.shape[1]
hidden_dim = 10
output_dim = 2  # 假设是二分类问题
model1 = SimpleClassifier(input_dim, hidden_dim, output_dim)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model1.parameters(), lr=0.01)

# 训练神经网络
num_epochs = 100
for epoch in range(num_epochs):
    # 前向传播
    outputs = model1(train_input_tensor)
    loss = criterion(outputs, train_label_tensor)
    
    # 反向传播和优化
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if (epoch+1) % 10 == 0:
        print(f'Epoch [{epoch+1}/{num_epochs}], Loss: {loss.item()}')

# 测试集评估
with torch.no_grad():
    outputs = model1(test_input_tensor)
    _, predicted = torch.max(outputs.data, 1)
    accuracy = (predicted == test_label_tensor).sum().item() / len(test_label_tensor)
    print(f"Neural Network Accuracy: {accuracy}")

Epoch [10/100], Loss: 0.6714565753936768
Epoch [20/100], Loss: 0.519657552242279
Epoch [30/100], Loss: 0.4016408622264862
Epoch [40/100], Loss: 0.3351301848888397
Epoch [50/100], Loss: 0.30125096440315247
Epoch [60/100], Loss: 0.2788920998573303
Epoch [70/100], Loss: 0.26166898012161255
Epoch [80/100], Loss: 0.24820241332054138
Epoch [90/100], Loss: 0.23750551044940948
Epoch [100/100], Loss: 0.22848664224147797
Neural Network Accuracy: 0.9034

KAN训练

In [14]:

model = KAN(width=[2,2], grid=3, k=3)

def train_acc():
    return torch.mean((torch.argmax(model(dataset['train_input']), dim=1) == dataset['train_label']).float())

def test_acc():
    return torch.mean((torch.argmax(model(dataset['test_input']), dim=1) == dataset['test_label']).float())

results = model.train(dataset, opt="LBFGS", steps=20, metrics=(train_acc, test_acc), loss_fn=torch.nn.CrossEntropyLoss());

自动符号回归

In [15]:

lib = ['x','x^2','x^3','x^4','exp','log','sqrt','tanh','sin','abs']
model.auto_symbolic(lib=lib)

fixing (0,0,0) with x^3, r2=0.9097426990326932
fixing (0,0,1) with tanh, r2=0.9344546561832844
fixing (0,1,0) with tanh, r2=0.9959303072149012
fixing (0,1,1) with exp, r2=0.975737216980336

formula1, formula2 = model.symbolic_formula()[0]
formula1

In [17]:

formula2

301.53

这个公式有多准确？

In [18]:

def acc(formula1, formula2, X, y):
    batch = X.shape[0]
    correct = 0
    for i in range(batch):
        logit1 = np.array(formula1.subs('x_1', X[i,0]).subs('x_2', X[i,1])).astype(np.float64)
        logit2 = np.array(formula2.subs('x_1', X[i,0]).subs('x_2', X[i,1])).astype(np.float64)
        correct += (logit2 > logit1) == y[i]
    return correct/batch

print('train acc of the formula:', acc(formula1, formula2, dataset['train_input'], dataset['train_label']))
print('test acc of the formula:', acc(formula1, formula2, dataset['test_input'], dataset['test_label']))

train acc of the formula: tensor(0.8790)
test acc of the formula: tensor(0.8790)

小结

可以下一个初步的使用感受：

1. pykan看得出是一个学者制作的python库，

它更加倾向于探索数学物理定律，希望这个库能够为我所用，在气象领域有所建树 2. 正如大家所说，它的训练过程是较慢的，使用过程是非常吃资源的，但是它的版本目前为0.0.5，未来可期 3. 有没有到能颠覆业界的水平，以我浅薄的认知无法下判断，反正业界一三五要被颠覆，二四六被震惊 4. 在分类问题上准确率虽然被向量机吊打，但是能给出公式，这算是在可解释性上迈出一步了（当然准确率也可能是数据集的问题） 5. 限于个人水平，在此仅仅做简单测试，如有错漏，还请指正

本文参与腾讯云自媒体同步曝光计划，分享自微信公众号。

原始发表：2024-05-13，如有侵权请联系 cloudcommunity@tencent.com 删除

github