神经符号模型与增量学习——下一代可控人工智能的正统新范式

目录

1. 神经符号模型——DAHSF
2. 论文的Future Work：用RPA程度的增量学习方法训练DAHSF——智能模型的本地训练与本地部署——渐得如意智能自动化办公平台Introduction of DAHSF
3. Paper Link
4. Hugging Face
5. Podcast

Abstract

Text Normalization and Semantic Parsing have numerous applications in natural language processing, such as natural language programming, paraphrasing, data augmentation, constructing expert systems, text matching, and more. Despite the prominent achievements of deep learning in Large Language Models (LLMs), the interpretability of neural network architectures is still poor, which affects their credibility and hence limits the deployments of risk-sensitive scenarios. In certain scenario-specific domains with scarce data, rapidly obtaining a large number of supervised learning labels is challenging, and the workload of manually labeling data would be enormous. Catastrophic forgetting in neural networks further leads to low data utilization rates. In situations where swift responses are vital, the density of the model makes local deployment difficult and the response time long, which is not conducive to local applications of these fields. Inspired by the multiplication rule, a principle of combinatorial mathematics, and human thinking patterns, a multilayer framework along with its algorithm, the Digestion Algorithm in Hierarchical Symbolic Forests (DAHSF), is proposed to address these above issues, combining text normalization and semantic parsing workflows. The Chinese Scripting Language "Fire Bunny Intelligent Development Platform V2.0" is an important test and application of the technology discussed in this paper. DAHSF can run locally in scenario-specific domains on little datasets, with model size and memory usage optimized by at least two orders of magnitude, thus improving the execution speed, and possessing a promising optimization outlook.Significant DeclarationDue to very limited resources (esp. equipments), the data (not including DAHSF) in this table was generated by ChatGLM-4.

Podcast Transcript

John: Welcome to our seminar on Alternative Models in Natural Language Processing. Today's lecture is on the paper 'DIGESTION ALGORITHM IN HIERARCHICAL SYMBOLIC FORESTS' by Kevin You. We've seen a lot of work on optimizing LLMs, like in 'From Large to Super-Tiny', but this paper takes a step back from the deep learning paradigm entirely. It proposes a symbolic, rule-based approach, which is interesting given the field's current trajectory. The author appears to be an independent researcher, which provides a unique context for the work.

John: Yes, Noah?

Noah: Excuse me, Professor. You mentioned this is from an independent researcher. How does that context, especially the link to a personal blog and a copyrighted software platform, shape our interpretation of this as an academic contribution versus an engineering report?

John: That's an excellent point. It suggests the work is heavily application-driven. The primary motivation appears to be solving a practical problem—creating a lightweight, locally deployable NLP engine for a specific software product. While it's presented as research, we should evaluate it through the lens of a highly specialized, engineered solution that directly challenges the assumption that LLMs are the only path forward.

John: The main concept here is the Digestion Algorithm in Hierarchical Symbolic Forests, or DAHSF. The author positions it as a direct response to the key weaknesses of LLMs: their lack of interpretability, poor performance in data-scarce domains, and immense computational requirements. Instead of a neural network, the DAHSF uses a multi-layered symbolic framework. Each layer is a 'forest' where every node and edge has an explicit, human-understandable meaning. This design choice is fundamental to its interpretability.

Noah: So, is this essentially a modern, structured version of an expert system?

John: In a way, yes. It's a rule-based system that relies on predefined lexicons. The 'Digestion Algorithm' is the process of mapping input text to standardized forms. It uses equivalence classes to handle synonyms, mapping varied inputs to a single representative element. For example, 'start,' 'launch,' and 'open' might all be mapped to a single command. The system then parses the remaining text to identify keywords and data words, like a program name or a URL. This layered approach allows it to handle increasing levels of abstraction, much like human cognition.

Noah: How does a deterministic system like this handle the inherent ambiguity of natural language, something probabilistic models like LLMs are designed for?

John: It handles ambiguity by being domain-specific. The lexicons and rules are tailored for a particular scenario, which sharply reduces the scope of possible meanings. The paper's example is the 'Fire Bunny Intelligent Development Platform'. In that context, the system isn't trying to understand all of human language, just commands relevant to controlling a computer. The second layer of its architecture is dedicated to contextual analysis to resolve ambiguities that persist after the initial normalization, like omissions common in Chinese.

John: Let's look at the application. The paper uses the 'Fire Bunny' platform to demonstrate the DAHSF in a two-layer pipeline. The first layer acts as a 'perceiver'. It takes raw text like '打开www.baidu.com'—which means 'Open www.baidu.com'—and performs initial normalization. It identifies '打开' as a keyword and maps it to its core synonym, '开'. It then identifies 'www.baidu.com' as a data word and labels it as a URL. The output is a structured tuple.

Noah: So the first layer tokenizes and standardizes?

John: Correct. Then, the second layer performs contextual analysis on these structured tuples. It matches the normalized command structure with the identified data words to form a complete, executable instruction. This is how it achieves natural language programming. The performance results are the most notable part of the paper. The author reports that the model is just 1.58 megabytes on disk and uses under 10 megabytes of memory for parsing, which is orders of magnitude smaller and more efficient than even compact LLMs. Execution speed is described as 'nearly imperceptible'.

Noah: A quick question on the evaluation. The paper says the test set was generated by simulating user inputs with ChatGLM-4. Isn't there a risk of creating a biased benchmark? An LLM might generate inputs that are syntactically simple or repetitive, which could favor a rule-based system and not reflect the complexity of real human input.

John: That's a valid critique of the methodology. Using an LLM to generate test data for a system designed to be an alternative to LLMs can introduce biases. A more robust evaluation would involve a dataset of genuine, unscripted user commands. However, for a proof-of-concept from an independent developer, this approach provides a baseline for comparison, and the performance metrics on speed and resource usage are significant regardless of the test data's origin. They clearly demonstrate the feasibility of local deployment on consumer hardware.

John: The broader implications are quite interesting. This work directly addresses the need for interpretable AI in risk-sensitive fields. In scenarios like industrial control or medical systems, a transparent, deterministic decision process is often preferable to a black-box model, even if the latter is more flexible. The paper also champions natural language programming as a way to lower barriers, especially for non-English speakers, by breaking the 'monopoly of English computer languages'. This connects to a larger conversation in the field about making technology more accessible.

Noah: The emphasis on interpretability and reasoning reminds me of the goals in neurosymbolic research, like the work on 'Proof of Thought'. Does the DAHSF have any capacity for learning or adaptation, or is it entirely static until the rules are manually updated?

John: As presented, the core system is static and relies on manually curated lexicons. This avoids issues like catastrophic forgetting seen in LLMs, which is a key point in the paper. However, the 'Future Works' section outlines a vision for a self-learning symbolic system. The plan is to use generative AI to create test cases and automatically update the knowledge base and lexicons from failures. This points toward a hybrid approach, using LLMs as a tool to bootstrap and refine a symbolic reasoning engine, which is a compelling research direction.

John: So, to wrap up, the DAHSF is not positioned as a replacement for general-purpose LLMs. Instead, it serves as a strong proof-of-concept for an alternative approach tailored to specific, resource-constrained scenarios where speed, low overhead, and interpretability are paramount. It demonstrates that for certain NLP tasks like command parsing, a well-engineered symbolic system can significantly outperform deep learning models. The key takeaway is that the optimal architecture is highly dependent on the problem context, and older AI paradigms still have much to offer.

John: Thanks for listening. If you have any further questions, ask us or drop a comment.

Gradual Magic Intelligent Office Automation Platform

It's our second step/project of the Office Agents and Their Incremental Learning Framework series.