人工智能学术速递[8.17]

cs.AI人工智能，共计59篇

【1】 APReL: A Library for Active Preference-based Reward Learning Algorithms 标题：APREL：一个基于主动偏好的奖励学习算法库 链接：https://arxiv.org/abs/2108.07259

作者：Erdem Bıyı k,Aditi Talati,Dorsa Sadigh 机构： Department of Electrical Engineering, Stanford University, Department of Computer Science, Stanford University 备注：5 pages, 1 figures. Library is available at: this https URL 摘要：奖励学习是机器人学中的一个基本问题，即让机器人按照人类用户的需求进行操作。许多基于偏好的学习算法和主动查询技术已经被提出来解决这个问题。在本文中，我们介绍了APReL，一个基于主动偏好的奖励学习算法库，它使研究人员和实践者能够使用现有技术进行实验，并轻松地为问题的各个模块开发自己的算法。 摘要：Reward learning is a fundamental problem in robotics to have robots that operate in alignment with what their human user wants. Many preference-based learning algorithms and active querying techniques have been proposed as a solution to this problem. In this paper, we present APReL, a library for active preference-based reward learning algorithms, which enable researchers and practitioners to experiment with the existing techniques and easily develop their own algorithms for various modules of the problem.

【2】 On the Opportunities and Risks of Foundation Models 标题：论基础模型的机遇与风险 链接：https://arxiv.org/abs/2108.07258

作者：Rishi Bommasani,Drew A. Hudson,Ehsan Adeli,Russ Altman,Simran Arora,Sydney von Arx,Michael S. Bernstein,Jeannette Bohg,Antoine Bosselut,Emma Brunskill,Erik Brynjolfsson,Shyamal Buch,Dallas Card,Rodrigo Castellon,Niladri Chatterji,Annie Chen,Kathleen Creel,Jared Quincy Davis,Dora Demszky,Chris Donahue,Moussa Doumbouya,Esin Durmus,Stefano Ermon,John Etchemendy,Kawin Ethayarajh,Li Fei-Fei,Chelsea Finn,Trevor Gale,Lauren Gillespie,Karan Goel,Noah Goodman,Shelby Grossman,Neel Guha,Tatsunori Hashimoto,Peter Henderson,John Hewitt,Daniel E. Ho,Jenny Hong,Kyle Hsu,Jing Huang,Thomas Icard,Saahil Jain,Dan Jurafsky,Pratyusha Kalluri,Siddharth Karamcheti,Geoff Keeling,Fereshte Khani,Omar Khattab,Pang Wei Koh,Mark Krass,Ranjay Krishna,Rohith Kuditipudi,Ananya Kumar,Faisal Ladhak,Mina Lee,Tony Lee,Jure Leskovec,Isabelle Levent,Xiang Lisa Li,Xuechen Li,Tengyu Ma,Ali Malik,Christopher D. Manning,Suvir Mirchandani,Eric Mitchell,Zanele Munyikwa,Suraj Nair,Avanika Narayan,Deepak Narayanan,Ben Newman,Allen Nie,Juan Carlos Niebles,Hamed Nilforoshan,Julian Nyarko,Giray Ogut,Laurel Orr,Isabel Papadimitriou,Joon Sung Park,Chris Piech,Eva Portelance,Christopher Potts,Aditi Raghunathan,Rob Reich,Hongyu Ren,Frieda Rong,Yusuf Roohani,Camilo Ruiz,Jack Ryan,Christopher Ré,Dorsa Sadigh,Shiori Sagawa,Keshav Santhanam,Andy Shih,Krishnan Srinivasan,Alex Tamkin,Rohan Taori,Armin W. Thomas,Florian Tramèr,Rose E. Wang,William Wang,Bohan Wu,Jiajun Wu,Yuhuai Wu,Sang Michael Xie,Michihiro Yasunaga,Jiaxuan You,Matei Zaharia,Michael Zhang,Tianyi Zhang,Xikun Zhang,Yuhui Zhang,Lucia Zheng,Kaitlyn Zhou,Percy Liang 机构：Dorottya Demszky, Center for Research on Foundation Models (CRFM) — Stanford University 备注：Published by the Center for Research on Foundation Models (this https URL) 摘要：人工智能正在经历一场范式转换，模型（例如，BERT、DALL-e、GPT-3）的兴起，这些模型在大规模的大数据上进行训练，并且能够适应广泛的下游任务。我们把这些模型称为基础模型来强调它们的中心性和不完整性。该报告提供了基础模型的机会和风险，包括从能力（例如，语言、视觉、机器人、推理、人类交互）和技术原理（例如，模型体系结构、训练程序、数据、系统、安全性、评估、理论）到它们的应用（例如，法律）。医疗、教育）和社会影响（例如，不平等、滥用、经济和环境影响、法律和道德考虑）。虽然基础模型是基于传统的深度学习和迁移学习，但它们的规模导致新的应急能力，并且它们在许多任务上的有效性都会激励均质化。均质化提供了强大的杠杆作用，但需要谨慎，因为基础模型的缺陷是由下游所有适应模型所继承的。尽管迫在眉睫的广泛部署的基础模型，我们目前还没有一个明确的了解，他们如何工作，当他们失败，以及他们甚至能够由于其紧急性质。为了解决这些问题，我们相信对基础模型的大量研究将需要与他们的基本社会技术本质相称的深刻的跨学科合作。 摘要：AI is undergoing a paradigm shift with the rise of models (e.g., BERT, DALL-E, GPT-3) that are trained on broad data at scale and are adaptable to a wide range of downstream tasks. We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models, ranging from their capabilities (e.g., language, vision, robotics, reasoning, human interaction) and technical principles (e.g., model architectures, training procedures, data, systems, security, evaluation, theory) to their applications (e.g., law, healthcare, education) and societal impact (e.g., inequity, misuse, economic and environmental impact, legal and ethical considerations). Though foundation models are based on conventional deep learning and transfer learning, their scale results in new emergent capabilities, and their effectiveness across so many tasks incentivizes homogenization. Homogenization provides powerful leverage but demands caution, as the defects of the foundation model are inherited by all the adapted models downstream. Despite the impending widespread deployment of foundation models, we currently lack a clear understanding of how they work, when they fail, and what they are even capable of due to their emergent properties. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration commensurate with their fundamentally sociotechnical nature.

【3】 Patch Attack Invariance: How Sensitive are Patch Attacks to 3D Pose? 标题：补丁攻击不变性：补丁攻击对3D姿势有多敏感？ 链接：https://arxiv.org/abs/2108.07229

作者：Max Lennon,Nathan Drenkow,Philippe Burlina 机构：The Johns Hopkins University Applied Physics Laboratory, Johns Hopkins Road, Laurel, Maryland 摘要：基于扰动的攻击虽然在物理上无法实现，但一直是对抗式机器学习（ML）研究的重点。相比之下，基于补丁的攻击在物理上是可以实现的，但大多数工作都集中在2D领域，最近又涉足3D领域。描述面片攻击的鲁棒性及其对三维姿态的不变性是重要的，但尚未完全阐明，这是本文的重点。为此，本文做出了以下几点贡献：A）我们开发了一种新的度量，称为变换平均攻击成功率（mAST），用于评估补丁攻击的鲁棒性和不变性；和B），我们系统地评估了补丁攻击在各种条件下对3D位置和方向的鲁棒性；特别是，我们进行了敏感性分析，该分析提供了关于攻击有效性的重要定性见解，作为面片相对于相机的3D姿势（旋转、平移）的函数，并阐述了面片攻击3D不变性的一些特性；和C），我们得出了新的定性结论，包括：1）我们证明，对于一些3D变换，即旋转和织布机，增加训练分布支持可以提高测试时整个范围内的补丁成功率。2） 我们提供了一个新的见解，以了解面片攻击有效性的基本截止极限的存在，该极限取决于面外旋转角度的范围。这些发现将共同指导3D补丁攻击和防御的未来设计。 摘要：Perturbation-based attacks, while not physically realizable, have been the main emphasis of adversarial machine learning (ML) research. Patch-based attacks by contrast are physically realizable, yet most work has focused on 2D domain with recent forays into 3D. Characterizing the robustness properties of patch attacks and their invariance to 3D pose is important, yet not fully elucidated, and is the focus of this paper. To this end, several contributions are made here: A) we develop a new metric called mean Attack Success over Transformations (mAST) to evaluate patch attack robustness and invariance; and B), we systematically assess robustness of patch attacks to 3D position and orientation for various conditions; in particular, we conduct a sensitivity analysis which provides important qualitative insights into attack effectiveness as a function of the 3D pose of a patch relative to the camera (rotation, translation) and sets forth some properties for patch attack 3D invariance; and C), we draw novel qualitative conclusions including: 1) we demonstrate that for some 3D transformations, namely rotation and loom, increasing the training distribution support yields an increase in patch success over the full range at test time. 2) We provide new insights into the existence of a fundamental cutoff limit in patch attack effectiveness that depends on the extent of out-of-plane rotation angles. These findings should collectively guide future design of 3D patch attacks and defenses.

【4】 Hierarchical Infinite Relational Model 标题：分层无限关系模型 链接：https://arxiv.org/abs/2108.07208

作者：Feras A. Saad,Vikash K. Mansinghka 机构： Massachusetts Institute of Technology, Cambridge, MA, USA 备注：11 pages, 6 figures, 4 tables. Appearing in UAI 2021 摘要：本文描述了分层无限关系模型（HIRM），这是一种针对噪声、稀疏和异构关系数据的新概率生成模型。给定一组定义在一组域上的关系，该模型首先使用顶级中餐馆流程推断出多个不重叠的关系簇。在每个关系簇中，使用Dirichlet过程混合对域实体进行划分，并对关系值的概率分布进行建模。HIRM概括了标准的无限关系模型，可用于各种数据分析任务，包括相关性检测、聚类和密度估计。本文提出了一种新的基于Gibbs抽样的完全贝叶斯后验推理算法。我们在20个对象属性数据集的密度估计基准上展示了该方法的有效性，这些数据集包含多达1800万个单元，并使用它来发现来自政治学和基因组学的真实世界数据集中的关系结构。 摘要：This paper describes the hierarchical infinite relational model (HIRM), a new probabilistic generative model for noisy, sparse, and heterogeneous relational data. Given a set of relations defined over a collection of domains, the model first infers multiple non-overlapping clusters of relations using a top-level Chinese restaurant process. Within each cluster of relations, a Dirichlet process mixture is then used to partition the domain entities and model the probability distribution of relation values. The HIRM generalizes the standard infinite relational model and can be used for a variety of data analysis tasks including dependence detection, clustering, and density estimation. We present new algorithms for fully Bayesian posterior inference via Gibbs sampling. We illustrate the efficacy of the method on a density estimation benchmark of twenty object-attribute datasets with up to 18 million cells and use it to discover relational structure in real-world datasets from politics and genomics.

【5】 The Emergence of Wireless MAC Protocols with Multi-Agent Reinforcement Learning 标题：基于多Agent强化学习的无线MAC协议的出现 链接：https://arxiv.org/abs/2108.07144

作者：Mateus P. Mota,Alvaro Valcarce,Jean-Marie Gorce,Jakob Hoydis 机构：∗Nokia Bell Labs, Nozay, France, †National Institute of Applied Sciences, Lyon, France, ‡NVIDIA, Paris, France 备注：Submitted to Globecom 2021 Workshop on Wireless Communications for Distributed Intelligence 摘要：在本文中，我们提出了一个新的框架，利用多代理深度确定性策略梯度（MADDPG）算法，使基站（BS）和用户设备（UE）能够在多址场景中提出媒体访问控制（MAC）协议。在该框架中，BS和ue是强化学习（RL）代理，需要学习合作以交付数据。网络节点可以交换控制消息以在网络上协作和传递数据，但无需事先就控制消息的含义达成任何协议。在这样的框架下，代理不仅要学习信道访问策略，还要学习信令策略。通过将所提出的算法与删除代理之间通信或中心批评家的删除版本进行比较，表明代理之间的协作是重要的。与无争用基线的比较表明，我们的框架在goodput方面取得了优异的性能，并且可以有效地用于学习新的协议。 摘要：In this paper, we propose a new framework, exploiting the multi-agent deep deterministic policy gradient (MADDPG) algorithm, to enable a base station (BS) and user equipment (UE) to come up with a medium access control (MAC) protocol in a multiple access scenario. In this framework, the BS and UEs are reinforcement learning (RL) agents that need to learn to cooperate in order to deliver data. The network nodes can exchange control messages to collaborate and deliver data across the network, but without any prior agreement on the meaning of the control messages. In such a framework, the agents have to learn not only the channel access policy, but also the signaling policy. The collaboration between agents is shown to be important, by comparing the proposed algorithm to ablated versions where either the communication between agents or the central critic is removed. The comparison with a contention-free baseline shows that our framework achieves a superior performance in terms of goodput and can effectively be used to learn a new protocol.

【6】 Efficient Feature Representations for Cricket Data Analysis using Deep Learning based Multi-Modal Fusion Model 标题：基于深度学习的多模态融合模型在板球数据分析中的高效特征表示 链接：https://arxiv.org/abs/2108.07139

作者：Souridas Alaka,Rishikesh Sreekumar,Hrithwik Shalu 机构：Indian Institute of Technology Madras, India 摘要：数据分析已成为现代板球运动的必要条件。从有效的团队管理到比赛获胜预测，一切都使用某种形式的分析。有效分析数据需要有意义的数据表示。在本研究中，我们研究了自适应（可学习）嵌入的使用，以表示相互关联的特征（如球员、团队等）。本研究使用的数据来自经典的T20锦标赛IPL（印度超级联赛）。为了自然地促进学习有意义的特征表示以进行准确的数据分析，我们制定了一个深度表示学习框架，该框架通过最小化对比损失来共同学习一组自定义嵌入（代表我们感兴趣的特征）。我们的目标是根据一局的总跑动率通过分层聚类得到的一组类。据评估，该框架可确保获得的嵌入具有更大的通用性，在此基础上，对总体运行率预测进行了基于任务的分析，以显示该框架的可靠性。 摘要：Data analysis has become a necessity in the modern era of cricket. Everything from effective team management to match win predictions use some form of analytics. Meaningful data representations are necessary for efficient analysis of data. In this study we investigate the use of adaptive (learnable) embeddings to represent inter-related features (such as players, teams, etc). The data used for this study is collected from a classical T20 tournament IPL (Indian Premier League). To naturally facilitate the learning of meaningful representations of features for accurate data analysis, we formulate a deep representation learning framework which jointly learns a custom set of embeddings (which represents our features of interest) through the minimization of a contrastive loss. We base our objective on a set of classes obtained as a result of hierarchical clustering on the overall run rate of an innings. It's been assessed that the framework ensures greater generality in the obtained embeddings, on top of which a task based analysis of overall run rate prediction was done to show the reliability of the framework.

【7】 Autoencoders as Tools for Program Synthesis 标题：作为程序综合工具的自动编码器 链接：https://arxiv.org/abs/2108.07129

作者：Sander de Bruin,Vadim Liventsev,Milan Petković 机构：Eindhoven University of Technology, Eindhoven, The Netherlands, Milan Petkovic 备注：Source code is available at this https URL 摘要：最近，源代码语言建模的研究取得了许多进展。应用范围从代码建议和完成到代码摘要。然而，工业级编程语言的完整程序综合还没有得到广泛的研究。在这项工作中，我们介绍了一种用于工业级编程语言程序综合的变分自动编码器模型。我们的模型结合了源代码的内部层次结构，并对解析树进行操作。通过学习树上源代码的潜在表示，我们可以捕获更多信息，并获得比标准自回归自动编码器模型更高的性能。此外，由于我们模型的树结构性质，在树的路径上执行自回归操作，而不是线性序列。因此，自回归模型处理的序列大小与树的宽度和深度成比例，而不是与树的总大小成比例，这缓解了梯度爆炸和消失的常见问题。 摘要：Recently there have been many advances in research on language modeling of source code. Applications range from code suggestion and completion to code summarization. However, complete program synthesis of industry-grade programming languages has not been researched extensively. In this work, we introduce a variational autoencoder model for program synthesis of industry-grade programming languages. Our model incorporates the internal hierarchical structure of source codes and operates on parse trees. By learning a latent representation of source code over trees, we capture more information and achieve a higher performance than standard autoregressive autoencoder models. Furthermore, due to the tree-structured nature of our model, the autoregressive operations are performed on paths of trees instead of linear sequences. Therefore, the size of the sequences that the autoregressive model processes, scales proportionally to the width and depth of the tree instead of the total size of the tree which mitigates the common problem of exploding and vanishing gradients.

【8】 AIREX: Neural Network-based Approach for Air Quality Inference in Unmonitored Cities 标题：AIREX：基于神经网络的非监测城市空气质量推断方法 链接：https://arxiv.org/abs/2108.07120

作者：Yuya Sasaki,Kei Harada,Shohei Yamasaki,Makoto Onizuka 机构：Osaka university 摘要：城市空气污染是影响人类健康和生活质量的主要环境问题。已建立监测站，以不断获取空气质量信息，但监测站并不覆盖所有地区。因此，有许多方法用于空间细粒度空气质量推断。由于现有方法的目的是仅推断受监测城市中各地点的空气质量，因此它们不假设推断未受监测城市的空气质量。在本文中，我们首先研究了无监测城市的空气质量推断。为了准确推断未受监测城市的空气质量，我们提出了一种基于神经网络的AIREX方法。AIREX的创新之处在于采用了一种混合专家方法，这是一种基于分治原理的机器学习技术，用于学习多个城市之间空气质量的相关性。为了进一步提高性能，它采用注意机制来计算从受监测城市到未受监测城市位置的空气质量推断的影响。我们通过对真实空气质量数据集的实验表明，AIREX比最先进的方法具有更高的精度。 摘要：Urban air pollution is a major environmental problem affecting human health and quality of life. Monitoring stations have been established to continuously obtain air quality information, but they do not cover all areas. Thus, there are numerous methods for spatially fine-grained air quality inference. Since existing methods aim to infer air quality of locations only in monitored cities, they do not assume inferring air quality in unmonitored cities. In this paper, we first study the air quality inference in unmonitored cities. To accurately infer air quality in unmonitored cities, we propose a neural network-based approach AIREX. The novelty of AIREX is employing a mixture-of-experts approach, which is a machine learning technique based on the divide-and-conquer principle, to learn correlations of air quality between multiple cities. To further boost the performance, it employs attention mechanisms to compute impacts of air quality inference from the monitored cities to the locations in the unmonitored city. We show, through experiments on a real-world air quality dataset, that AIREX achieves higher accuracy than state-of-the-art methods.

【9】 Creating and Querying Personalized Versions ofWikidata on a Laptop 标题：在笔记本电脑上创建和查询维基数据的个性化版本 链接：https://arxiv.org/abs/2108.07119

作者：Hans Chalupsky,Pedro Szekely,Filip Ilievski,Daniel Garijo,Kartik Shenoy 机构： Information Sciences Institute, University of Southern California, Ontology Engineering Group, Universidad Polit´ecnica de Madrid 摘要：今天，应用程序开发人员有三种选择来利用Wikidata中的知识：他们可以下载JSON或RDF格式的Wikidata转储，他们可以使用Wikidata API获取有关单个实体的数据，或者他们可以使用Wikidata SPARQL端点。这些方法都不能支持复杂但常见的查询用例，例如检索大量数据或对大量Wikidata进行聚合。本文介绍了KGTK Kypher，一种查询语言和处理器，允许用户在笔记本电脑上创建Wikidata的个性化变体。我们展示了几个用例，这些用例说明了Kypher允许用户在笔记本电脑上运行完整的Wikidata KG，并结合来自外部资源（如DBpedia）的数据进行分析的类型。这些用例的Kypher查询在笔记本电脑上的运行速度比在Wikidata克隆上运行的具有24小时超时限制的强大服务器上的等效SPARQL查询要快得多。 摘要：Application developers today have three choices for exploiting the knowledge present in Wikidata: they can download the Wikidata dumps in JSON or RDF format, they can use the Wikidata API to get data about individual entities, or they can use the Wikidata SPARQL endpoint. None of these methods can support complex, yet common, query use cases, such as retrieval of large amounts of data or aggregations over large fractions of Wikidata. This paper introduces KGTK Kypher, a query language and processor that allows users to create personalized variants of Wikidata on a laptop. We present several use cases that illustrate the types of analyses that Kypher enables users to run on the full Wikidata KG on a laptop, combining data from external resources such as DBpedia. The Kypher queries for these use cases run much faster on a laptop than the equivalent SPARQL queries on a Wikidata clone running on a powerful server with 24h time-out limits.

【10】 NIST SRE CTS Superset: A large-scale dataset for telephony speaker recognition 标题：NIST SRE CTS超集：用于电话说话人识别的大规模数据集 链接：https://arxiv.org/abs/2108.07118

作者：Seyed Omid Sadjadi 摘要：本文件简要介绍了美国国家标准与技术研究所（NIST）说话人识别评估（SRE）对话电话语音（CTS）超集。创建CTS超集是为了向研究界提供大规模数据集和统一元数据，这些元数据可用于有效训练和开发电话（窄带）说话人识别系统。它包含来自6800多个扬声器的大量电话语音片段，语音持续时间均匀分布在[10s，60s]范围内。这些片段是从用于编译先前SRE数据集（SRE1996-2012）的源语料库中提取的，包括灰胡子语料库以及语言数据联盟（LDC）收集的交换机和混音器系列。除了简要说明外，我们还报告了NIST 2020 CTS说话人识别挑战赛的说话人识别结果，该挑战赛是使用CTS超集训练的系统获得的。结果将作为挑战的参考基线。 摘要：This document provides a brief description of the National Institute of Standards and Technology (NIST) speaker recognition evaluation (SRE) conversational telephone speech (CTS) Superset. The CTS Superset has been created in an attempt to provide the research community with a large-scale dataset along with uniform metadata that can be used to effectively train and develop telephony (narrowband) speaker recognition systems. It contains a large number of telephony speech segments from more than 6800 speakers with speech durations distributed uniformly in the [10s, 60s] range. The segments have been extracted from the source corpora used to compile prior SRE datasets (SRE1996-2012), including the Greybeard corpus as well as the Switchboard and Mixer series collected by the Linguistic Data Consortium (LDC). In addition to the brief description, we also report speaker recognition results on the NIST 2020 CTS Speaker Recognition Challenge, obtained using a system trained with the CTS Superset. The results will serve as a reference baseline for the challenge.

【11】 Learning Canonical View Representation for 3D Shape Recognition with Arbitrary Views 标题：任意视点三维形状识别的规范视图表示学习 链接：https://arxiv.org/abs/2108.07084

作者：Xin Wei,Yifei Gong,Fudong Wang,Xing Sun 机构：Xi’an Jiaotong University, Tencent Youtu Lab 摘要：在本文中，我们着重于从任意视图（即任意数量和位置的视点）识别三维形状。对于基于视图的三维形状识别来说，这是一个具有挑战性和现实性的设置。我们提出了一种规范化的视图表示来应对这一挑战。我们首先将任意视图的原始特征转换为固定数量的视图特征，称为规范视图表示，方法是使用最佳传输将任意视图特征与一组可学习的参考视图特征对齐。通过这种方式，具有任意视图的每个三维形状由固定数量的规范视图特征表示，这些特征进一步聚合以生成用于形状识别的丰富而健壮的三维形状表示。我们还提出了一个规范视图特征分离约束，以强制将规范视图表示中的视图特征嵌入到欧氏空间中的散乱点中。在ModelNet40、ScanObjectNN和RGBD数据集上的实验表明，我们的方法在固定视点设置下取得了有竞争力的结果，并且在任意视图设置下显著优于适用的方法。 摘要：In this paper, we focus on recognizing 3D shapes from arbitrary views, i.e., arbitrary numbers and positions of viewpoints. It is a challenging and realistic setting for view-based 3D shape recognition. We propose a canonical view representation to tackle this challenge. We first transform the original features of arbitrary views to a fixed number of view features, dubbed canonical view representation, by aligning the arbitrary view features to a set of learnable reference view features using optimal transport. In this way, each 3D shape with arbitrary views is represented by a fixed number of canonical view features, which are further aggregated to generate a rich and robust 3D shape representation for shape recognition. We also propose a canonical view feature separation constraint to enforce that the view features in canonical view representation can be embedded into scattered points in a Euclidean space. Experiments on the ModelNet40, ScanObjectNN, and RGBD datasets show that our method achieves competitive results under the fixed viewpoint settings, and significantly outperforms the applicable methods under the arbitrary view setting.

【12】 Toward the Understanding of Deep Text Matching Models for Information Retrieval 标题：对信息检索中的深层文本匹配模型的理解 链接：https://arxiv.org/abs/2108.07081

作者：Lijuan Chen,Yanyan Lan,Liang Pang,Jiafeng Guo,Xueqi Cheng 机构：Sogou Inc., Beijing, China, Institute for AI Industry Research, Tsinghua University, Beijing, China, Data Intelligence System Research Center, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 摘要：语义文本匹配是信息检索中的一个关键问题。近年来，深度学习技术在这一领域得到了广泛的应用，并取得了显著的性能改进。然而，由于深度学习的可解释性差，大多数模型都是黑盒，很难理解匹配过程中发生了什么。本文旨在解决这一问题。其关键思想是测试现有的深层文本匹配方法是否满足信息检索中的一些基本启发式。具体来说，我们的研究中使用了四种启发式方法，即术语频率约束、术语区分约束、长度规范化约束和TF长度约束。由于深度匹配模型通常包含许多参数，因此很难对这些复杂函数进行理论研究。本文提出了一种实证检验方法。具体来说，我们首先构造一些查询和文档，使它们满足约束中的假设，然后测试在原始数据集上训练的深层文本匹配模型满足相应约束的扩展。此外，采用著名的基于属性的解释方法，即综合梯度法，进行详细分析和指导可行的改进。在LETOR 4.0和MS Marco上的实验结果表明，所研究的所有深层文本匹配方法，无论是基于表示的方法还是基于交互的方法，都满足上述约束条件，并且具有很高的统计概率。我们进一步将这些约束扩展到语义设置，这对于所有深度文本匹配模型来说都是更好的满足。这些实证结果清楚地解释了为什么深层文本匹配模型在信息检索中表现良好。我们相信，提出的评估方法将有助于测试未来的深层文本匹配模型。 摘要：Semantic text matching is a critical problem in information retrieval. Recently, deep learning techniques have been widely used in this area and obtained significant performance improvements. However, most models are black boxes and it is hard to understand what happened in the matching process, due to the poor interpretability of deep learning. This paper aims at tackling this problem. The key idea is to test whether existing deep text matching methods satisfy some fundamental heuristics in information retrieval. Specifically, four heuristics are used in our study, i.e., term frequency constraint, term discrimination constraint, length normalization constraints, and TF-length constraint. Since deep matching models usually contain many parameters, it is difficult to conduct a theoretical study for these complicated functions. In this paper, We propose an empirical testing method. Specifically, We first construct some queries and documents to make them satisfy the assumption in a constraint, and then test to which extend a deep text matching model trained on the original dataset satisfies the corresponding constraint. Besides, a famous attribution based interpretation method, namely integrated gradient, is adopted to conduct detailed analysis and guide for feasible improvement. Experimental results on LETOR 4.0 and MS Marco show that all the investigated deep text matching methods, both representation and interaction based methods, satisfy the above constraints with high probabilities in statistics. We further extend these constraints to the semantic settings, which are shown to be better satisfied for all the deep text matching models. These empirical findings give clear understandings on why deep text matching models usually perform well in information retrieval. We believe the proposed evaluation methodology will be useful for testing future deep text matching models.

【13】 An Effective Non-Autoregressive Model for Spoken Language Understanding 标题：一种有效的口语理解非自回归模型 链接：https://arxiv.org/abs/2108.07005

作者：Lizhi Cheng,Weijia Jia,Wenmian Yang 机构：Department of Computer Science and, Engineering, Shanghai Jiao Tong University, Shanghai, PR China, BNU-UIC Institute of Artificial, Intelligence and Future Networks, Beijing Normal University (BNU, Zhuhai), Guangdong Key Lab of AI, and Multi-Modal Data Processing 摘要：口语理解（SLU）是面向任务的对话系统的核心组成部分，由于人类的不耐烦，它期望推理延迟更短。非自回归SLU模型明显提高了推理速度，但由于每个时隙块之间缺乏顺序依赖信息，导致时隙问题不协调。为了弥补这一缺点，本文提出了一种新的非自回归SLU模型分层细化变换器，该模型包含一个时隙标签生成（SLG）任务和一个分层细化机制（LRM）。SLG定义为使用令牌序列和生成的插槽标签生成下一个插槽标签。使用SLG，非自回归模型可以在训练过程中有效地获取依赖信息，并且不会花费额外的时间进行推理。LRM根据Transformer的中间状态预测初步SLU结果，并利用这些结果指导最终预测。在两个公共数据集上的实验表明，我们的模型显著提高了SLU性能（总体准确率为1.5%），同时大大加快了推理过程（超过10倍）。 摘要：Spoken Language Understanding (SLU), a core component of the task-oriented dialogue system, expects a shorter inference latency due to the impatience of humans. Non-autoregressive SLU models clearly increase the inference speed but suffer uncoordinated-slot problems caused by the lack of sequential dependency information among each slot chunk. To gap this shortcoming, in this paper, we propose a novel non-autoregressive SLU model named Layered-Refine Transformer, which contains a Slot Label Generation (SLG) task and a Layered Refine Mechanism (LRM). SLG is defined as generating the next slot label with the token sequence and generated slot labels. With SLG, the non-autoregressive model can efficiently obtain dependency information during training and spend no extra time in inference. LRM predicts the preliminary SLU results from Transformer's middle states and utilizes them to guide the final prediction. Experiments on two public datasets indicate that our model significantly improves SLU performance (1.5\% on Overall accuracy) while substantially speed up (more than 10 times) the inference process over the state-of-the-art baseline.

【14】 Aegis: A Trusted, Automatic and Accurate Verification Framework for Vertical Federated Learning 标题：Aegis：一种可信、自动、准确的垂直联合学习验证框架 链接：https://arxiv.org/abs/2108.06958

作者：Cengguang Zhang,Junxue Zhang,Di Chai,Kai Chen 机构：SING Lab, Hong Kong University of Science and Technology, Clustar 备注：7 pages, International Workshop on Federated Learning for User Privacy and Data Confidentiality in Conjunction with IJCAI 2021 (FL-IJCAI'21) 摘要：垂直联合学习（VFL）利用各种隐私保护算法，例如同态加密或基于秘密共享的SecureBoost，以确保数据隐私。然而，这些算法都需要一个半诚实的安全定义，这在实际应用中引起了关注。在本文中，我们提出了Aegis，一个可信的、自动的、准确的验证框架来验证VFL作业的安全性。宙斯盾与当地各方分离，以确保框架的安全。此外，它通过将VFL作业定义为有限状态机来统一验证不同的算法，并再现整个作业以提供更精确的验证，从而自动适应不断发展的VFL算法。我们在金融和医疗数据集上使用不同的威胁模型实施和评估宙斯盾。评估结果表明：1）Aegis可以检测95%的威胁模型，2）它在总VFL作业时间的84%内提供细粒度验证结果。 摘要：Vertical federated learning (VFL) leverages various privacy-preserving algorithms, e.g., homomorphic encryption or secret sharing based SecureBoost, to ensure data privacy. However, these algorithms all require a semi-honest secure definition, which raises concerns in real-world applications. In this paper, we present Aegis, a trusted, automatic, and accurate verification framework to verify the security of VFL jobs. Aegis is separated from local parties to ensure the security of the framework. Furthermore, it automatically adapts to evolving VFL algorithms by defining the VFL job as a finite state machine to uniformly verify different algorithms and reproduce the entire job to provide more accurate verification. We implement and evaluate Aegis with different threat models on financial and medical datasets. Evaluation results show that: 1) Aegis can detect 95% threat models, and 2) it provides fine-grained verification results within 84% of the total VFL job time.

【15】 An Effective System for Multi-format Information Extraction 标题：一种高效的多格式信息抽取系统 链接：https://arxiv.org/abs/2108.06957

作者：Yaduo Liu,Longhui Zhang,Shujuan Yin,Xiaofeng Zhao,Feiliang Ren 机构：School of Computer Science and Engineering, Northeastern University, Shenyang, China 备注：NLPCC-Evaluation 2021 摘要：2021年语言与情报挑战赛中的多格式信息提取任务旨在从不同维度综合评估信息提取。它由一个多插槽关系提取子任务和两个事件提取子任务组成，这些子任务从句子级和文档级提取事件。在这里，我们描述了我们的系统，用于这个多格式信息提取竞赛任务。具体来说，对于关系抽取子任务，我们将其转换为传统的三重抽取任务，并设计了一种基于投票的方法，该方法充分利用了现有的模型。对于句子级事件提取子任务，我们将其转换为NER任务，并使用基于指针标记的方法进行提取。此外，考虑到带注释的触发器信息可能有助于事件提取，我们设计了一个辅助触发器识别模型，并使用多任务学习机制将触发器特征集成到事件提取模型中。对于文档级事件提取子任务，我们设计了一种基于编码器-解码器的方法，并提出了一种类似转换器的解码器。最后，我们的系统在该多格式信息抽取任务的测试集排行榜上排名第4，其关系抽取、句子级事件抽取和文档级事件抽取的子任务F1得分分别为79.887%、85.179%和70.828%。我们型号的代码可在{https://github.com/neukg/MultiIE}. 摘要：The multi-format information extraction task in the 2021 Language and Intelligence Challenge is designed to comprehensively evaluate information extraction from different dimensions. It consists of an multiple slots relation extraction subtask and two event extraction subtasks that extract events from both sentence-level and document-level. Here we describe our system for this multi-format information extraction competition task. Specifically, for the relation extraction subtask, we convert it to a traditional triple extraction task and design a voting based method that makes full use of existing models. For the sentence-level event extraction subtask, we convert it to a NER task and use a pointer labeling based method for extraction. Furthermore, considering the annotated trigger information may be helpful for event extraction, we design an auxiliary trigger recognition model and use the multi-task learning mechanism to integrate the trigger features into the event extraction model. For the document-level event extraction subtask, we design an Encoder-Decoder based method and propose a Transformer-alike decoder. Finally,our system ranks No.4 on the test set leader-board of this multi-format information extraction task, and its F1 scores for the subtasks of relation extraction, event extractions of sentence-level and document-level are 79.887%, 85.179%, and 70.828% respectively. The codes of our model are available at {https://github.com/neukg/MultiIE}.

【16】 TL-SDD: A Transfer Learning-Based Method for Surface Defect Detection with Few Samples 标题：TL-SDD：一种基于转移学习的小样本表面缺陷检测方法 链接：https://arxiv.org/abs/2108.06939

作者：Jiahui Cheng,Bin Guo,Jiaqi Liu,Sicong Liu,Guangzhi Wu,Yueqi Sun,Zhiwen Yu 机构：Northwestern Polytechnical University, Xi’an, China 摘要：在制造业中，表面缺陷检测在保证产品质量方面发挥着越来越重要的作用。许多深度学习方法已广泛应用于表面缺陷检测任务中，并已被证明在缺陷分类和定位方面表现良好。然而，基于深度学习的检测方法往往需要大量的数据进行训练，由于缺陷类别的分布往往是不平衡的，因此无法应用于实际的工业场景。换句话说，常见缺陷类有很多样本，而稀有缺陷类的样本非常少，这些方法很难很好地检测稀有缺陷类。为了解决不平衡分布问题，本文提出了TL-SDD：一种新的基于转移学习的表面缺陷检测方法。首先，我们采用两阶段训练方案将知识从常见缺陷类转移到罕见缺陷类。其次，我们提出了一种新的基于度量的表面缺陷检测（M-SDD）模型。该模型设计了三个模块：（1）特征提取模块：包含高层次语义信息和低层次结构信息的特征融合(2） 特征重新加权模块：将示例转换为指示特征重要性的重新加权向量(3） 距离度量模块：学习度量空间，其中缺陷通过计算到每个类别表示的距离进行分类。最后，我们在包含铝型材表面缺陷的真实数据集上验证了我们提出的方法的性能。与基线方法相比，对于罕见缺陷类别，我们提出的方法的性能提高了11.98%。 摘要：Surface defect detection plays an increasingly important role in manufacturing industry to guarantee the product quality. Many deep learning methods have been widely used in surface defect detection tasks, and have been proven to perform well in defects classification and location. However, deep learning-based detection methods often require plenty of data for training, which fail to apply to the real industrial scenarios since the distribution of defect categories is often imbalanced. In other words, common defect classes have many samples but rare defect classes have extremely few samples, and it is difficult for these methods to well detect rare defect classes. To solve the imbalanced distribution problem, in this paper we propose TL-SDD: a novel Transfer Learning-based method for Surface Defect Detection. First, we adopt a two-phase training scheme to transfer the knowledge from common defect classes to rare defect classes. Second, we propose a novel Metric-based Surface Defect Detection (M-SDD) model. We design three modules for this model: (1) feature extraction module: containing feature fusion which combines high-level semantic information with low-level structural information. (2) feature reweighting module: transforming examples to a reweighting vector that indicates the importance of features. (3) distance metric module: learning a metric space in which defects are classified by computing distances to representations of each category. Finally, we validate the performance of our proposed method on a real dataset including surface defects of aluminum profiles. Compared to the baseline methods, the performance of our proposed method has improved by up to 11.98% for rare defect classes.

【17】 Blockchain-based Trustworthy Federated Learning Architecture 标题：基于区块链的可信联邦学习体系结构 链接：https://arxiv.org/abs/2108.06912

作者：Sin Kit Lo,Yue Liu,Qinghua Lu,Chen Wang,Xiwei Xu,Hye-Young Paik,Liming Zhu 机构：∗Data, CSIRO, Sydney, Australia, †School of Computer Science and Engineering, UNSW, Sydney, Australia 摘要：联合学习（Federated learning）是一种新兴的隐私保护人工智能技术，客户（即组织或设备）在本地训练模型，并基于本地模型更新制定全局模型，而无需向外部传输本地数据。然而，联邦学习系统很难实现可信度并体现负责任的人工智能原则。特别是，由于多方利益相关者的参与和客户端数据分发的异构性，联邦学习系统面临着责任和公平性方面的挑战。为了增强联邦学习系统的可问责性和公平性，我们提出了一种基于区块链的可信联邦学习体系结构。我们首先设计了一个基于智能合约的数据模型出处注册中心，以实现可问责性。此外，我们还提出了一种加权公平数据采样器算法来增强训练数据的公平性。我们使用COVID-19 X射线检测用例来评估所提出的方法。评估结果表明，该方法在实现问责制和提高公平性方面是可行的。在模型的泛化性和准确性方面，该算法比默认的联邦学习设置具有更好的性能。 摘要：Federated learning is an emerging privacy-preserving AI technique where clients (i.e., organisations or devices) train models locally and formulate a global model based on the local model updates without transferring local data externally. However, federated learning systems struggle to achieve trustworthiness and embody responsible AI principles. In particular, federated learning systems face accountability and fairness challenges due to multi-stakeholder involvement and heterogeneity in client data distribution. To enhance the accountability and fairness of federated learning systems, we present a blockchain-based trustworthy federated learning architecture. We first design a smart contract-based data-model provenance registry to enable accountability. Additionally, we propose a weighted fair data sampler algorithm to enhance fairness in training data. We evaluate the proposed approach using a COVID-19 X-ray detection use case. The evaluation results show that the approach is feasible to enable accountability and improve fairness. The proposed algorithm can achieve better performance than the default federated learning setting in terms of the model's generalisation and accuracy.

【18】 AutoChart: A Dataset for Chart-to-Text Generation Task 标题：AutoChart：图表到文本生成任务的数据集 链接：https://arxiv.org/abs/2108.06897

作者：Jiawen Zhu,Jinye Ran,Roy Ka-wei Lee,Kenny Choo,Zhi Li 机构：Singapore University of Technology and Design, University of Saskatchewan, China Merchants Bank 摘要：图表的分析描述是一个激动人心的重要研究领域，在学术界和工业界都有许多应用。然而，这项具有挑战性的任务受到了计算语言学研究界的有限关注。本文提出了\textsf{AutoChart}，一个用于图表分析描述的大型数据集，旨在鼓励对这一重要领域进行更多研究。具体来说，我们提供了一个新的框架，可以自动生成图表及其分析描述。我们对生成的图表和描述进行了广泛的人机评估，并证明生成的文本信息丰富、连贯，并且与相应的图表相关。 摘要：The analytical description of charts is an exciting and important research area with many applications in academia and industry. Yet, this challenging task has received limited attention from the computational linguistics research community. This paper proposes \textsf{AutoChart}, a large dataset for the analytical description of charts, which aims to encourage more research into this important area. Specifically, we offer a novel framework that generates the charts and their analytical description automatically. We conducted extensive human and machine evaluations on the generated charts and descriptions and demonstrate that the generated texts are informative, coherent, and relevant to the corresponding charts.

【19】 Interpreting Attributions and Interactions of Adversarial Attacks 标题：解读对抗性攻击的归因与互动 链接：https://arxiv.org/abs/2108.06895

作者：Xin Wang,Shuyun Lin,Hao Zhang,Yufei Zhu,Quanshi Zhang 机构：Shanghai Jiao Tong University 摘要：本文旨在从对抗性干扰如何影响攻击任务的角度来解释对抗性攻击。我们根据Shapley值估计不同图像区域的属性以降低攻击成本。我们定义并量化对抗扰动像素之间的相互作用，并将整个扰动图分解为相对独立的扰动分量。对扰动图的分解表明，对抗训练的DNN比正常训练的DNN在前景中具有更多的扰动分量。此外，与正常训练的DNN相比，对抗训练的DNN有更多的成分，这主要降低了真实类别的得分。上述分析为理解对抗性攻击提供了新的见解。 摘要：This paper aims to explain adversarial attacks in terms of how adversarial perturbations contribute to the attacking task. We estimate attributions of different image regions to the decrease of the attacking cost based on the Shapley value. We define and quantify interactions among adversarial perturbation pixels, and decompose the entire perturbation map into relatively independent perturbation components. The decomposition of the perturbation map shows that adversarially-trained DNNs have more perturbation components in the foreground than normally-trained DNNs. Moreover, compared to the normally-trained DNN, the adversarially-trained DNN have more components which mainly decrease the score of the true category. Above analyses provide new insights into the understanding of adversarial attacks.

【20】 Causal Incremental Graph Convolution for Recommender System Retraining 标题：基于因果增量图卷积的推荐系统再训练 链接：https://arxiv.org/abs/2108.06889

作者：Sihao Ding,Fuli Feng,Xiangnan He,Yong Liao,Jun Shi,Yongdong Zhang 备注：submitted to TNNLS 摘要：现实世界的推荐系统需要定期接受再训练，以适应新数据。在这项工作中，我们考虑如何有效地重新训练基于图形卷积网络（GCN）的推荐模型，这是最先进的技术，用于协同推荐。为了追求高效率，我们将目标设定为只使用新数据进行模型更新，同时与完全模型再训练相比，不牺牲推荐精度。这是很容易实现的，因为交互数据参与模型构建的图形结构和模型学习的损失函数，而旧的图形结构不允许用于模型更新。为此，我们提出了一种因果增量图卷积方法，该方法由两个新的算子组成，分别称为IGC和CED来估计全图卷积的输出。特别是，我们为IGC设计了简单有效的模块，巧妙地结合了旧的表示和增量图，并有效地融合了长期和短期偏好信号。CED旨在避免不在增量图中的非活动节点的过时问题，增量图通过因果推理将新数据与非活动节点连接起来。特别是，CED通过对撞机的控制来估计新数据对非活动节点表示的因果影响。在三个真实数据集上进行的大量实验表明，与现有的再训练机制相比，精度提高，速度显著提高。 摘要：Real-world recommender system needs to be regularly retrained to keep with the new data. In this work, we consider how to efficiently retrain graph convolution network (GCN) based recommender models, which are state-of-the-art techniques for collaborative recommendation. To pursue high efficiency, we set the target as using only new data for model updating, meanwhile not sacrificing the recommendation accuracy compared with full model retraining. This is non-trivial to achieve, since the interaction data participates in both the graph structure for model construction and the loss function for model learning, whereas the old graph structure is not allowed to use in model updating. Towards the goal, we propose a \textit{Causal Incremental Graph Convolution} approach, which consists of two new operators named \textit{Incremental Graph Convolution} (IGC) and \textit{Colliding Effect Distillation} (CED) to estimate the output of full graph convolution. In particular, we devise simple and effective modules for IGC to ingeniously combine the old representations and the incremental graph and effectively fuse the long-term and short-term preference signals. CED aims to avoid the out-of-date issue of inactive nodes that are not in the incremental graph, which connects the new data with inactive nodes through causal inference. In particular, CED estimates the causal effect of new data on the representation of inactive nodes through the control of their collider. Extensive experiments on three real-world datasets demonstrate both accuracy gains and significant speed-ups over the existing retraining mechanism.

【21】 Neural Architecture Dilation for Adversarial Robustness 标题：用于对抗健壮性的神经结构扩张 链接：https://arxiv.org/abs/2108.06885

作者：Yanxi Li,Zhaohui Yang,Yunhe Wang,Chang Xu 机构： School of Computer Science, University of Sydney, Australia, Noah’s Ark Lab, Huawei Technologies, China, Key Lab of Machine Perception (MOE), Department of Machine Intelligence, Peking University, China 备注：9 pages of main text, 5 pages of appendix, 4 figures, 9 tables 摘要：在过去的几十年里，随着卷积神经网络（CNN）的结构和规模的巨大进步，它们在某些任务中很容易达到甚至超过人类的性能。然而，最近发现的CNN的一个缺点是，它们容易受到敌对攻击。尽管对抗训练可以提高CNN的对抗鲁棒性，但在标准准确性和对抗鲁棒性之间存在权衡。从神经网络体系结构的角度出发，本文旨在提高主干CNN的对抗鲁棒性，使其具有令人满意的准确性。在计算开销最小的情况下，扩容架构的引入有望与主干CNN的标准性能保持友好关系，同时追求对抗鲁棒性。对标准误差界和对抗误差界的理论分析自然激发了所提出的神经结构膨胀算法。在真实数据集和基准神经网络上的实验结果证明了该算法在平衡精度和对抗鲁棒性方面的有效性。 摘要：With the tremendous advances in the architecture and scale of convolutional neural networks (CNNs) over the past few decades, they can easily reach or even exceed the performance of humans in certain tasks. However, a recently discovered shortcoming of CNNs is that they are vulnerable to adversarial attacks. Although the adversarial robustness of CNNs can be improved by adversarial training, there is a trade-off between standard accuracy and adversarial robustness. From the neural architecture perspective, this paper aims to improve the adversarial robustness of the backbone CNNs that have a satisfactory accuracy. Under a minimal computational overhead, the introduction of a dilation architecture is expected to be friendly with the standard performance of the backbone CNN while pursuing adversarial robustness. Theoretical analyses on the standard and adversarial error bounds naturally motivate the proposed neural architecture dilation algorithm. Experimental results on real-world datasets and benchmark neural networks demonstrate the effectiveness of the proposed algorithm to balance the accuracy and adversarial robustness.

【22】 AdaCon: Adaptive Context-Aware Object Detection for Resource-Constrained Embedded Devices 标题：AdaCon：资源受限嵌入式设备的自适应上下文感知对象检测 链接：https://arxiv.org/abs/2108.06850

作者：Marina Neseem,Sherief Reda 机构：School of Engineering, Brown University, Providence, RI 备注：9 pages, 6 figures, 2021 IEEE/ACM International Conference on Computer-Aided Design (ICCAD 2021) 摘要：卷积神经网络在目标检测任务中达到了最先进的精度。然而，它们有巨大的计算和能源需求，这对它们在资源受限的边缘设备上的部署提出了挑战。对象检测将图像作为输入，并识别现有对象类及其在图像中的位置。在本文中，我们利用关于不同对象类别联合出现概率的先验知识来提高对象检测模型的效率。特别是，我们的技术基于对象的空间共现概率对对象类别进行聚类。我们使用这些集群来设计自适应网络。在运行期间，分支控制器根据输入帧的空间上下文决定要执行的网络部分。我们使用COCO数据集进行的实验表明，我们的自适应对象检测模型实现了高达45%的能耗降低，高达27%的延迟降低，并且对象检测的平均精度（AP）损失很小。 摘要：Convolutional Neural Networks achieve state-of-the-art accuracy in object detection tasks. However, they have large computational and energy requirements that challenge their deployment on resource-constrained edge devices. Object detection takes an image as an input, and identifies the existing object classes as well as their locations in the image. In this paper, we leverage the prior knowledge about the probabilities that different object categories can occur jointly to increase the efficiency of object detection models. In particular, our technique clusters the object categories based on their spatial co-occurrence probability. We use those clusters to design an adaptive network. During runtime, a branch controller decides which part(s) of the network to execute based on the spatial context of the input frame. Our experiments using COCO dataset show that our adaptive object detection model achieves up to 45% reduction in the energy consumption, and up to 27% reduction in the latency, with a small loss in the average precision (AP) of object detection.

【23】 A Fast Algorithm for Computing the Deficiency Number of a Mahjong Hand 标题：一种快速计算麻将牌亏数的算法 链接：https://arxiv.org/abs/2108.06832

作者：Xueqing Yan,Yongming Li,Sanjiang Li 机构：School of Computer Science, Shaanxi Normal University, Xi’an, China, Centre for Quantum Software and Information, University of Technology, Sydney, Sydney, Australia 备注：32 pages, 3 figures 摘要：基于瓷砖的多人游戏麻将在亚洲被广泛使用，并且在世界范围内也越来越流行。面对面或在线，每位玩家从一手13张牌开始，玩家依次抽牌和弃牌，直到完成一手获胜牌。麻将中的一个重要概念是一手牌的缺牌数（日本麻将中的shanten数），它估计完成一手牌到一手牌所需的牌数变化。缺陷数在重大决策任务（如选择要丢弃的瓷砖）中起着至关重要的作用。本文提出了一种计算麻将手缺陷数的快速算法。与基线算法相比，新算法通常快100倍，更重要的是，它尊重代理对可用分片的知识。该算法可作为基于规则和基于机器学习的麻将AI在所有麻将变体中的基本程序。 摘要：The tile-based multiplayer game Mahjong is widely played in Asia and has also become increasingly popular worldwide. Face-to-face or online, each player begins with a hand of 13 tiles and players draw and discard tiles in turn until they complete a winning hand. An important notion in Mahjong is the deficiency number (a.k.a. shanten number in Japanese Mahjong) of a hand, which estimates how many tile changes are necessary to complete the hand into a winning hand. The deficiency number plays an essential role in major decision-making tasks such as selecting a tile to discard. This paper proposes a fast algorithm for computing the deficiency number of a Mahjong hand. Compared with the baseline algorithm, the new algorithm is usually 100 times faster and, more importantly, respects the agent's knowledge about available tiles. The algorithm can be used as a basic procedure in all Mahjong variants by both rule-based and machine learning-based Mahjong AI.

【24】 Learning from Images: Proactive Caching with Parallel Convolutional Neural Networks 标题：从图像中学习：基于并行卷积神经网络的主动缓存 链接：https://arxiv.org/abs/2108.06817

作者：Yantong Wang,Ye Hu,Zhaohui Yang,Walid Saad,Kai-Kit Wong,Vasilis Friderikos 备注：30 pages, 6 tables, 8 figures 摘要：随着数据爆炸的持续趋势，从数据服务器向最终用户传送数据包对移动网络的前向和回程流量造成了越来越大的压力。为了缓解这一问题，缓存更接近最终用户的流行内容已成为减少网络拥塞和改善用户体验的有效方法。为了找到内容缓存的最佳位置，许多传统方法构造各种混合整数线性规划（MILP）模型。然而，由于固有的维数灾难，这些方法可能无法支持在线决策。本文提出了一种新的主动缓存框架。该框架通过将优化问题转化为灰度图像，将基于模型的优化与数据驱动技术相结合。为了并行训练和简单设计，首先将所提出的MILP模型分解为若干子问题，然后训练卷积神经网络（CNN）预测这些子问题的内容缓存位置。此外，由于MILP模型分解忽略了子问题之间的内部影响，CNN的输出有成为不可行解的风险。因此，提供了两种算法：第一种算法使用CNN的预测作为额外约束，以减少决策变量的数量；第二种方法利用CNN的输出加速本地搜索。数值结果表明，与MILP方案相比，该方案可以减少71.6%的计算时间，仅增加0.8%的性能成本，从而提供高质量的实时决策。 摘要：With the continuous trend of data explosion, delivering packets from data servers to end users causes increased stress on both the fronthaul and backhaul traffic of mobile networks. To mitigate this problem, caching popular content closer to the end-users has emerged as an effective method for reducing network congestion and improving user experience. To find the optimal locations for content caching, many conventional approaches construct various mixed integer linear programming (MILP) models. However, such methods may fail to support online decision making due to the inherent curse of dimensionality. In this paper, a novel framework for proactive caching is proposed. This framework merges model-based optimization with data-driven techniques by transforming an optimization problem into a grayscale image. For parallel training and simple design purposes, the proposed MILP model is first decomposed into a number of sub-problems and, then, convolutional neural networks (CNNs) are trained to predict content caching locations of these sub-problems. Furthermore, since the MILP model decomposition neglects the internal effects among sub-problems, the CNNs' outputs have the risk to be infeasible solutions. Therefore, two algorithms are provided: the first uses predictions from CNNs as an extra constraint to reduce the number of decision variables; the second employs CNNs' outputs to accelerate local search. Numerical results show that the proposed scheme can reduce 71.6% computation time with only 0.8% additional performance cost compared to the MILP solution, which provides high quality decision making in real-time.

【25】 Weakly Supervised Temporal Anomaly Segmentation with Dynamic Time Warping 标题：基于动态时间规整的弱监督时间异常分割 链接：https://arxiv.org/abs/2108.06816

作者：Dongha Lee,Sehun Yu,Hyunjun Ju,Hwanjo Yu 机构：University of Illinois at Urbana-Champaign (UIUC), Urbana, IL, United States, Pohang University of Science and Technology (POSTECH), Pohang, South Korea 备注：ICCV 2021. 8 pages, References (2 pages), Appendix (3 pages), 6 figures 摘要：最近关于检测和定位时间异常的研究主要是利用深度神经网络以无监督的方式学习时间数据的正常模式。与它们不同的是，我们工作的目标是充分利用实例级（或弱）异常标签，它只指示在每个时态数据实例中是否发生了任何异常事件。在本文中，我们提出了一种新的框架WETAS，它可以有效地识别输入实例中的异常时间段（即连续时间点）。WETAS从实例级别的标签中学习鉴别特征，从而推断每个实例中正常和异常段的顺序，这可以用作粗分割掩码。基于输入实例与其分割掩码之间的动态时间扭曲（DTW）对齐，WETAS获得时间分割的结果，同时，通过使用掩码作为附加监控，它进一步增强了自身。我们的实验表明，在时间异常的定位方面，WETAS大大优于其他基线，并且与点级检测方法相比，它提供了更多信息。 摘要：Most recent studies on detecting and localizing temporal anomalies have mainly employed deep neural networks to learn the normal patterns of temporal data in an unsupervised manner. Unlike them, the goal of our work is to fully utilize instance-level (or weak) anomaly labels, which only indicate whether any anomalous events occurred or not in each instance of temporal data. In this paper, we present WETAS, a novel framework that effectively identifies anomalous temporal segments (i.e., consecutive time points) in an input instance. WETAS learns discriminative features from the instance-level labels so that it infers the sequential order of normal and anomalous segments within each instance, which can be used as a rough segmentation mask. Based on the dynamic time warping (DTW) alignment between the input instance and its segmentation mask, WETAS obtains the result of temporal segmentation, and simultaneously, it further enhances itself by using the mask as additional supervision. Our experiments show that WETAS considerably outperforms other baselines in terms of the localization of temporal anomalies, and also it provides more informative results than point-level detection methods.

【26】 Deep Adversarially-Enhanced k-Nearest Neighbors 标题：深层对抗性增强的k-最近邻域 链接：https://arxiv.org/abs/2108.06797

作者：Ren Wang,Tianqi Chen 机构：University of Michigan 摘要：最近的研究从理论和经验上表明，深度神经网络（DNN）对小扰动具有固有的脆弱性。应用深度k-最近邻（DkNN）分类器，我们观察到随着层的加深，鲁棒性和准确性的权衡显著增加。在这项工作中，我们提出了一种深度敌对增强的k-最近邻（DAEkNN）方法，该方法比DkNN具有更高的鲁棒性，并通过两个关键因素缓解了深层鲁棒性-准确性权衡。首先，DAEkNN基于一个经过对抗训练的模型。其次，DAEkNN利用良性和对抗性训练数据的加权组合进行预测。从经验上看，我们发现DAEkNN改善了MNIST和CIFAR-10数据集的鲁棒性和鲁棒性-准确性权衡。 摘要：Recent works have theoretically and empirically shown that deep neural networks (DNNs) have an inherent vulnerability to small perturbations. Applying the Deep k-Nearest Neighbors (DkNN) classifier, we observe a dramatically increasing robustness-accuracy trade-off as the layer goes deeper. In this work, we propose a Deep Adversarially-Enhanced k-Nearest Neighbors (DAEkNN) method which achieves higher robustness than DkNN and mitigates the robustness-accuracy trade-off in deep layers through two key elements. First, DAEkNN is based on an adversarially trained model. Second, DAEkNN makes predictions by leveraging a weighted combination of benign and adversarial training data. Empirically, we find that DAEkNN improves both the robustness and the robustness-accuracy trade-off on MNIST and CIFAR-10 datasets.

【27】 Augmenting GRIPS with Heuristic Sampling for Planning Feasible Trajectories of a Car-Like Robot 标题：用启发式采样增强夹点规划汽车机器人可行轨迹 链接：https://arxiv.org/abs/2108.06789

作者：Brian Angulo,Konstantin Yakovlev,Ivan Radionov 备注：6 pages, 6 figures 摘要：非全息移动机器人的动力学运动规划是一个具有挑战性的问题，目前还没有一个通用的解决方案。解决这一问题的一个有效的计算方法是首先构建一条几何路径，然后将该路径转换为运动学上可行的路径。梯度信息路径平滑（GRAPS）是最近引入的一种用于此类变换的方法。夹点迭代地使路径变形，并添加/删除航路点，同时尝试通过提供的符合运动学约束的转向功能连接每对连续的航路点。该算法相对较快，但不幸的是，不能保证它会成功。在实践中，它往往无法为具有大转弯半径的类车机器人生成可行的轨迹。在这项工作中，我们介绍了一系列的修改，旨在提高汽车机器人夹持的成功率。主要的改进是增加了额外的步骤，即沿几何路径的瓶颈部分（如急转弯）启发式采样航路点。实验评估的结果清楚地表明，与原始夹点相比，建议算法的成功率高达40%，命中率高达90%，而运行时间较低。 摘要：Kinodynamic motion planning for non-holomonic mobile robots is a challenging problem that is lacking a universal solution. One of the computationally efficient ways to solve it is to build a geometric path first and then transform this path into a kinematically feasible one. Gradient-informed Path Smoothing (GRIPS) is a recently introduced method for such transformation. GRIPS iteratively deforms the path and adds/deletes the waypoints while trying to connect each consecutive pair of them via the provided steering function that respects the kinematic constraints. The algorithm is relatively fast but, unfortunately, does not provide any guarantees that it will succeed. In practice, it often fails to produce feasible trajectories for car-like robots with large turning radius. In this work, we introduce a range of modifications that are aimed at increasing the success rate of GRIPS for car-like robots. The main enhancement is adding the additional step that heuristically samples waypoints along the bottleneck parts of the geometric paths (such as sharp turns). The results of the experimental evaluation provide a clear evidence that the success rate of the suggested algorithm is up to 40% higher compared to the original GRIPS and hits the bar of 90%, while its runtime is lower.

【28】 Two Eyes Are Better Than One: Exploiting Binocular Correlation for Diabetic Retinopathy Severity Grading 标题：双眼胜于一眼：利用双目相关性进行糖尿病视网膜病变严重程度分级 链接：https://arxiv.org/abs/2108.06763

作者：Peisheng Qian,Ziyuan Zhao,Cong Chen,Zeng Zeng,Xiaoli Li 机构： This research is supportedby Institute for Infocomm Research (I 2R), 1 Institute for Infocomm Research(I 2R), 2 National University of Singapore 备注：Accepted in 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE EMBC 2021 摘要：糖尿病视网膜病变（DR）是糖尿病患者最常见的眼部疾病之一。然而，视力丧失主要发生在DR的晚期，视力损害的症状，从轻微到严重，可能差异很大，增加了临床实践中的诊断和治疗负担。基于视网膜图像的深度学习方法在DR自动分级方面取得了显著的成功，但大多数方法忽视了糖尿病的存在通常会影响双眼，眼科医生通常同时对双眼进行DR诊断比较，从而导致左右眼之间的相关性未被开发。在本研究中，模拟诊断过程，我们提出了一种双流双目网络来捕获左右眼之间的细微相关性，其中，眼睛的成对图像在训练期间分别馈入两个相同的子网络。我们设计了一种对比分级损失学习双目相关的五类DR检测方法，该方法在最大化类间差异的同时最小化类内差异。在EyePACS数据集上的实验结果表明了所提出的双目模型的优越性，大大优于单目方法。 摘要：Diabetic retinopathy (DR) is one of the most common eye conditions among diabetic patients. However, vision loss occurs primarily in the late stages of DR, and the symptoms of visual impairment, ranging from mild to severe, can vary greatly, adding to the burden of diagnosis and treatment in clinical practice. Deep learning methods based on retinal images have achieved remarkable success in automatic DR grading, but most of them neglect that the presence of diabetes usually affects both eyes, and ophthalmologists usually compare both eyes concurrently for DR diagnosis, leaving correlations between left and right eyes unexploited. In this study, simulating the diagnostic process, we propose a two-stream binocular network to capture the subtle correlations between left and right eyes, in which, paired images of eyes are fed into two identical subnetworks separately during training. We design a contrastive grading loss to learn binocular correlation for five-class DR detection, which maximizes inter-class dissimilarity while minimizing the intra-class difference. Experimental results on the EyePACS dataset show the superiority of the proposed binocular model, outperforming monocular methods by a large margin.

【29】 Multi-Slice Dense-Sparse Learning for Efficient Liver and Tumor Segmentation 标题：多层密集-稀疏学习在肝脏和肿瘤分割中的应用 链接：https://arxiv.org/abs/2108.06761

作者：Ziyuan Zhao,Zeyu Ma,Yanjie Liu,Zeng Zeng,Pierce KH Chow 机构： 2 NationalUniversity of Singapore 备注：Accepted in 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, IEEE EMBC 2021 摘要：准确的肝脏和肿瘤自动分割在治疗计划和疾病监测中起着至关重要的作用。近年来，深度卷积神经网络（DCNNs）在二维和三维医学图像分割中取得了巨大的成功。然而，2D DCNN不能充分利用片间信息，而3D DCNN计算成本高且内存密集。为了解决这些问题，我们首先从数据的角度提出了一种新的密集稀疏训练流，其中，密集相邻切片和稀疏相邻切片被提取作为正则化DCNN的输入，从而提高模型性能。此外，我们还从网络的角度设计了一个2.5D轻型nnU网络，其中采用了深度可分离卷积来提高效率。在LiTS数据集上的大量实验证明了该方法的优越性。 摘要：Accurate automatic liver and tumor segmentation plays a vital role in treatment planning and disease monitoring. Recently, deep convolutional neural network (DCNNs) has obtained tremendous success in 2D and 3D medical image segmentation. However, 2D DCNNs cannot fully leverage the inter-slice information, while 3D DCNNs are computationally expensive and memory intensive. To address these issues, we first propose a novel dense-sparse training flow from a data perspective, in which, densely adjacent slices and sparsely adjacent slices are extracted as inputs for regularizing DCNNs, thereby improving the model performance. Moreover, we design a 2.5D light-weight nnU-Net from a network perspective, in which, depthwise separable convolutions are adopted to improve the efficiency. Extensive experiments on the LiTS dataset have demonstrated the superiority of the proposed method.

【30】 Exploring Generalization Ability of Pretrained Language Models on Arithmetic and Logical Reasoning 标题：探索预先训练的语言模型对算术和逻辑推理的泛化能力 链接：https://arxiv.org/abs/2108.06743

作者：Cunxiang Wang,Boyuan Zheng,Yuchen Niu,Yue Zhang 机构：♠Zhejiang University, China, ♣School of Engineering, Westlake University, China, ♥Institute of Advanced Technology, Westlake Institute for Advanced Study, China, ♦Johns Hopkins University, ♭Imperial College London 备注：Accepted by NLPCC2021 摘要：为了定量和直观地探索预训练语言模型（PLM）的泛化能力，我们设计了几个算术和逻辑推理任务。我们分析了当测试数据与列车数据处于相同分布时以及当其不同时PLMs的通用性，对于后一种分析，我们还设计了一个交叉分布测试集，而不是分布内测试集。我们在最先进和公开发布的可生成PLM-BART上进行实验。我们的研究发现，当分布相同时，PLM可以很容易地进行推广，但是，从分布中推广出来仍然很困难。 摘要：To quantitatively and intuitively explore the generalization ability of pre-trained language models (PLMs), we have designed several tasks of arithmetic and logical reasoning. We both analyse how well PLMs generalize when the test data is in the same distribution as the train data and when it is different, for the latter analysis, we have also designed a cross-distribution test set other than the in-distribution test set. We conduct experiments on one of the most advanced and publicly released generative PLM - BART. Our research finds that the PLMs can easily generalize when the distribution is the same, however, it is still difficult for them to generalize out of the distribution.

【31】 Development of the InBan_CIDO Ontology by Reusing the Concepts along with Detecting Overlapping Information 标题：基于概念重用和重叠信息检测的Inban_cido本体开发 链接：https://arxiv.org/abs/2108.06742

作者：Archana Patel,Narayan C Debnath 机构： and Narayan C. Debnath, Department of Software Engineering, School of Computing and Information, Technology, Eastern International University, Vietnam 备注：3rd International Conference on Inventive Computation and Information Technologies (ICICIT 2021) 摘要：新冠疫情是一种全球紧急情况，严重影响了各国的经济。当印度的经济增长率处于过去10年的最低水平时，冠状病毒19型袭击了印度。为了从语义上分析这场流行病对经济的影响，有一个本体论是很有必要的。CIDO本体是一个标准化良好的本体，专门设计用于评估冠状病毒疾病的影响，并将其结果用于政府、行业专家和各个领域（如研究、医学进步、技术创新采用等）的专业人员的未来决策预测。然而，该本体论并未分析新冠疫情对印度银行业的影响。另一方面，已经开发了Covid19IBO本体，以分析Covid19大流行对印度银行业的影响，但该本体并未反映Covid19数据的完整信息。结果，用户无法获得关于新冠疫情19及其对印度经济影响的所有相关信息。本文旨在通过重用来自其他数据源的概念，扩展CIDO本体，展示Covid19对印度经济部门的影响。我们还提供了一种简化的模式匹配方法来检测本体之间的重叠信息。实验分析表明，该方法具有合理的效果。 摘要：The covid19 pandemic is a global emergency that badly impacted the economies of various countries. Covid19 hit India when the growth rate of the country was at the lowest in the last 10 years. To semantically analyze the impact of this pandemic on the economy, it is curial to have an ontology. CIDO ontology is a well standardized ontology that is specially designed to assess the impact of coronavirus disease and utilize its results for future decision forecasting for the government, industry experts, and professionals in the field of various domains like research, medical advancement, technical innovative adoptions, and so on. However, this ontology does not analyze the impact of the Covid19 pandemic on the Indian banking sector. On the other side, Covid19IBO ontology has been developed to analyze the impact of the Covid19 pandemic on the Indian banking sector but this ontology does not reflect complete information of Covid19 data. Resultantly, users cannot get all the relevant information about Covid19 and its impact on the Indian economy. This article aims to extend the CIDO ontology to show the impact of Covid19 on the Indian economy sector by reusing the concepts from other data sources. We also provide a simplified schema matching approach that detects the overlapping information among the ontologies. The experimental analysis proves that the proposed approach has reasonable results.

【32】 A Two-Layer Near-Optimal Strategy for Substation Constraint Management via Home Batteries 标题：一种基于家用蓄电池的两层近似最优变电站约束管理策略 链接：https://arxiv.org/abs/2108.06735

作者：Igor Melatti,Federico Mari,Toni Mancini,Milan Prodanovic,Enrico Tronci 机构： Author Prodanovic is with Electrical Systems Unit of IMDEA EnergyInstitute 备注：None 摘要：在配电网中，变电站约束管理要求住宅用户的总电力需求保持在适当的范围内。变电站约束管理的效率可以通过减少违反w.r.t.非管理需求的约束来衡量。家用电池有望实现高效、用户无意识的变电站约束管理。家庭电池的集中控制将实现最佳效率。然而，用户很难接受，因为服务提供商（如公用事业或聚合商）将直接控制用户场所的电池。不幸的是，设计有效的分层控制策略，从而克服上述问题，远非易事。我们提出了一种新的家庭电池两层控制策略，避免了服务提供商对家庭设备的直接控制，同时产生了接近最优的变电站约束管理效率。我们对丹麦62户家庭的现场数据进行的模拟结果表明，我们的方法实现的变电站约束管理效率至少是理论最优集中策略的82%。 摘要：Within electrical distribution networks, substation constraints management requires that aggregated power demand from residential users is kept within suitable bounds. Efficiency of substation constraints management can be measured as the reduction of constraints violations w.r.t. unmanaged demand. Home batteries hold the promise of enabling efficient and user-oblivious substation constraints management. Centralized control of home batteries would achieve optimal efficiency. However, it is hardly acceptable by users, since service providers (e.g., utilities or aggregators) would directly control batteries at user premises. Unfortunately, devising efficient hierarchical control strategies, thus overcoming the above problem, is far from easy. We present a novel two-layer control strategy for home batteries that avoids direct control of home devices by the service provider and at the same time yields near-optimal substation constraints management efficiency. Our simulation results on field data from 62 households in Denmark show that the substation constraints management efficiency achieved with our approach is at least 82% of the one obtained with a theoretical optimal centralized strategy.

【33】 Towards Visual Explainable Active Learning for Zero-Shot Classification 标题：面向视觉可解释的主动学习在Zero-Shot分类中的应用 链接：https://arxiv.org/abs/2108.06730

作者：Shichao Jia,Zeyu Li,Nuo Chen,Jiawan Zhang 机构：c, d, b, a, f, e 摘要：零炮分类是解决训练类和测试类不相交问题的一种很有前途的范例。实现这一点通常需要专家通过手动指定类属性矩阵来定义哪些类具有哪些属性，从而将其领域知识具体化。设计一个合适的类属性矩阵是后续过程的关键，但是这个设计过程是单调乏味的，没有指导的反复试验。为了解决上述问题，本文提出了一种可视化的、可解释的主动学习方法&语义导航器。这种方法通过在每个交互循环中的四个动作（询问、解释、推荐、响应）来促进人工智能团队合作。机器提出对比问题，以指导人类思考属性的过程。一种称为语义图的新颖可视化方法解释了机器的当前状态。因此，分析人员可以更好地理解为什么机器会对对象进行错误分类。此外，机器为每个属性推荐类的标签，以减轻标签负担。最后，人类可以通过交互修改标签来控制模型，机器会调整其建议。与无指导的方法相比，视觉解释主动学习方法提高了人类交互构建Zero-Shot分类模型的效率。我们使用零炮分类的标准基准，通过用户研究来证明我们的结果。 摘要：Zero-shot classification is a promising paradigm to solve an applicable problem when the training classes and test classes are disjoint. Achieving this usually needs experts to externalize their domain knowledge by manually specifying a class-attribute matrix to define which classes have which attributes. Designing a suitable class-attribute matrix is the key to the subsequent procedure, but this design process is tedious and trial-and-error with no guidance. This paper proposes a visual explainable active learning approach with its design and implementation called semantic navigator to solve the above problems. This approach promotes human-AI teaming with four actions (ask, explain, recommend, respond) in each interaction loop. The machine asks contrastive questions to guide humans in the thinking process of attributes. A novel visualization called semantic map explains the current status of the machine. Therefore analysts can better understand why the machine misclassifies objects. Moreover, the machine recommends the labels of classes for each attribute to ease the labeling burden. Finally, humans can steer the model by modifying the labels interactively, and the machine adjusts its recommendations. The visual explainable active learning approach improves humans' efficiency of building zero-shot classification models interactively, compared with the method without guidance. We justify our results with user studies using the standard benchmarks for zero-shot classification.

【34】 Towards Understanding Theoretical Advantages of Complex-Reaction Networks 标题：认识络合反应网络的理论优势 链接：https://arxiv.org/abs/2108.06711

作者：Shao-Qun Zhang,Gao Wei,Zhi-Hua Zhou 机构： National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing , China 摘要：近年来，复值神经网络受到了越来越多的关注，但与实值网络相比，复值神经网络的优势仍然是一个未知数。这项工作通过引入具有全连接前馈结构的\emph{复杂反应网络}朝着这个方向迈出了一步。我们证明了复杂反应网络的普适逼近性，并证明了一类径向函数可以用多项式参数数由复杂反应网络逼近，而实值网络至少需要指数参数才能达到相同的逼近水平。对于经验风险最小化，我们的理论结果表明，复杂反应网络的临界点集是实值网络临界点集的一个适当子集，这可能为更容易找到复杂反应网络的最优解提供一些见解。 摘要：Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks. This work takes one step on this direction by introducing the \emph{complex-reaction network} with fully-connected feed-forward architecture. We prove the universal approximation property for complex-reaction networks, and show that a class of radial functions can be approximated by a complex-reaction network using the polynomial number of parameters, whereas real-valued networks need at least exponential parameters to reach the same approximation level. For empirical risk minimization, our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks, which may show some insights on finding the optimal solutions more easily for complex-reaction networks.

【35】 Deepfake Representation with Multilinear Regression 标题：多元线性回归的深伪表示 链接：https://arxiv.org/abs/2108.06702

作者：Sara Abdali,M. Alex O. Vasilescu,Evangelos E. Papalexakis 机构：University of California, Riverside, University of California, Los Angeles, Tensor Vision, Los Angeles 摘要：生成型神经网络结构（如GANs）可用于生成合成实例，以弥补实际数据的不足。然而，他们可能被用来创建可能导致社会、政治或经济动荡的媒体。一种新兴媒体是“深度伪造”。能够区分这类媒体的技术是必不可少的。在本文中，我们提出了一种改进的多线性（张量）方法，一种线性和多线性回归的组合，用于表示假数据和真实数据。我们通过使用改进的多线性（张量）方法表示深度伪造来测试我们的方法，并执行SVM分类，结果令人鼓舞。 摘要：Generative neural network architectures such as GANs, may be used to generate synthetic instances to compensate for the lack of real data. However, they may be employed to create media that may cause social, political or economical upheaval. One emerging media is "Deepfake".Techniques that can discriminate between such media is indispensable. In this paper, we propose a modified multilinear (tensor) method, a combination of linear and multilinear regressions for representing fake and real data. We test our approach by representing Deepfakes with our modified multilinear (tensor) approach and perform SVM classification with encouraging results.

【36】 ST3D++: Denoised Self-training for Unsupervised Domain Adaptation on 3D Object Detection 标题：ST3D++：三维目标检测中无监督域自适应的去噪自训练 链接：https://arxiv.org/abs/2108.06682

作者：Jihan Yang,Shaoshuai Shi,Zhe Wang,Hongsheng Li,Xiaojuan Qi 机构： This•Jihan Yang and Xiaojuan Qi are with the Department of Electrical andElectronic Engineering at The University of Hong Kong, Shaoshuai Shi and Hongsheng Li are with the Department of ElectronicEngineering at The Chinese University of Hong Kong 摘要：在本文中，我们提出了一种称为ST3D++的自训练方法，该方法具有整体伪标签去噪管道，用于三维目标检测的无监督域自适应。ST3D++旨在减少伪标签生成过程中的噪声，同时缓解噪声伪标签对模型训练的负面影响。首先，ST3D++使用随机对象缩放（random object SCALATION，ROS）在标记的源域上预训练3D对象检测器，该方法旨在减少源域对象缩放偏差引起的目标域伪标签噪声。然后，通过交替生成伪标签和使用伪标签目标域数据训练目标检测器，逐步改进检测器。在这里，我们为伪标签生成过程配备了混合质量感知三重态存储器，以提高生成的伪标签的质量和稳定性。同时，在模型训练阶段，我们提出了源数据辅助训练策略和课程数据扩充策略，以有效地校正噪声梯度方向，避免模型过度拟合噪声伪标记数据。这些特定的设计使得检测器能够在经过精心精炼的伪标记目标数据上使用去噪的训练信号进行训练，从而有效地促进对象检测器适应目标域，而无需注释。最后，我们的方法在四个3D基准数据集（即Waymo、KITTI、Lyft和nuScenes）上对三个常见类别（即汽车、行人和自行车）进行了评估。ST3D++在所有评估设置上都达到了最先进的性能，大大超过了相应的基线（例如，Waymo$\rightarrow$KITTI上的9.6%$\sim$38.16%（以AP${\text{3D}}$$）计算），甚至在KITTI 3D对象检测基准上超过了完全监督的oracle结果（目标优先）。代码将可用。 摘要：In this paper, we present a self-training method, named ST3D++, with a holistic pseudo label denoising pipeline for unsupervised domain adaptation on 3D object detection. ST3D++ aims at reducing noise in pseudo label generation as well as alleviating the negative impacts of noisy pseudo labels on model training. First, ST3D++ pre-trains the 3D object detector on the labeled source domain with random object scaling (ROS) which is designed to reduce target domain pseudo label noise arising from object scale bias of the source domain. Then, the detector is progressively improved through alternating between generating pseudo labels and training the object detector with pseudo-labeled target domain data. Here, we equip the pseudo label generation process with a hybrid quality-aware triplet memory to improve the quality and stability of generated pseudo labels. Meanwhile, in the model training stage, we propose a source data assisted training strategy and a curriculum data augmentation policy to effectively rectify noisy gradient directions and avoid model over-fitting to noisy pseudo labeled data. These specific designs enable the detector to be trained on meticulously refined pseudo labeled target data with denoised training signals, and thus effectively facilitate adapting an object detector to a target domain without requiring annotations. Finally, our method is assessed on four 3D benchmark datasets (i.e., Waymo, KITTI, Lyft, and nuScenes) for three common categories (i.e., car, pedestrian and bicycle). ST3D++ achieves state-of-the-art performance on all evaluated settings, outperforming the corresponding baseline by a large margin (e.g., 9.6% $\sim$ 38.16% on Waymo $\rightarrow$ KITTI in terms of AP$_{\text{3D}}$), and even surpasses the fully supervised oracle results on the KITTI 3D object detection benchmark with target prior. Code will be available.

【37】 Deep Geospatial Interpolation Networks 标题：深层次地理空间内插网络 链接：https://arxiv.org/abs/2108.06670

作者：Sumit Kumar Varshney,Jeetu Kumar,Aditya Tiwari,Rishabh Singh,Venkata M. V. Gunturi,Narayanan C. Krishnan 机构：Indian Institute of Technology Ropar, Punjab, India 摘要：时空数据插值在气候、交通和采矿等领域有着广泛的应用。由于复杂的时空关系，时空插值具有很大的挑战性。然而，传统的技术，如克里金法，在空间和时间维度上表现出高度差异的数据上运行时间长，性能差。为此，我们提出了一种新的深度神经网络，称为深度地理空间插值网络（DGIN），它结合了空间和时间关系，并且具有显著较低的训练时间。DGIN由三个主要组件组成：用于捕捉空间相关性的空间编码器、用于合并时间动态的顺序模块以及用于了解间隙周围时间邻域重要性的注意块。我们在来自两个不同区域的MODIS反射率数据集上评估DGIN。我们的实验结果表明，DGIN有两个优点：（a）它优于其他方法（具有较低的MSE，p值<0.01）和（b）它的执行时间比Kriging低得多。 摘要：Interpolation in Spatio-temporal data has applications in various domains such as climate, transportation, and mining. Spatio-Temporal interpolation is highly challenging due to the complex spatial and temporal relationships. However, traditional techniques such as Kriging suffer from high running time and poor performance on data that exhibit high variance across space and time dimensions. To this end, we propose a novel deep neural network called as Deep Geospatial Interpolation Network(DGIN), which incorporates both spatial and temporal relationships and has significantly lower training time. DGIN consists of three major components: Spatial Encoder to capture the spatial dependencies, Sequential module to incorporate the temporal dynamics, and an Attention block to learn the importance of the temporal neighborhood around the gap. We evaluate DGIN on the MODIS reflectance dataset from two different regions. Our experimental results indicate that DGIN has two advantages: (a) it outperforms alternative approaches (has lower MSE with p-value < 0.01) and, (b) it has significantly low execution time than Kriging.

【38】 HCR-Net: A deep learning based script independent handwritten character recognition network 标题：HCR-Net：一种基于深度学习的手写体独立字符识别网络 链接：https://arxiv.org/abs/2108.06663

作者：Vinod Kumar Chauhan,Sukhdeep Singh,Anuj Sharma 机构：Received: date Accepted: date 备注：21 pages, 5 figures, 16 tables (under review) 摘要：手写字符识别（HCR）是模式识别中一个具有挑战性的学习问题，主要原因是字符结构相似、书写风格不同、数据集噪声大以及语言和脚本种类繁多。HCR问题已经被广泛研究了几十年，但是对于脚本无关模型的研究非常有限。这是因为脚本的多样性、大多数传统研究工作的重点都是针对特定语言/脚本的手工特征提取技术，并且并不总是可用，以及公共数据集和代码无法再现结果等因素。另一方面，深度学习在模式识别的不同领域（包括HCR）取得了巨大成功，并提供了端到端的学习，即自动特征提取和识别。在本文中，我们提出了一种新的深度学习体系结构，称为HCR-Net，该体系结构利用迁移学习和图像增强进行端到端学习，用于与脚本无关的手写字符识别。该网络基于一种新的HCR转移学习方法，其中使用了预先训练的VGG16网络的一些较低层。由于迁移学习和图像增强，HCR网络提供了更快的训练、更好的性能和更好的概括。在孟加拉语、旁遮普语、印地语、英语、瑞典语、乌尔都语、波斯语、藏语、卡纳达语、马拉雅拉姆语、泰卢固语、马拉地语、尼泊尔语和阿拉伯语的公开数据集上的实验结果证明了HCR网络的有效性，并建立了若干新的基准。对于结果的再现性和HCR研究的进展，完整代码在\href公开发布{https://github.com/jmdvinodjmd/HCR-Net}{GitHub}。 摘要：Handwritten character recognition (HCR) is a challenging learning problem in pattern recognition, mainly due to similarity in structure of characters, different handwriting styles, noisy datasets and a large variety of languages and scripts. HCR problem is studied extensively for a few decades but there is very limited research on script independent models. This is because of factors, like, diversity of scripts, focus of the most of conventional research efforts on handcrafted feature extraction techniques which are language/script specific and are not always available, and unavailability of public datasets and codes to reproduce the results. On the other hand, deep learning has witnessed huge success in different areas of pattern recognition, including HCR, and provides end-to-end learning, i.e., automated feature extraction and recognition. In this paper, we have proposed a novel deep learning architecture which exploits transfer learning and image-augmentation for end-to-end learning for script independent handwritten character recognition, called HCR-Net. The network is based on a novel transfer learning approach for HCR, where some of lower layers of a pre-trained VGG16 network are utilised. Due to transfer learning and image-augmentation, HCR-Net provides faster training, better performance and better generalisations. The experimental results on publicly available datasets of Bangla, Punjabi, Hindi, English, Swedish, Urdu, Farsi, Tibetan, Kannada, Malayalam, Telugu, Marathi, Nepali and Arabic languages prove the efficacy of HCR-Net and establishes several new benchmarks. For reproducibility of the results and for the advancements of the HCR research, complete code is publicly released at \href{https://github.com/jmdvinodjmd/HCR-Net}{GitHub}.

【39】 SAPPHIRE: Approaches for Enhanced Concept-to-Text Generation 标题：蓝宝石：增强概念到文本生成的方法 链接：https://arxiv.org/abs/2108.06643

作者：Steven Y. Feng,Jessica Huynh,Chaitanya Narisetty,Eduard Hovy,Varun Gangal 机构：Language Technologies Institute, Carnegie Mellon University 备注：INLG 2021. Code available at this https URL 摘要：我们激发并提出了一套简单但有效的改进概念到文本生成的SAPPHIRE：集合增强和后期短语填充和重组。通过使用BART和T5模型的实验，我们证明了它们在生成性常识推理，即CommonGen任务上的有效性。通过广泛的自动和人工评估，我们发现SAPPHIRE显著提高了模型性能。深入的定性分析表明，SAPPHIRE有效地解决了基线模型生成的许多问题，包括缺乏常识、不够具体和流利性差。 摘要：We motivate and propose a suite of simple but effective improvements for concept-to-text generation called SAPPHIRE: Set Augmentation and Post-hoc PHrase Infilling and REcombination. We demonstrate their effectiveness on generative commonsense reasoning, a.k.a. the CommonGen task, through experiments using both BART and T5 models. Through extensive automatic and human evaluation, we show that SAPPHIRE noticeably improves model performance. An in-depth qualitative analysis illustrates that SAPPHIRE effectively addresses many issues of the baseline model generations, including lack of commonsense, insufficient specificity, and poor fluency.

【40】 DEXTER: Deep Encoding of External Knowledge for Named Entity Recognition in Virtual Assistants 标题：Dexter：用于虚拟助手命名实体识别的外部知识深度编码 链接：https://arxiv.org/abs/2108.06633

作者：Deepak Muralidharan,Joel Ruben Antony Moniz,Weicheng Zhang,Stephen Pulman,Lin Li,Megan Barnes,Jingjing Pan,Jason Williams,Alex Acero 机构：Apple, USA, Apple, UK, University of Washington, USA 备注：Interspeech 2021 摘要：命名实体识别（NER）通常是在书面来源良好的文本上开发和测试的。然而，在智能语音助理中，NER是一个重要组件，由于用户或语音识别错误，NER的输入可能会有噪声。在应用程序中，实体标签可能会频繁更改，并且可能需要非文本属性（如主题性或流行性）来在备选方案中进行选择。我们描述了一个旨在解决这些问题的NER系统。我们在一个专有的用户派生数据集上测试和训练这个系统。我们将其与基线纯文本NER系统进行比较；使用外部地名录增强基线；我们下面介绍的搜索和间接标记技术增强了基线。最终配置使NER错误率降低约6%。我们还表明，该技术改进了相关任务，如语义分析，错误率提高了5%。 摘要：Named entity recognition (NER) is usually developed and tested on text from well-written sources. However, in intelligent voice assistants, where NER is an important component, input to NER may be noisy because of user or speech recognition error. In applications, entity labels may change frequently, and non-textual properties like topicality or popularity may be needed to choose among alternatives. We describe a NER system intended to address these problems. We test and train this system on a proprietary user-derived dataset. We compare with a baseline text-only NER system; the baseline enhanced with external gazetteers; and the baseline enhanced with the search and indirect labelling techniques we describe below. The final configuration gives around 6% reduction in NER error rate. We also show that this technique improves related tasks, such as semantic parsing, with an improvement of up to 5% in error rate.

【41】 Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer 标题：基于时态图协同变换的连续时间序贯推荐 链接：https://arxiv.org/abs/2108.06625

作者：Ziwei Fan,Zhiwei Liu,Jiawei Zhang,Yun Xiong,Lei Zheng,Philip S. Yu 机构：Department of Computer Science, University of Illinois at Chicago, USA, IFM Lab, Department of Computer, Science, University of California, Davis, Fudan University, China, Pinterest Inc. 备注：accepted by CIKM2021 摘要：为了对用户偏好的演化进行建模，我们应该学习基于时间顺序的物品购买序列的用户/物品嵌入，这被定义为顺序推荐（SR）问题。现有方法利用顺序模式对项目转换进行建模。然而，它们中的大多数忽略了关键的时间协作信号，这些信号潜藏在不断演化的用户项交互中，并与序列模式共存。因此，我们建议统一序列模式和时态协作信号来提高推荐的质量，这是一个相当具有挑战性的问题。首先，很难同时编码序列模式和协作信号。其次，表达协作信号的时间效应是非常重要的。因此，我们在定义的连续时间二部图的基础上设计了一个新的时序图推荐器（TGSRec）。我们在TGSRec中提出了一种新的时间协同转换层（TCT），该层通过采用一种新的协同注意来改进自我注意机制。TCT层可以同时捕获来自用户和项目的协作信号，并考虑序列模式中的时间动态。我们通过时态图传播从TCTLayer学习到的信息，以统一序列模式和时态协作信号。对五个数据集的实证结果表明，TGSRec显著优于其他基线，平均提高22.5%和22.1%Recall@10and分别为MRR。 摘要：In order to model the evolution of user preference, we should learn user/item embeddings based on time-ordered item purchasing sequences, which is defined as Sequential Recommendation (SR) problem. Existing methods leverage sequential patterns to model item transitions. However, most of them ignore crucial temporal collaborative signals, which are latent in evolving user-item interactions and coexist with sequential patterns. Therefore, we propose to unify sequential patterns and temporal collaborative signals to improve the quality of recommendation, which is rather challenging. Firstly, it is hard to simultaneously encode sequential patterns and collaborative signals. Secondly, it is non-trivial to express the temporal effects of collaborative signals. Hence, we design a new framework Temporal Graph Sequential Recommender (TGSRec) upon our defined continuous-time bi-partite graph. We propose a novel Temporal Collaborative Trans-former (TCT) layer in TGSRec, which advances the self-attention mechanism by adopting a novel collaborative attention. TCT layer can simultaneously capture collaborative signals from both users and items, as well as considering temporal dynamics inside sequential patterns. We propagate the information learned fromTCTlayerover the temporal graph to unify sequential patterns and temporal collaborative signals. Empirical results on five datasets show that TGSRec significantly outperforms other baselines, in average up to 22.5% and 22.1%absolute improvements in Recall@10and MRR, respectively.

【42】 Adaptive Selection of Informative Path Planning Strategies via Reinforcement Learning 标题：基于强化学习的信息路径规划策略自适应选择 链接：https://arxiv.org/abs/2108.06618

作者：Taeyeong Choi,Grzegorz Cielniak 备注：Published in the proceedings of ECMR 2021 摘要：在我们之前的工作中，我们设计了一个系统策略，通过使用高斯过程回归（GPR）的预测不确定性作为路径规划中部署机器人的“吸引力”，对采样位置进行优先排序，从而显著提高空间插值的精度。尽管与旅行商问题（TSP）解算器的集成也显示出相对较短的旅行距离，但我们在此假设了几个可能降低整体预测精度的因素，因为次优位置最终可能包含在其路径中。为了解决这个问题，在本文中，我们首先探讨了采用不同空间范围的“局部规划”方法，在这些空间范围内，下一个采样位置被优先排序，以调查它们对预测性能以及产生的旅行距离的影响。此外，训练基于强化学习（RL）的高级控制器，从一组特定的本地计划员自适应生成混合计划，以根据最新的预测状态从选择中继承独特的优势。我们在温度监测机器人用例上的实验表明，动态混合的计划者不仅可以生成复杂的，信息型计划，单个计划员无法单独创建，但也可以确保显著缩短行程距离，而无需任何最短路径计算模块的辅助，且不以预测可靠性为代价。 摘要：In our previous work, we designed a systematic policy to prioritize sampling locations to lead significant accuracy improvement in spatial interpolation by using the prediction uncertainty of Gaussian Process Regression (GPR) as "attraction force" to deployed robots in path planning. Although the integration with Traveling Salesman Problem (TSP) solvers was also shown to produce relatively short travel distance, we here hypothesise several factors that could decrease the overall prediction precision as well because sub-optimal locations may eventually be included in their paths. To address this issue, in this paper, we first explore "local planning" approaches adopting various spatial ranges within which next sampling locations are prioritized to investigate their effects on the prediction performance as well as incurred travel distance. Also, Reinforcement Learning (RL)-based high-level controllers are trained to adaptively produce blended plans from a particular set of local planners to inherit unique strengths from that selection depending on latest prediction states. Our experiments on use cases of temperature monitoring robots demonstrate that the dynamic mixtures of planners can not only generate sophisticated, informative plans that a single planner could not create alone but also ensure significantly reduced travel distances at no cost of prediction reliability without any assist of additional modules for shortest path calculation.

【43】 Offline-Online Reinforcement Learning for Energy Pricing in Office Demand Response: Lowering Energy and Data Costs 标题：办公室需求响应中能源定价的离线-在线强化学习：降低能源和数据成本 链接：https://arxiv.org/abs/2108.06594

作者：Doseok Jang,Lucas Spangher,Manan Khattar,Utkarsha Agwan,Selvaprabuh Nadarajah,Costas Spanos 机构：. Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, Berkeley, California, USA ,. Department of Information and Decision Sciences, University of Illinois, Chicago 摘要：我们的团队提议在一座办公楼中进行一次全面的能源需求响应实验。尽管这是一项激动人心的工作，将为社区提供价值，但为强化学习代理收集训练数据的成本高昂且有限。在这项工作中，我们将研究如何利用离线训练来最小化数据成本（加速收敛）和项目实施成本。我们提出了两种方法来实现这一点：对模型进行预训练，用模拟的任务来热身开始实验，以及使用经过训练的计划模型来模拟真实世界对代理的奖励。我们给出的结果证明了离线强化学习在能源需求响应问题中有效定价的效用。 摘要：Our team is proposing to run a full-scale energy demand response experiment in an office building. Although this is an exciting endeavor which will provide value to the community, collecting training data for the reinforcement learning agent is costly and will be limited. In this work, we examine how offline training can be leveraged to minimize data costs (accelerate convergence) and program implementation costs. We present two approaches to doing so: pretraining our model to warm start the experiment with simulated tasks, and using a planning model trained to simulate the real world's rewards to the agent. We present results that demonstrate the utility of offline reinforcement learning to efficient price-setting in the energy demand response problem.

【44】 A Microscopic Pandemic Simulator for Pandemic Prediction Using Scalable Million-Agent Reinforcement Learning 标题：一种基于可扩展百万智能体强化学习的微观流感大流行模拟器 链接：https://arxiv.org/abs/2108.06589

作者：Zhenggang Tang,Kai Yan,Liting Sun,Wei Zhan,Changliu Liu 机构：Peking University, University of California, Berkeley, Carnegie Mellon University 备注：14 pages 摘要：微观传染病模型是政府决策者预测和模拟传染病暴发的有力工具，可以捕捉个体行为对宏观现象的影响。然而，现有的模型只考虑简单的基于规则的个体行为，限制了它们的适用性。提出了一种基于深度强化学习的微观模型——微观流行病模拟器（MPS）。MPS将基于规则的代理替换为理性代理，理性代理的行为被驱动以实现回报最大化，从而更好地近似真实世界的动态。为了有效地模拟MPS中的大量代理，我们提出了可扩展的百万代理DQN（SMADQN）。MPS使我们能够有效地评估不同政府战略的影响。本文首先根据美国阿勒格尼的真实数据校准了MPS，然后实证评估了两种政府战略：信息披露和隔离。结果验证了该方法的有效性。作为广泛的影响，本文为DRL在大规模基于代理的网络（如经济和社会网络）中的应用提供了新的见解。 摘要：Microscopic epidemic models are powerful tools for government policy makers to predict and simulate epidemic outbreaks, which can capture the impact of individual behaviors on the macroscopic phenomenon. However, existing models only consider simple rule-based individual behaviors, limiting their applicability. This paper proposes a deep-reinforcement-learning-powered microscopic model named Microscopic Pandemic Simulator (MPS). By replacing rule-based agents with rational agents whose behaviors are driven to maximize rewards, the MPS provides a better approximation of real world dynamics. To efficiently simulate with massive amounts of agents in MPS, we propose Scalable Million-Agent DQN (SMADQN). The MPS allows us to efficiently evaluate the impact of different government strategies. This paper first calibrates the MPS against real-world data in Allegheny, US, then demonstratively evaluates two government strategies: information disclosure and quarantine. The results validate the effectiveness of the proposed method. As a broad impact, this paper provides novel insights for the application of DRL in large scale agent-based networks such as economic and social networks.

【45】 Stacked Hourglass Network with a Multi-level Attention Mechanism: Where to Look for Intervertebral Disc Labeling 标题：具有多层次注意机制的堆叠沙漏网络：在哪里寻找椎间盘标记 链接：https://arxiv.org/abs/2108.06554

作者：Reza Azad,Lucas Rouhier,Julien Cohen-Adad 机构： NeuroPoly Lab, Institute of Biomedical Engineering, Polytechnique Montreal, Mila, Quebec AI Institute, Canada, Functional Neuroimaging Unit, CRIUGM, University of Montreal, Montreal 备注：None 摘要：从MRI扫描中标记椎间盘对于正确诊断脊柱相关疾病非常重要，包括多发性硬化症、肌萎缩侧索硬化症、退行性颈脊髓病和癌症。在MRI数据中自动标记椎间盘是一项困难的任务，因为椎间盘和骨区域之间的相似性，个体间脊柱和周围组织几何结构的变异性，以及扫描（制造商、脉冲序列、图像对比度、分辨率和人工制品）的变异性。在以前的研究中，椎间盘标记通常是在椎间盘检测步骤之后进行的，当定位算法遗漏椎间盘或检测到假阳性时，大多会失败。在这项工作中，我们的目标是通过使用姿势估计技术重新制定语义椎间盘标记来缓解这个问题。为此，我们提出了一种具有多级注意机制的叠层沙漏网络来共同学习椎间盘位置及其骨架结构。所提出的深度学习模型考虑了语义分割和姿态估计技术的优点来处理缺失区域和误报检测。为了进一步提高该方法的性能，我们提出了一种基于骨架的搜索空间来减少误报检测。该方法在spine通用公共多中心数据集上进行了评估，在T1w和T2w对比度方面，与之前的工作相比，表现出更好的性能。该方法在ivadomed中实现(https://ivadomed.org). 摘要：Labeling vertebral discs from MRI scans is important for the proper diagnosis of spinal related diseases, including multiple sclerosis, amyotrophic lateral sclerosis, degenerative cervical myelopathy and cancer. Automatic labeling of the vertebral discs in MRI data is a difficult task because of the similarity between discs and bone area, the variability in the geometry of the spine and surrounding tissues across individuals, and the variability across scans (manufacturers, pulse sequence, image contrast, resolution and artefacts). In previous studies, vertebral disc labeling is often done after a disc detection step and mostly fails when the localization algorithm misses discs or has false positive detection. In this work, we aim to mitigate this problem by reformulating the semantic vertebral disc labeling using the pose estimation technique. To do so, we propose a stacked hourglass network with multi-level attention mechanism to jointly learn intervertebral disc position and their skeleton structure. The proposed deep learning model takes into account the strength of semantic segmentation and pose estimation technique to handle the missing area and false positive detection. To further improve the performance of the proposed method, we propose a skeleton-based search space to reduce false positive detection. The proposed method evaluated on spine generic public multi-center dataset and demonstrated better performance comparing to previous work, on both T1w and T2w contrasts. The method is implemented in ivadomed (https://ivadomed.org).

【46】 Neuron Campaign for Initialization Guided by Information Bottleneck Theory 标题：信息瓶颈理论指导的神经元初始化运动 链接：https://arxiv.org/abs/2108.06530

作者：Haitao Mao,Xu Chen,Qiang Fu,Lun Du,Shi Han,Dongmei Zhang 机构：University of Electronic Science and, Technology of China, Chengdu, China, Peking University, Beijing, China, Microsoft Research Asia, Domei Zhang 备注：5 pages, Accepted by CIKM'21 摘要：初始化在深度神经网络（DNN）的训练中起着至关重要的作用。现有的初始化策略主要侧重于稳定训练过程，以缓解梯度消失/爆炸问题。然而，这些初始化方法缺乏对如何提高泛化能力的考虑。信息瓶颈（IB）理论是一个著名的解释DNN泛化的理解框架。在IB理论的指导下，我们设计了两个更好地初始化DNN的标准。我们进一步设计了一种神经元运动初始化算法，以便在给定的数据集上为神经网络有效地选择一个良好的初始化。在MNIST数据集上的实验表明，该方法具有更快的收敛速度和更好的泛化性能。 摘要：Initialization plays a critical role in the training of deep neural networks (DNN). Existing initialization strategies mainly focus on stabilizing the training process to mitigate gradient vanish/explosion problems. However, these initialization methods are lacking in consideration about how to enhance generalization ability. The Information Bottleneck (IB) theory is a well-known understanding framework to provide an explanation about the generalization of DNN. Guided by the insights provided by IB theory, we design two criteria for better initializing DNN. And we further design a neuron campaign initialization algorithm to efficiently select a good initialization for a neural network on a given dataset. The experiments on MNIST dataset show that our method can lead to a better generalization performance with faster convergence.

【47】 Fractional Transfer Learning for Deep Model-Based Reinforcement Learning 标题：基于深度模型强化学习的分数转移学习 链接：https://arxiv.org/abs/2108.06526

作者：Remo Sasso,Matthia Sabatelli,Marco A. Wiering 机构：Dept. Artificial Intelligence, University of Groningen 备注：21 pages, 8 figures, 7 tables 摘要：强化学习（RL）因需要大量数据才能让RL代理学习执行复杂任务而闻名。基于模型的RL的最新进展使代理能够更加高效地处理数据，因为它使代理能够利用环境的内部世界模型在想象中学习视觉环境的行为。通过重用以前学习过的任务中的知识也可以提高样本效率，但迁移学习在RL中仍然是一个具有挑战性的课题。基于参数的迁移学习通常采用全有或全无的方法，其中网络的参数要么完全转移，要么随机初始化。在这项工作中，我们提出了一种简单的替代方法：分数转移学习。其思想是转移知识的一小部分，而不是像随机初始化那样丢弃潜在有用的知识。使用基于世界模型的Dreamer算法，我们确定该方法适用于哪种类型的组件，并在新的多源迁移学习环境中进行实验。结果表明，与从头开始学习和随机初始化相比，分数转移学习通常能显著提高性能和更快的学习速度。 摘要：Reinforcement learning (RL) is well known for requiring large amounts of data in order for RL agents to learn to perform complex tasks. Recent progress in model-based RL allows agents to be much more data-efficient, as it enables them to learn behaviors of visual environments in imagination by leveraging an internal World Model of the environment. Improved sample efficiency can also be achieved by reusing knowledge from previously learned tasks, but transfer learning is still a challenging topic in RL. Parameter-based transfer learning is generally done using an all-or-nothing approach, where the network's parameters are either fully transferred or randomly initialized. In this work we present a simple alternative approach: fractional transfer learning. The idea is to transfer fractions of knowledge, opposed to discarding potentially useful knowledge as is commonly done with random initialization. Using the World Model-based Dreamer algorithm, we identify which type of components this approach is applicable to, and perform experiments in a new multi-source transfer learning setting. The results show that fractional transfer learning often leads to substantially improved performance and faster learning compared to learning from scratch and random initialization.

【48】 Appropriate Fairness Perceptions? On the Effectiveness of Explanations in Enabling People to Assess the Fairness of Automated Decision Systems 标题：适当的公平观念？论解释在评价自动决策系统公平性中的有效性 链接：https://arxiv.org/abs/2108.06500

作者：Jakob Schoeffer,Niklas Kuehl 机构：Karlsruhe Institute of Technology (KIT), Germany 备注：Companion Publication of the 2021 Conference on Computer Supported Cooperative Work and Social Computing (CSCW '21 Companion), October 23--27, 2021, Virtual Event, USA 摘要：人们经常认为，解释自动决策系统（ADS）的一个目标是促进用户对此类系统的积极认知（例如，公平性或可信度）。然而，这一观点从一开始就隐含着一个假设，即给定的广告是公平和可信的。如果ADS发布了不公平的结果，那么人们可能会期望有关系统运行的解释将揭示其缺陷，从而导致公平观念的下降。因此，我们建议，根据解释的有效性来评估解释更具意义，使人们能够适当地评估相关广告的质量（例如，公平性）。我们认为，对于有效的解释，只有当且仅当基础广告是公平的时，公平感才会增加。在这项正在进行的工作中，我们介绍了适当公平感知的必要性，提出了一个新的研究设计来评估它，并概述了迈向全面实验的下一步。 摘要：It is often argued that one goal of explaining automated decision systems (ADS) is to facilitate positive perceptions (e.g., fairness or trustworthiness) of users towards such systems. This viewpoint, however, makes the implicit assumption that a given ADS is fair and trustworthy, to begin with. If the ADS issues unfair outcomes, then one might expect that explanations regarding the system's workings will reveal its shortcomings and, hence, lead to a decrease in fairness perceptions. Consequently, we suggest that it is more meaningful to evaluate explanations against their effectiveness in enabling people to appropriately assess the quality (e.g., fairness) of an associated ADS. We argue that for an effective explanation, perceptions of fairness should increase if and only if the underlying ADS is fair. In this in-progress work, we introduce the desideratum of appropriate fairness perceptions, propose a novel study design for evaluating it, and outline next steps towards a comprehensive experiment.

【49】 Joint Optimization in Edge-Cloud Continuum for Federated Unsupervised Person Re-identification 标题：联合无监督人员身份识别的边云连续体联合优化 链接：https://arxiv.org/abs/2108.06493

作者：Weiming Zhuang,Yonggang Wen,Shuai Zhang 机构：S-Lab, Nanyang Technological University, SenseTime Research 备注：ACMMM'21 摘要：人员重新识别（ReID）旨在从非重叠摄像机视图中重新识别人员。由于person ReID数据包含敏感的个人信息，研究人员采用了联邦学习（federated learning）这一新兴的分布式训练方法来降低隐私泄露风险。然而，现有的研究依赖于数据标签，获取这些标签费时费力。我们提出FedUReID，一个联邦的无监督的person-ReID系统来学习person-ReID模型，而不需要任何标签，同时保护隐私。FedUReID允许在带有未标记数据的边上进行现场模型训练。云服务器从边缘聚合模型，而不是集中原始数据以保护数据隐私。此外，为了解决边缘在数据量和分布方面存在差异的问题，我们通过云和边缘的联合优化对边缘进行个性化训练。具体地说，我们提出了个性化epoch来重新分配整个训练过程中的计算，个性化聚类来迭代地预测未标记数据的合适标签，以及个性化更新来使服务器聚合模型适应每个边缘。在8人ReID数据集上的大量实验表明，FedUReID不仅具有更高的精度，而且还将计算成本降低了29%。我们的FedUReID系统和联合优化将有助于实现联邦学习，使更多的多媒体任务不需要数据标签。 摘要：Person re-identification (ReID) aims to re-identify a person from non-overlapping camera views. Since person ReID data contains sensitive personal information, researchers have adopted federated learning, an emerging distributed training method, to mitigate the privacy leakage risks. However, existing studies rely on data labels that are laborious and time-consuming to obtain. We present FedUReID, a federated unsupervised person ReID system to learn person ReID models without any labels while preserving privacy. FedUReID enables in-situ model training on edges with unlabeled data. A cloud server aggregates models from edges instead of centralizing raw data to preserve data privacy. Moreover, to tackle the problem that edges vary in data volumes and distributions, we personalize training in edges with joint optimization of cloud and edge. Specifically, we propose personalized epoch to reassign computation throughout training, personalized clustering to iteratively predict suitable labels for unlabeled data, and personalized update to adapt the server aggregated model to each edge. Extensive experiments on eight person ReID datasets demonstrate that FedUReID not only achieves higher accuracy but also reduces computation cost by 29%. Our FedUReID system with the joint optimization will shed light on implementing federated learning to more multimedia tasks without data labels.

【50】 Collaborative Unsupervised Visual Representation Learning from Decentralized Data 标题：分散数据的协作式无监督视觉表示学习 链接：https://arxiv.org/abs/2108.06492

作者：Weiming Zhuang,Xin Gan,Yonggang Wen,Shuai Zhang,Shuai Yi 机构：S-Lab, Nanyang Technological University,Nanyang Technological University,SenseTime Research 备注：ICCV'21 摘要：利用互联网上的集中数据，无监督表征学习取得了优异的成绩。然而，隐私保护意识的提高限制了分散的未标记图像数据的共享，这些数据在多方（如手机和相机）中爆炸性增长。因此，一个自然的问题是如何利用这些数据来学习下游任务的可视化表示，同时保护数据隐私。为了解决这个问题，我们提出了一个新的联邦无监督学习框架FedU。在这个框架中，各方使用在线网络和目标网络的对比学习，独立地从未标记的数据中训练模型。然后，中央服务器聚合经过训练的模型，并使用聚合的模型更新客户机的模型。它保护了数据隐私，因为各方只能访问其原始数据。多方之间分散的数据通常是非独立且分布相同的（非IID），导致性能下降。为了应对这一挑战，我们提出了两种简单但有效的方法：1）设计通信协议，仅上传在线网络的编码器进行服务器聚合，并使用聚合编码器进行更新；2） 我们引入了一个新的模块，根据非IID引起的偏差动态决定如何更新预测器。预测器是在线网络的另一个组成部分。大量的实验和烧蚀证明了FedU的有效性和重要性。在非IID数据的线性和半监督评估中，它的表现优于仅使用一方的训练，超过5%，其他方法超过14%。 摘要：Unsupervised representation learning has achieved outstanding performances using centralized data available on the Internet. However, the increasing awareness of privacy protection limits sharing of decentralized unlabeled image data that grows explosively in multiple parties (e.g., mobile phones and cameras). As such, a natural problem is how to leverage these data to learn visual representations for downstream tasks while preserving data privacy. To address this problem, we propose a novel federated unsupervised learning framework, FedU. In this framework, each party trains models from unlabeled data independently using contrastive learning with an online network and a target network. Then, a central server aggregates trained models and updates clients' models with the aggregated model. It preserves data privacy as each party only has access to its raw data. Decentralized data among multiple parties are normally non-independent and identically distributed (non-IID), leading to performance degradation. To tackle this challenge, we propose two simple but effective methods: 1) We design the communication protocol to upload only the encoders of online networks for server aggregation and update them with the aggregated encoder; 2) We introduce a new module to dynamically decide how to update predictors based on the divergence caused by non-IID. The predictor is the other component of the online network. Extensive experiments and ablations demonstrate the effectiveness and significance of FedU. It outperforms training with only one party by over 5% and other methods by over 14% in linear and semi-supervised evaluation on non-IID data.

【51】 Investigating Bias In Automatic Toxic Comment Detection: An Empirical Study 标题：毒物评论自动检测中的偏差调查：一项实证研究 链接：https://arxiv.org/abs/2108.06487

作者：Ayush Kumar,Pratik Kumar 机构：Georgia Institute of Technology, Atlanta, US 摘要：随着在线平台的激增，通过评论和反应，用户在这些平台上的参与度也在激增。这些文字评论中有很大一部分是辱骂、粗鲁和冒犯观众的。有了机器学习系统来检查平台上的评论，训练数据中存在的偏见就会传递到分类器上，导致对一组阶级、宗教和性别的歧视。在这项工作中，我们评估了不同的分类器和特征，以估计这些分类器中的偏差以及它们在毒性分类下游任务中的性能。结果表明，自动毒性评论检测模型性能的改善与这些模型中的偏差的缓解正相关。在我们的工作中，有注意机制的LSTM被证明是比CNN模型更好的建模策略。进一步的分析表明，在毒性评价检测的训练模型上，fasttext嵌入略优于手套嵌入。更深入的分析揭示了这样一个发现，即这种自动模型特别偏向于特定的身份群体，即使该模型具有较高的AUC分数。最后，为了减轻毒性检测模型中的偏差，用毒性亚型辅助任务训练的多任务设置被证明是有用的，导致AUC得分增加高达0.26%（6%相对）。 摘要：With surge in online platforms, there has been an upsurge in the user engagement on these platforms via comments and reactions. A large portion of such textual comments are abusive, rude and offensive to the audience. With machine learning systems in-place to check such comments coming onto platform, biases present in the training data gets passed onto the classifier leading to discrimination against a set of classes, religion and gender. In this work, we evaluate different classifiers and feature to estimate the bias in these classifiers along with their performance on downstream task of toxicity classification. Results show that improvement in performance of automatic toxic comment detection models is positively correlated to mitigating biases in these models. In our work, LSTM with attention mechanism proved to be a better modelling strategy than a CNN model. Further analysis shows that fasttext embeddings is marginally preferable than glove embeddings on training models for toxicity comment detection. Deeper analysis reveals the findings that such automatic models are particularly biased to specific identity groups even though the model has a high AUC score. Finally, in effort to mitigate bias in toxicity detection models, a multi-task setup trained with auxiliary task of toxicity sub-types proved to be useful leading to upto 0.26% (6% relative) gain in AUC scores.

【52】 MatSat: a matrix-based differentiable SAT solver 标题：MatSat：一种基于矩阵的可微SAT求解器 链接：https://arxiv.org/abs/2108.06481

作者：Taisuke Sato,Ryosuke Kojima 机构： National Institute of Informatics (NII), Tokyo, Japan, Graduate School of Medicine, Kyoto University, Japan 摘要：我们提出了一种新的SAT求解方法，该方法将向量空间中的SAT问题作为一个非负可微代价函数J^SAT的代价最小化问题来求解。在我们的方法中，n个变量中的SAT问题的解，即满足赋值，由{0,1}^n中的一个使J^SAT（u）为零的二进制向量u表示。我们通过成本最小化在向量空间R^n中搜索这样的u，即从初始u_0开始，将J最小化为零，同时通过牛顿方法迭代更新u。我们将我们的方法实现为基于矩阵的差分SAT解算器MatSat。尽管现有主流SAT解算器逐个决定解决方案分配的每一位，无论是冲突驱动子句学习（CDCL）类型还是随机局部搜索（SLS）类型，但MatSat与它们的根本区别在于它在向量空间中不断逼近解。我们进行了一项实验，用随机3-SAT问题来衡量MatSat的可伸缩性，其中MatSat可以找到最多n=10^5个变量的解决方案。我们还使用2018年SAT竞赛的随机基准集和人工随机3-SAT实例集，将MatSat与四个最先进的SAT解算器进行了比较，其中包括2018年SAT竞赛和2019年SAT竞赛的获胜者。结果表明，MatSat在两个测试集中均排名第二，优于所有CDCL类型的解算器。 摘要：We propose a new approach to SAT solving which solves SAT problems in vector spaces as a cost minimization problem of a non-negative differentiable cost function J^sat. In our approach, a solution, i.e., satisfying assignment, for a SAT problem in n variables is represented by a binary vector u in {0,1}^n that makes J^sat(u) zero. We search for such u in a vector space R^n by cost minimization, i.e., starting from an initial u_0 and minimizing J to zero while iteratively updating u by Newton's method. We implemented our approach as a matrix-based differential SAT solver MatSat. Although existing main-stream SAT solvers decide each bit of a solution assignment one by one, be they of conflict driven clause learning (CDCL) type or of stochastic local search (SLS) type, MatSat fundamentally differs from them in that it continuously approach a solution in a vector space. We conducted an experiment to measure the scalability of MatSat with random 3-SAT problems in which MatSat could find a solution up to n=10^5 variables. We also compared MatSat with four state-of-the-art SAT solvers including winners of SAT competition 2018 and SAT Race 2019 in terms of time for finding a solution, using a random benchmark set from SAT 2018 competition and an artificial random 3-SAT instance set. The result shows that MatSat comes in second in both test sets and outperforms all the CDCL type solvers.

【53】 Contrastive Self-supervised Sequential Recommendation with Robust Augmentation 标题：稳健增强的对比性自监督序贯推荐 链接：https://arxiv.org/abs/2108.06479

作者：Zhiwei Liu,Yongjun Chen,Jia Li,Philip S. Yu,Julian McAuley,Caiming Xiong 机构：University of Illinois at Chicago, Salesforce Research, UC San Diego 备注：Under-review. Work done during Zhiwei's intern at Salesforce. Inc 摘要：顺序推荐描述了一组建模动态用户行为的技术，以预测顺序用户数据中的未来交互。这些方法的核心是对序列中项目之间的转移概率进行建模，无论是通过马尔可夫链、循环网络，还是最近的Transformer。然而，新老问题仍然存在，包括数据稀疏和噪声数据；这些问题可能会影响性能，特别是在复杂的、需要参数的模型中。在本文中，我们研究了对比自监督学习（SSL）在顺序推荐中的应用，以缓解其中的一些问题。对比SSL构造了未标记实例的扩充，其中正对之间的协议最大化。由于顺序推荐的离散性、项目之间的相关性和长度分布的偏斜性，设计一个用于顺序推荐的对比SSL框架是一个挑战。为此，我们提出了一个新的框架，顺序推荐对比自监督学习（CoSeRec）。我们介绍了两个利用项目相关性创建高质量对比学习视图的信息增强算子。在三个真实数据集上的实验结果表明了该方法在提高模型性能和对稀疏和噪声数据的鲁棒性方面的有效性。我们的实现可在线访问\url{https://github.com/YChen1993/CoSeRec} 摘要：Sequential Recommendationdescribes a set of techniques to model dynamic user behavior in order to predict future interactions in sequential user data. At their core, such approaches model transition probabilities between items in a sequence, whether through Markov chains, recurrent networks, or more recently, Transformers. However both old and new issues remain, including data-sparsity and noisy data; such issues can impair the performance, especially in complex, parameter-hungry models. In this paper, we investigate the application of contrastive Self-Supervised Learning (SSL) to the sequential recommendation, as a way to alleviate some of these issues. Contrastive SSL constructs augmentations from unlabelled instances, where agreements among positive pairs are maximized. It is challenging to devise a contrastive SSL framework for a sequential recommendation, due to its discrete nature, correlations among items, and skewness of length distributions. To this end, we propose a novel framework, Contrastive Self-supervised Learning for sequential Recommendation (CoSeRec). We introduce two informative augmentation operators leveraging item correlations to create high-quality views for contrastive learning. Experimental results on three real-world datasets demonstrate the effectiveness of the proposed method on improving model performance and the robustness against sparse and noisy data. Our implementation is available online at \url{https://github.com/YChen1993/CoSeRec}

【54】 Optimal Approximation with Sparse Neural Networks and Applications 标题：稀疏神经网络的最优逼近及其应用 链接：https://arxiv.org/abs/2108.06467

作者：Khay Boon Hong 机构： Their performancesare compared and concluded based on the datasets in Modified National Instituteof Standards and Technology (MNIST) or the ImageNet Large Scale Visual Recog-nition Challenge (ILSVRC) [Sandler et al 备注：37 pages, no figures. Undergraduate Final Year Project 摘要：我们使用深度稀疏连接的神经网络，通过限制存储神经网络的连接性和内存需求，以$L^2（\mathbb R^d）$度量函数类的复杂性。我们还介绍了表示系统——一个用于指导神经网络的可数函数集合，因为表示系统的近似理论在数学上已经得到了很好的发展。然后，我们证明了基本定界定理，这意味着函数类本身固有的一个量可以提供有关神经网络和表示系统的逼近能力的信息。我们还提供了一种将现有的表示系统近似理论转换为神经网络近似理论的方法，极大地扩大了神经网络的实用价值。最后，利用神经网络逼近B样条函数，生成B样条曲线。然后，我们利用率失真理论和楔子构造分析了一类名为$\beta$卡通函数的复杂性。 摘要：We use deep sparsely connected neural networks to measure the complexity of a function class in $L^2(\mathbb R^d)$ by restricting connectivity and memory requirement for storing the neural networks. We also introduce representation system - a countable collection of functions to guide neural networks, since approximation theory with representation system has been well developed in Mathematics. We then prove the fundamental bound theorem, implying a quantity intrinsic to the function class itself can give information about the approximation ability of neural networks and representation system. We also provides a method for transferring existing theories about approximation by representation systems to that of neural networks, greatly amplifying the practical values of neural networks. Finally, we use neural networks to approximate B-spline functions, which are used to generate the B-spline curves. Then, we analyse the complexity of a class called $\beta$ cartoon-like functions using rate-distortion theory and wedgelets construction.

【55】 Metadata-based Multi-Task Bandits with Bayesian Hierarchical Models 标题：基于贝叶斯层次模型的元数据多任务BITITS 链接：https://arxiv.org/abs/2108.06422

作者：Runzhe Wan,Lin Ge,Rui Song 机构：Department of Statistics, North Carolina State University 摘要：如何有效地探索是多武装匪徒研究的中心问题。在本文中，我们介绍了基于元数据的多任务bandit问题，其中agent需要解决大量相关的多武装bandit任务，并且可以利用一些特定于任务的特性（即元数据）跨任务共享知识。作为一个通用框架，我们提出通过贝叶斯层次模型的视角来捕捉任务关系，在此基础上设计了一个汤普森抽样算法，以有效地学习任务关系、共享信息和最小化累积遗憾。详细分析了高斯土匪和伯努利土匪的两个具体例子。高斯土匪的贝叶斯遗憾清楚地表明了我们的算法共享信息的好处。大量实验进一步证明了该方法的有效性。 摘要：How to explore efficiently is a central problem in multi-armed bandits. In this paper, we introduce the metadata-based multi-task bandit problem, where the agent needs to solve a large number of related multi-armed bandit tasks and can leverage some task-specific features (i.e., metadata) to share knowledge across tasks. As a general framework, we propose to capture task relations through the lens of Bayesian hierarchical models, upon which a Thompson sampling algorithm is designed to efficiently learn task relations, share information, and minimize the cumulative regrets. Two concrete examples for Gaussian bandits and Bernoulli bandits are carefully analyzed. The Bayes regret for Gaussian bandits clearly demonstrates the benefits of information sharing with our algorithm. The proposed method is further supported by extensive experiments.

【56】 Planning with Incomplete Information in Quantified Answer Set Programming 标题：量化答案集规划中的不完全信息规划 链接：https://arxiv.org/abs/2108.06405

作者：Jorge Fandinno,François Laferrière,Javier Romero,Torsten Schaub,Tran Cao Son 机构：New Mexico State University, USA, Omaha State University, USA, University of Potsdam, Germany 备注：Under consideration for publication in Theory and Practice of Logic Programming (TPLP) 摘要：我们提出了一种在回答集编程（ASP）中使用不完全信息进行规划的通用方法。更确切地说，我们考虑一致性和有条件规划的问题与传感行动和假设。我们使用一种简单的形式来表示规划问题，其中逻辑程序描述状态、初始状态和目标状态之间的转换函数。为了解决规划问题，我们使用量化答案集编程（QASP），这是ASP的一个扩展，在原子上具有存在和通用量词，类似于量化布尔公式（QBF）。我们定义了量化逻辑程序的语言，并使用它来表示一致性规划和条件规划的不同变体的解决方案。在实践方面，我们提出了一种基于翻译的QASP求解器，它将量化逻辑程序转换为QBF，然后执行QBF求解器，并在一致性和条件规划基准上对该方法进行了实验评估。正在考虑在TPLP中验收。 摘要：We present a general approach to planning with incomplete information in Answer Set Programming (ASP). More precisely, we consider the problems of conformant and conditional planning with sensing actions and assumptions. We represent planning problems using a simple formalism where logic programs describe the transition function between states, the initial states and the goal states. For solving planning problems, we use Quantified Answer Set Programming (QASP), an extension of ASP with existential and universal quantifiers over atoms that is analogous to Quantified Boolean Formulas (QBFs). We define the language of quantified logic programs and use it to represent the solutions to different variants of conformant and conditional planning. On the practical side, we present a translation-based QASP solver that converts quantified logic programs into QBFs and then executes a QBF solver, and we evaluate experimentally the approach on conformant and conditional planning benchmarks. Under consideration for acceptance in TPLP.

【57】 Near-Optimal Reviewer Splitting in Two-Phase Paper Reviewing and Conference Experiment Design 标题：两期审稿与会议实验设计中的近优评委分派 链接：https://arxiv.org/abs/2108.06371

作者：Steven Jecmen,Hanrui Zhang,Ryan Liu,Fei Fang,Vincent Conitzer,Nihar B. Shah 机构：Carnegie Mellon University, Duke University 摘要：许多科学会议采用两阶段的论文评审过程，在提交初始评审后，一些论文会被指派额外的评审员。许多会议还设计并运行论文评审过程的实验，其中一些论文被指定为在实验条件下提供评审的评审员。在本文中，我们考虑的问题：如何审计员分阶段或条件，以最大限度地提高总的分配相似性？我们为回答这个问题做出了几点贡献。首先，我们证明了当需要额外审查的论文集未知时，这个问题的一个简化变体是NP难问题。第二，我们的经验表明，在与实际会议数据相关的多个数据集中，在阶段/条件之间均匀随机地划分审阅者，可以实现几乎与oracle最佳分配一样好的分配。这种均匀随机选择对于两阶段和会议实验设计设置都是可行的。第三，我们通过提供在特定自然条件下该随机策略次优性的理论界来解释这一现象。从这些易于解释的条件出发，我们为会议项目主席提供了关于随机评审员分组是否适合其会议的可操作见解。 摘要：Many scientific conferences employ a two-phase paper review process, where some papers are assigned additional reviewers after the initial reviews are submitted. Many conferences also design and run experiments on their paper review process, where some papers are assigned reviewers who provide reviews under an experimental condition. In this paper, we consider the question: how should reviewers be divided between phases or conditions in order to maximize total assignment similarity? We make several contributions towards answering this question. First, we prove that when the set of papers requiring additional review is unknown, a simplified variant of this problem is NP-hard. Second, we empirically show that across several datasets pertaining to real conference data, dividing reviewers between phases/conditions uniformly at random allows an assignment that is nearly as good as the oracle optimal assignment. This uniformly random choice is practical for both the two-phase and conference experiment design settings. Third, we provide explanations of this phenomenon by providing theoretical bounds on the suboptimality of this random strategy under certain natural conditions. From these easily-interpretable conditions, we provide actionable insights to conference program chairs about whether a random reviewer split is suitable for their conference.

【58】 Bayesian Parameter Estimations for Grey System Models in Online Traffic Speed Predictions 标题：在线车速预测中灰色系统模型的贝叶斯参数估计 链接：https://arxiv.org/abs/2108.06839

作者：Gurcan Comert,Negash Begashaw,Negash G. Medhin 机构：Department of Computer Science, Physics, and Engineering, Benedict College, Columbia, SC , USA, Information Trust Institute, University of Illinois Urbana-Champaign, Urbana, IL , USA 备注：11 pages, 6 figures 摘要：本文提出了一阶灰色系统模型参数（有时也称为超参数）的贝叶斯参数估计。一阶灰色系统模型有不同的形式。其中包括$GM（1,1）$，$GM（1,1 | \cos（\omega t）$，$GM（1,1 | \sin（\omega t）$）和$GM（1,1 | \cos（\omega t），\sin（\omega t）$。这些模型的白化方程是一阶线性微分方程，形式为\[\frac{dx}{dt}+a x=f（t）\]，其中$a$是参数，$f（t）=b$in$GM（1,1 |）$，$f（t）=b_1\cos（\omega t）+b_2$GM（1,1 | cos（\omega t）$，$f（t）=b_1\sin sin（\omega t）+b|2$，GM，$f（t）=b_1\sin（\omega t）+b_2\cos（\omega t）+b_3$in$GM（1,1\cos（\omega t）），\sin（\omega t）$，$f（t）=b x^2$in灰色Verhulst模型（GVM），其中$b、b_1、b_2$和$b_3$是参数。将贝叶斯估计的结果与固定$\omega$的最小二乘估计模型进行比较。我们发现，对GM参数使用滚动贝叶斯估计可以让我们以所有可能的形式估计参数。基于用于比较的数据，数值结果表明，采用贝叶斯参数估计的模型的均方误差精度高达45%。 摘要：This paper presents Bayesian parameter estimation for first order Grey system models' parameters (or sometimes referred to as hyperparameters). There are different forms of first-order Grey System Models. These include $GM(1,1)$, $GM(1,1| \cos(\omega t)$, $GM(1,1| \sin(\omega t)$, and $GM(1,1| \cos(\omega t), \sin(\omega t)$. The whitenization equation of these models is a first-order linear differential equation of the form \[ \frac{dx}{dt} + a x = f(t) \] where $a$ is a parameter and $f(t) = b$ in $GM(1,1|)$ , $f(t) = b_1\cos(\omega t) + b_2$ in $GM(1,1| cos(\omega t)$, $f(t) = b_1\sin(\omega t)+b_2$ in $GM(1,1| \sin(\omega t)$, $f(t) = b_1\sin(\omega t) + b_2\cos(\omega t) + b_3$ in $GM(1,1| \cos(\omega t), \sin(\omega t)$, $f(t) = b x^2$ in Grey Verhulst model (GVM), and where $b, b_1, b_2$, and $b_3$ are parameters. The results from Bayesian estimations are compared to the least square estimated models with fixed $\omega$. We found that using rolling Bayesian estimations for GM parameters can allow us to estimate the parameters in all possible forms. Based on the data used for the comparison, the numerical results showed that models with Bayesian parameter estimations are up to 45\% more accurate in mean squared errors.

【59】 The Proximal ID Algorithm 标题：近似值ID算法 链接：https://arxiv.org/abs/2108.06818

作者：Ilya Shpitser,Zach Wood-Doughty,Eric J. Tchetgen Tchetgen 机构：Department of Computer Science, Johns Hopkins University, Baltimore, MD , USA, Department of Statistics, The Wharton School, Locust Walk, Philadelphia, PA , USA, Editor: Anonymous 摘要：未观察到的混淆是从观察数据中建立有效因果结论的根本障碍。为解决这一障碍，已经制定了两种互补的办法。广泛的工作是基于利用偶然的外部辅助（例如存在工具变量或其他代理），以及其他假设来确保识别。最近一系列的近端因果推断（Miao等人，2018a）旨在提供一种新的方法，在不依赖严格的参数假设的情况下，使用代理来处理未观察到的混淆。另一方面，使用图形模型语言开发了具有隐藏变量的任意因果模型中一大类因果参数的可识别性的完整表征，从而产生了ID算法和相关扩展（Tian和Pearl，2002；希皮瑟和珀尔，2006a，b）。这种方法的著名特例，如前门模型，能够在看似违反直觉的情况下获得非参数识别，当治疗和结果共享任意复杂的未观察到的共同原因时。在本文中，我们的目标是开发一个综合的近端和图形化的方法来识别因果推理，以产生目前已知的多变量系统中最通用的识别算法-近端ID算法。除了能够在ID算法成功的所有情况下获得非参数识别外，我们的方法还允许我们系统地利用代理来调整未观察到的混杂因素的存在，否则将阻止识别。此外，在一个重要的特殊情况下，我们概述了一类由我们的方法识别的因果参数的估计策略。我们通过模拟研究来说明我们的方法。 摘要：Unobserved confounding is a fundamental obstacle to establishing valid causal conclusions from observational data. Two complementary types of approaches have been developed to address this obstacle. An extensive line of work is based on taking advantage of fortuitous external aids (such as the presence of an instrumental variable or other proxy), along with additional assumptions to ensure identification. A recent line of work of proximal causal inference (Miao et al., 2018a) has aimed to provide a novel approach to using proxies to deal with unobserved confounding without relying on stringent parametric assumptions. On the other hand, a complete characterization of identifiability of a large class of causal parameters in arbitrary causal models with hidden variables has been developed using the language of graphical models, resulting in the ID algorithm and related extensions (Tian and Pearl, 2002; Shpitser and Pearl, 2006a,b). Celebrated special cases of this approach, such as the front-door model, are able to obtain non-parametric identification in seemingly counter-intuitive situations when a treatment and an outcome share an arbitrarily complicated unobserved common cause. In this paper we aim to develop a synthesis of the proximal and graphical approaches to identification in causal inference to yield the most general identification algorithm in multi- variate systems currently known - the proximal ID algorithm. In addition to being able to obtain non-parametric identification in all cases where the ID algorithm succeeds, our approach allows us to systematically exploit proxies to adjust for the presence of unobserved confounders that would have otherwise prevented identification. In addition, we outline a class of estimation strategies for causal parameters identified by our method in an important special case. We illustration our approach by simulation studies.