The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

Psychological Review, Vol. 65, No. 6, 1958, pp. 386–408

《心理学评论》（Psychological Review），第 65 卷第 6 期，1958 年，第 386–408 页

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain¹

感知机：大脑中信息存储与组织的概率模型¹

F. Rosenblatt, Cornell Aeronautical Laboratory

F. 罗森布拉特（F. Rosenblatt），康奈尔航空实验室

If we are eventually to understand the capability of higher organisms for perceptual recognition, generalization, recall, and thinking, we must first have answers to three fundamental questions:

1. How is information about the physical world sensed, or detected, by the biological system?
2. In what form is information stored, or remembered?
3. How does information contained in storage, or in memory, influence recognition and behavior?

如果我们最终要理解高等生物在知觉识别、泛化、回忆和思维方面的能力，就必须先回答三个基本问题：

1. 关于物理世界的信息是如何被生物系统感知或检测到的？
2. 信息以什么形式被存储或记住？
3. 存储中或记忆中的信息如何影响识别和行为？

The first of these questions is in the province of sensory physiology, and is the only one for which appreciable understanding has been achieved. This article will be concerned primarily with the second and third questions, which are still subject to a vast amount of speculation, and where the few relevant facts currently supplied by neurophysiology have not yet been integrated into an acceptable theory.

第一个问题属于感觉生理学的领域，也是三个问题中唯一已经取得相当理解的一个。本文主要讨论第二和第三个问题：它们仍有大量推测成分，而神经生理学目前提供的少数相关事实，还没有被整合进一个可以接受的理论。

With regard to the second question, two alternative positions have been maintained. The first suggests that storage of sensory information is in the form of coded representations or images, with some sort of one-to-one mapping between the sensory stimulus and the stored pattern. According to this hypothesis, if one understood the code or "wiring diagram" of the nervous system, one should, in principle, be able to discover exactly what an organism remembers by reconstructing the original sensory patterns from the "memory traces" which they have left, much as we might develop a photographic negative, or translate the pattern of electrical charges in the "memory" of a digital computer. This hypothesis is appealing in its simplicity and ready intelligibility, and a large family of theoretical brain models has been developed around the idea of a coded, representational memory (2, 3, 9, 14). The alternative approach, which stems from the tradition of British empiricism, hazards the guess that the images of stimuli may never really be recorded at all, and that the central nervous system simply acts as an intricate switching network, where retention takes the form of new connections, or pathways, between centers of activity. In many of the more recent developments of this position (Hebb's "cell assembly," and Hull's "cortical anticipatory goal response," for example) the "responses" which are associated to stimuli may be entirely contained within the CNS itself. In this case the response represents an "idea" rather than an action. The important feature of this approach is that there is never any simple mapping of the stimulus into memory, according to some code which would permit its later reconstruction. Whatever information is retained must somehow be stored as a preference for a particular response; i.e., the information is contained in connections or associations rather than topographic representations. (The term response, for the remainder of this presentation, should be understood to mean any distinguishable state of the organism, which may or may not involve externally detectable muscular activity. The activation of some nucleus of cells in the central nervous system, for example, can constitute a response, according to this definition.)

关于第二个问题，存在两种对立的立场。第一种认为，感觉信息以编码表征或图像的形式存储，感觉刺激与所存储的模式之间存在某种一一映射。按照这一假说，如果我们理解了神经系统的编码或“接线图”，原则上就应该能够从感觉模式留下的“记忆痕迹”中重建出原始感觉模式，从而准确发现一个生物记住了什么，就像冲洗一张照相底片，或者解读数字计算机“存储器”中的电荷模式一样。这一假说因其简单易懂而颇具吸引力，围绕编码式、表征式记忆的观念已经发展出一大批理论性的大脑模型（2, 3, 9, 14）。另一种思路源于英国经验主义传统，它大胆推测：刺激的图像也许根本从未被真正记录下来，中枢神经系统只是充当一个复杂的开关网络，记忆的保持表现为各活动中心之间形成新的连接或通路。在这一立场较新的许多发展中（例如赫布的“细胞集合”和赫尔的“皮层预期目标反应”），与刺激相联结的“反应”可能完全包含在中枢神经系统内部。这时，反应代表的是一个“观念”而非一个动作。这种思路的重要特点在于：刺激从来不会按照某种允许日后重建的编码被简单地映射到记忆中。无论保留下什么信息，都必须以对某个特定反应的偏好的形式存储；也就是说，信息包含在连接或联想之中，而不是包含在地形式的表征之中。（在本文余下部分，反应一词应理解为生物的任何可区分状态，它可以涉及、也可以不涉及外部可检测的肌肉活动。按照这一定义，例如中枢神经系统中某个细胞核团的激活，就可以构成一个反应。）

Corresponding to these two positions on the method of information retention, there exist two hypotheses with regard to the third question, the manner in which stored information exerts its influence on current activity. The "coded memory theorists" are forced to conclude that recognition of any stimulus involves the matching or systematic comparison of the contents of storage with incoming sensory patterns, in order to determine whether the current stimulus has been seen before, and to determine the appropriate response from the organism. The theorists in the empiricist tradition, on the other hand, have essentially combined the answer to the third question with their answer to the second: since the stored information takes the form of new connections, or transmission channels in the nervous system (or the creation of conditions which are functionally equivalent to new connections), it follows that the new stimuli will make use of these new pathways which have been created, automatically activating the appropriate response without requiring any separate process for their recognition or identification.

与关于信息保持方式的这两种立场相对应，关于第三个问题，即所存储的信息以何种方式影响当前活动，也存在两种假说。“编码记忆论者”不得不得出这样的结论：对任何刺激的识别，都涉及把存储内容与传入的感觉模式进行匹配或系统比较，以确定当前刺激是否曾经见过，并确定生物应作出的适当反应。另一方面，经验主义传统的理论家基本上把第三个问题的答案与第二个问题的答案合而为一：既然所存储的信息表现为神经系统中新的连接或传输通道（或者形成在功能上等价于新连接的条件），那么新的刺激就会利用这些已经形成的新通路，自动激活适当的反应，而不需要任何单独的识别或辨认过程。

The theory to be presented here takes the empiricist, or "connectionist" position with regard to these questions. The theory has been developed for a hypothetical nervous system, or machine, called a perceptron. The perceptron is designed to illustrate some of the fundamental properties of intelligent systems in general, without becoming too deeply enmeshed in the special, and frequently unknown, conditions which hold for particular biological organisms. The analogy between the perceptron and biological systems should be readily apparent to the reader.

这里要介绍的理论，在这些问题上采取经验主义的或“联结主义”的立场。这一理论是为一种假想的神经系统或机器而发展的，我们称之为感知机（perceptron）。感知机旨在说明一般智能系统的某些基本性质，而不过深地陷入特定生物体所特有的、往往还不为人知的条件之中。感知机与生物系统之间的类比，读者应当一望即知。

During the last few decades, the development of symbolic logic, digital computers, and switching theory has impressed many theorists with the functional similarity between a neuron and the simple on-off units of which computers are constructed, and has provided the analytical methods necessary for representing highly complex logical functions in terms of such elements. The result has been a profusion of brain models which amount simply to logical contrivances for performing particular algorithms (representing "recall," stimulus comparison, transformation, and various kinds of analysis) in response to sequences of stimuli—e.g., Rashevsky (14), McCulloch (10), McCulloch & Pitts (11), Culbertson (2), Kleene (8), and Minsky (13). A relatively small number of theorists, like Ashby (1) and von Neumann (17, 18), have been concerned with the problems of how an imperfect neural network, containing many random connections, can be made to perform reliably those functions which might be represented by idealized wiring diagrams. Unfortunately, the language of symbolic logic and Boolean algebra is less well suited for such investigations. The need for a suitable language for the mathematical analysis of events in systems where only the gross organization can be characterized, and the precise structure is unknown, has led the author to formulate the current model in terms of probability theory rather than symbolic logic.

在过去几十年间，符号逻辑、数字计算机和开关理论的发展，使许多理论家注意到神经元与构成计算机的简单开关单元在功能上的相似性，也提供了用这类元件表示高度复杂的逻辑函数所需的分析方法。其结果是涌现出大量大脑模型，它们只不过是一些逻辑装置，用来针对刺激序列执行特定的算法（代表“回忆”、刺激比较、变换以及各种分析）——例如 Rashevsky (14)、McCulloch (10)、McCulloch & Pitts (11)、Culbertson (2)、Kleene (8) 和 Minsky (13)。只有为数相对较少的理论家，如 Ashby (1) 和冯·诺伊曼 (17, 18)，关注过这样的问题：一个含有许多随机连接的不完善神经网络，如何才能可靠地执行那些可以用理想化接线图表示的功能。遗憾的是，符号逻辑和布尔代数的语言不太适合这类研究。对于只能刻画其大体组织、而不知其精确结构的系统，需要一种合适的语言来对其中的事件进行数学分析；正是这一需要，促使作者用概率论而不是符号逻辑来表述现在的模型。

The theorists referred to above were chiefly concerned with the question of how such functions as perception and recall might be achieved by a deterministic physical system of any sort, rather than how this is actually done by the brain. The models which have been produced all fail in some important respects (absence of equipotentiality, lack of neuroeconomy, excessive specificity of connections and synchronization requirements, unrealistic specificity of stimuli sufficient for cell firing, postulation of variables or functional features with no known neurological correlates, etc.) to correspond to a biological system. The proponents of this line of approach have maintained that, once it has been shown how a physical system of any variety might be made to perceive and recognize stimuli, or perform other brainlike functions, it would require only a refinement or modification of existing principles to understand the working of a more realistic nervous system, and to eliminate the shortcomings mentioned above. The writer takes the position, on the other hand, that these shortcomings are such that a mere refinement or improvement of the principles already suggested can never account for biological intelligence; a difference in principle is clearly indicated. The theory of statistical separability (Cf. 15), which is to be summarized here, appears to offer a solution in principle to all of these difficulties.

上面提到的理论家主要关心的是：任何一种确定性的物理系统如何能够实现知觉和回忆这类功能，而不是大脑实际上如何实现它们。已经提出的模型在若干重要方面都无法与生物系统相对应（缺乏等势性，缺乏神经经济性，对连接和同步的要求过于特定，足以使细胞放电的刺激特异性不切实际，假设了没有已知神经学对应物的变量或功能特征，等等）。这一路线的支持者主张：一旦表明了任何一种物理系统如何能被做成感知和识别刺激，或执行其他类脑功能，那么只需对现有原理加以完善或修改，就能理解更真实的神经系统如何工作，并消除上述缺陷。作者则持相反的立场：这些缺陷的性质决定了，仅仅完善或改进已经提出的原理，永远无法解释生物智能；显然需要原理上的不同。这里要概述的统计可分性理论（参见 15），似乎为所有这些困难提供了原理上的解决办法。

Those theorists—Hebb (7), Milner (12), Eccles (4), Hayek (6)—who have been more directly concerned with the biological nervous system and its activity in a natural environment, rather than with formally analogous machines, have generally been less exact in their formulations and far from rigorous in their analysis, so that it is frequently hard to assess whether or not the systems that they describe could actually work in a realistic nervous system, and what the necessary and sufficient conditions might be. Here again, the lack of an analytic language comparable in proficiency to the Boolean algebra of the network analysts has been one of the main obstacles. The contributions of this group should perhaps be considered as suggestions of what to look for and investigate, rather than as finished theoretical systems in their own right. Seen from this viewpoint, the most suggestive work, from the standpoint of the following theory, is that of Hebb and Hayek.

另一些理论家——Hebb (7)、Milner (12)、Eccles (4)、Hayek (6)——更直接地关注生物神经系统及其在自然环境中的活动，而不是形式上类似的机器；他们的表述通常不够精确，分析也远不严格，因此往往很难判断他们所描述的系统能否在真实的神经系统中实际运作，以及其充分必要条件是什么。这里，同样缺乏一种在效能上可与网络分析者的布尔代数相比的分析语言，这是主要障碍之一。这一派的贡献或许应当被看作关于该寻找和研究什么的建议，而不是本身已经完成的理论体系。从这个角度看，就下文的理论而言，最有启发性的工作是赫布和哈耶克的工作。

The position, elaborated by Hebb (7), Hayek (6), Uttley (16), and Ashby (1), in particular, upon which the theory of the perceptron is based, can be summarized by the following assumptions:

1. The physical connections of the nervous system which are involved in learning and recognition are not identical from one organism to another. At birth, the construction of the most important networks is largely random, subject to a minimum number of genetic constraints.

2. The original system of connected cells is capable of a certain amount of plasticity; after a period of neural activity, the probability that a stimulus applied to one set of cells will cause a response in some other set is likely to change, due to some relatively long-lasting changes in the neurons themselves.

3. Through exposure to a large sample of stimuli, those which are most "similar" (in some sense which must be defined in terms of the particular physical system) will tend to form pathways to the same sets of responding cells. Those which are markedly "dissimilar" will tend to develop connections to different sets of responding cells.

4. The application of positive and/or negative reinforcement (or stimuli which serve this function) may facilitate or hinder whatever formation of connections is currently in progress.

5. Similarity, in such a system, is represented at some level of the nervous system by a tendency of similar stimuli to activate the same sets of cells. Similarity is not a necessary attribute of particular formal or geometrical classes of stimuli, but depends on the physical organization of the perceiving system, an organization which evolves through interaction with a given environment. The structure of the system, as well as the ecology of the stimulus-environment, will affect, and will largely determine, the classes of "things" into which the perceptual world is divided.

感知机理论所依据的立场，特别是由 Hebb (7)、Hayek (6)、Uttley (16) 和 Ashby (1) 所阐发的立场，可以概括为以下假设：

1. 神经系统中参与学习和识别的物理连接，在不同个体之间并不相同。出生时，最重要的那些网络的构造在很大程度上是随机的，只受到最少量的遗传约束。

2. 最初的相互连接的细胞系统具有一定程度的可塑性；经过一段时间的神经活动后，由于神经元自身发生某些相对持久的变化，施加于一组细胞的刺激在另一组细胞中引起反应的概率很可能发生改变。

3. 在接触大量刺激样本的过程中，那些最“相似”的刺激（其含义须根据特定的物理系统来定义）倾向于形成通向同一组反应细胞的通路；那些明显“不相似”的刺激则倾向于发展出通向不同组反应细胞的连接。

4. 施加正强化和/或负强化（或起这种作用的刺激），可以促进或阻碍当时正在进行的连接形成。

5. 在这样的系统中，相似性在神经系统的某个层次上表现为相似刺激激活同一组细胞的倾向。相似性并不是某些形式类别或几何类别的刺激必然具有的属性，而是取决于感知系统的物理组织，而这种组织是在与给定环境的相互作用中演化出来的。系统的结构以及刺激环境的生态，将影响并在很大程度上决定知觉世界被划分成哪些“事物”类别。

¹ The development of this theory has been carried out at the Cornell Aeronautical Laboratory, Inc., under the sponsorship of the Office of Naval Research, Contract Nonr-2381(00). This article is primarily an adaptation of material reported in Ref. 15, which constitutes the first full report on the program.

¹ 这一理论是在美国海军研究办公室资助下（合同号 Nonr-2381(00)），于康奈尔航空实验室公司完成的。本文主要改编自参考文献 15 中报告的材料，该文献是这一项目的第一份完整报告。

The Organization of a Perceptron

感知机的组织

The organization of a typical photoperceptron (a perceptron responding to optical patterns as stimuli) is shown in Fig. 1. The rules of its organization are as follows:

1. Stimuli impinge on a retina of sensory units (S-points), which are assumed to respond on an all-or-nothing basis, in some models, or with a pulse amplitude or frequency proportional to the stimulus intensity, in other models. In the models considered here, an all-or-nothing response will be assumed.

2. Impulses are transmitted to a set of association cells (A-units) in a "projection area" ($A_{\mathrm{I}}$). This projection area may be omitted in some models, where the retina is connected directly to the association area ($A_{\mathrm{II}}$). The cells in the projection area each receive a number of connections from the sensory points. The set of S-points transmitting impulses to a particular A-unit will be called the origin points of that A-unit. These origin points may be either excitatory or inhibitory in their effect on the A-unit. If the algebraic sum of excitatory and inhibitory impulse intensities is equal to or greater than the threshold ($\theta$) of the A-unit, then the A-unit fires, again on an all-or-nothing basis (or, in some models, which will not be considered here, with a frequency which depends on the net value of the impulses received). The origin points of the A-units in the projection area tend to be clustered or focalized, about some central point, corresponding to each A-unit. The number of origin points falls off exponentially as the retinal distance from the central point for the A-unit in question increases. (Such a distribution seems to be supported by physiological evidence, and serves an important functional purpose in contour detection.)

3. Between the projection area and the association area ($A_{\mathrm{II}}$), connections are assumed to be random. That is, each A-unit in the $A_{\mathrm{II}}$ set receives some number of fibers from origin points in the $A_{\mathrm{I}}$ set, but these origin points are scattered at random throughout the projection area. Apart from their connection distribution, the $A_{\mathrm{II}}$ units are identical with the $A_{\mathrm{I}}$ units, and respond under similar conditions.

4. The "responses," $R_1, R_2, \ldots, R_n$ are cells (or sets of cells) which respond in much the same fashion as the A-units. Each response has a typically large number of origin points located at random in the $A_{\mathrm{II}}$ set. The set of A-units transmitting impulses to a particular response will be called the source-set for that response. (The source-set of a response is identical to its set of origin points in the A-system.) The arrows in Fig. 1 indicate the direction of transmission through the network. Note that up to $A_{\mathrm{II}}$ all connections are forward, and there is no feedback. When we come to the last set of connections, between $A_{\mathrm{II}}$ and the R-units, connections are established in both directions. The rule governing feedback connections, in most models of the perceptron, can be either of the following alternatives:

(a) Each response has excitatory feedback connections to the cells in its own source-set, or

(b) Each response has inhibitory feedback connections to the complement of its own source-set (i.e., it tends to prohibit activity in any association cells which do not transmit to it).

一个典型的光感知机（以光学图案为刺激作出反应的感知机）的组织如图 1 所示。其组织规则如下：

1. 刺激落在由感觉单元（S 点）组成的视网膜上。在某些模型中，假定这些单元以全或无的方式作出反应；在另一些模型中，则以与刺激强度成正比的脉冲幅度或频率作出反应。在这里考虑的模型中，假定为全或无的反应。

2. 冲动被传送到“投射区”（$A_{\mathrm{I}}$）中的一组联想细胞（A 单元）。在某些模型中可以省去这个投射区，让视网膜直接连接到联想区（$A_{\mathrm{II}}$）。投射区中的每个细胞都从感觉点接收若干连接。向某个特定 A 单元传送冲动的那组 S 点，称为该 A 单元的起源点。这些起源点对 A 单元的作用可以是兴奋性的，也可以是抑制性的。如果兴奋性冲动强度与抑制性冲动强度的代数和等于或大于该 A 单元的阈值（$\theta$），该 A 单元就放电，同样是全或无的（在某些这里不予考虑的模型中，则以取决于所接收冲动净值的频率放电）。投射区中 A 单元的起源点倾向于围绕与每个 A 单元相对应的某个中心点聚集或聚焦。随着与该 A 单元中心点的视网膜距离增大，起源点的数目呈指数下降。（这种分布似乎得到了生理学证据的支持，并在轮廓检测中起着重要的功能作用。）

3. 假定投射区与联想区（$A_{\mathrm{II}}$）之间的连接是随机的。也就是说，$A_{\mathrm{II}}$ 集合中的每个 A 单元从 $A_{\mathrm{I}}$ 集合中的起源点接收若干纤维，但这些起源点随机散布在整个投射区。除了连接分布不同外，$A_{\mathrm{II}}$ 单元与 $A_{\mathrm{I}}$ 单元完全相同，并在类似的条件下作出反应。

4. “反应” $R_1, R_2, \ldots, R_n$ 是以与 A 单元大致相同的方式作出反应的细胞（或细胞集合）。每个反应通常有大量随机位于 $A_{\mathrm{II}}$ 集合中的起源点。向某个特定反应传送冲动的那组 A 单元，称为该反应的源集（source-set）。（一个反应的源集与它在 A 系统中的起源点集合是同一个东西。）图 1 中的箭头表示网络中的传输方向。注意，直到 $A_{\mathrm{II}}$ 为止，所有连接都是前向的，没有反馈。到了最后一组连接，即 $A_{\mathrm{II}}$ 与 R 单元之间，连接是双向建立的。在大多数感知机模型中，支配反馈连接的规则可以是以下两种之一：

(a) 每个反应对其自身源集中的细胞具有兴奋性反馈连接；或

(b) 每个反应对其自身源集的补集具有抑制性反馈连接（即它倾向于禁止任何不向它传送的联想细胞的活动）。

Fig. 1. Organization of a perceptron.

图 1. 感知机的组织。

The first of these rules seems more plausible anatomically, since the R-units might be located in the same cortical area as their respective source-sets, making mutual excitation between the R-units and the A-units of the appropriate source-set highly probable. The alternative rule (b) leads to a more readily analyzed system, however, and will therefore be assumed for most of the systems to be evaluated here.

第一条规则在解剖学上似乎更合理，因为 R 单元可能与各自的源集位于同一皮层区域，从而使 R 单元与相应源集中的 A 单元之间极有可能相互兴奋。不过，另一条规则 (b) 得到的系统更容易分析，因此这里要评估的大多数系统都将采用它。

Fig. 2A. Schematic representation of connections in a simple perceptron.

Fig. 2B. Venn diagram of the same perceptron (shading shows active sets for $R_1$ response).

图 2A. 一个简单感知机中连接的示意图。

图 2B. 同一感知机的维恩图（阴影表示 $R_1$ 反应的活动集合）。

Figure 2 shows the organization of a simplified perceptron, which affords a convenient entry into the theory of statistical separability. After the theory has been developed for this simplified model, we will be in a better position to discuss the advantages of the system in Fig. 1. The feedback connections shown in Fig. 2 are inhibitory, and go to the complement of the source-set for the response from which they originate; consequently, this system is organized according to Rule b, above. The system shown here has only three stages, the first association stage having been eliminated. Each A-unit has a set of randomly located origin points in the retina. Such a system will form similarity concepts on the basis of coincident areas of stimuli, rather than by the similarity of contours or outlines. While such a system is at a disadvantage in many discrimination experiments, its capability is still quite impressive, as will be demonstrated presently. The system shown in Fig. 2 has only two responses, but there is clearly no limit on the number that might be included.

图 2 给出一个简化感知机的组织，它为进入统计可分性理论提供了便利的入口。在为这个简化模型建立起理论之后，我们就能更好地讨论图 1 中系统的优点。图 2 所示的反馈连接是抑制性的，通向其起源反应的源集的补集；因此，该系统是按上述规则 b 组织的。这里所示的系统只有三级，第一联想级已被省去。每个 A 单元在视网膜上有一组随机定位的起源点。这样的系统将根据刺激的重合区域而不是轮廓或外形的相似性来形成相似性概念。虽然这样的系统在许多辨别实验中处于不利地位，但它的能力仍然相当可观，下面马上就会说明。图 2 中的系统只有两个反应，但可以包含的反应数目显然没有限制。

The responses in a system organized in this fashion are mutually exclusive. If $R_1$ occurs, it will tend to inhibit $R_2$, and will also inhibit the source-set for $R_2$. Likewise, if $R_2$ should occur, it will tend to inhibit $R_1$. If the total impulse received from all the A-units in one source-set is stronger or more frequent than the impulse received by the alternative (antagonistic) response, then the first response will tend to gain an advantage over the other, and will be the one which occurs. If such a system is to be capable of learning, then it must be possible to modify the A-units or their connections in such a way that stimuli of one class will tend to evoke a stronger impulse in the $R_1$ source-set than in the $R_2$ source-set, while stimuli of another (dissimilar) class will tend to evoke a stronger impulse in the $R_2$ source-set than in the $R_1$ source-set.

按这种方式组织的系统中，各反应是互斥的。如果 $R_1$ 出现，它会倾向于抑制 $R_2$，并且也会抑制 $R_2$ 的源集。同样，如果 $R_2$ 出现，它会倾向于抑制 $R_1$。如果从一个源集中所有 A 单元接收到的总冲动，比另一个（拮抗的）反应所接收到的冲动更强或更频繁，那么前一个反应就会倾向于取得对另一个反应的优势，并成为实际出现的反应。如果这样的系统要能够学习，就必须能够以如下方式修改 A 单元或其连接：使一类刺激倾向于在 $R_1$ 的源集中引起比在 $R_2$ 的源集中更强的冲动，而另一类（不相似的）刺激倾向于在 $R_2$ 的源集中引起比在 $R_1$ 的源集中更强的冲动。

It will be assumed that the impulses delivered by each A-unit can be characterized by a value, $V$, which may be an amplitude, frequency, latency, or probability of completing transmission. If an A-unit has a high value, then all of its output impulses are considered to be more effective, more potent, or more likely to arrive at their endbulbs than impulses from an A-unit with a lower value. The value of an A-unit is considered to be a fairly stable characteristic, probably depending on the metabolic condition of the cell and the cell membrane, but it is not absolutely constant. It is assumed that, in general, periods of activity tend to increase a cell's value, while the value may decay (in some models) with inactivity. The most interesting models are those in which cells are assumed to compete for metabolic materials, the more active cells gaining at the expense of the less active cells. In such a system, if there is no activity, all cells will tend to remain in a relatively constant condition, and (regardless of activity) the net value of the system, taken in its entirety, will remain constant at all times. Three types of systems, which differ in their value dynamics, have been investigated quantitatively. Their principal logical features are compared in Table 1. In the alpha system, an active cell simply gains an increment of value for every impulse, and holds this gain indefinitely. In the beta system, each source-set is allowed a certain constant rate of gain, the increments being apportioned among the cells of the source-set in proportion to their activity. In the gamma system, active cells gain in value at the expense of the inactive cells of their source-set, so that the total value of a source-set is always constant.

我们假定，每个 A 单元发出的冲动可以用一个值（value）$V$ 来刻画，它可以是幅度、频率、潜伏期或完成传输的概率。如果一个 A 单元的值高，那么它的所有输出冲动都被认为比来自低值 A 单元的冲动更有效、更有力，或更有可能到达其末梢球。A 单元的值被认为是一个相当稳定的特性，大概取决于细胞及细胞膜的代谢状况，但并非绝对不变。一般假定，活动期倾向于提高细胞的值，而（在某些模型中）值会随不活动而衰减。最有意思的模型是这样一些模型：假定细胞彼此竞争代谢物质，较活跃的细胞以较不活跃的细胞为代价而获益。在这样的系统中，如果没有活动，所有细胞都会倾向于保持相对恒定的状态，并且（不论有无活动）整个系统的净值在任何时候都保持不变。我们定量研究了三类在值动力学上不同的系统，表 1 比较了它们的主要逻辑特征。在 alpha 系统中，活动细胞每发出一个冲动就获得一个值增量，并无限期地保持这一增益。在 beta 系统中，每个源集被允许以某个恒定速率获益，增量按活动程度分配给源集中的各细胞。在 gamma 系统中，活动细胞以其源集中不活动细胞为代价获得值，因此一个源集的总值始终不变。

	α-System (Uncompensated Gain System)	β-System (Constant Feed System)	γ-System (Parasitic Gain System)
Total value-gain of source set per reinforcement	$N_{a_r}$	$K$	0
ΔV for A-units active for 1 unit of time	+1	$K/N_{a_r}$	+1
ΔV for inactive A-units outside of dominant set	0	$K/N_{A_r}$	0
ΔV for inactive A-units of dominant set	0	0	$\dfrac{-N_{a_r}}{N_{A_r}-N_{a_r}}$
Mean value of A-system	Increases with number of reinforcements	Increases with time	Constant
Difference between mean values of source-sets	Proportional to differences of reinforcement frequency $(n_{s_{r_1}} - n_{s_{r_2}})$	0	0

Table 1. Comparison of Logical Characteristics of α, β, and γ Systems

Note: In the β and γ systems, the total value-change for any A-unit will be the sum of the ΔV's for all source-sets of which it is a member.

$N_{a_r}$ = Number of active units in source-set; $N_{A_r}$ = Total number of units in source-set; $n_{s_{r_j}}$ = Number of stimuli associated to response $r_j$; $K$ = Arbitrary constant

	α 系统（无补偿增益系统）	β 系统（恒定供给系统）	γ 系统（寄生增益系统）
每次强化中源集的总值增益	$N_{a_r}$	$K$	0
活动 1 个时间单位的 A 单元的 ΔV	+1	$K/N_{a_r}$	+1
优势集以外不活动 A 单元的 ΔV	0	$K/N_{A_r}$	0
优势集内不活动 A 单元的 ΔV	0	0	$\dfrac{-N_{a_r}}{N_{A_r}-N_{a_r}}$
A 系统的平均值	随强化次数增加	随时间增加	恒定
各源集平均值之差	与强化频率之差 $(n_{s_{r_1}} - n_{s_{r_2}})$ 成正比	0	0

表 1. α、β、γ 三类系统逻辑特征的比较

注：在 β 和 γ 系统中，任一 A 单元的总值变化，是它所属的所有源集的 ΔV 之和。

$N_{a_r}$ = 源集中活动单元的数目；$N_{A_r}$ = 源集中单元的总数；$n_{s_{r_j}}$ = 与反应 $r_j$ 相联结的刺激数目；$K$ = 任意常数

For purposes of analysis, it is convenient to distinguish two phases in the response of the system to a stimulus (Fig. 3). In the predominant phase, some proportion of A-units (represented by solid dots in the figure) responds to the stimulus, but the R-units are still inactive. This phase is transient, and quickly gives way to the postdominant phase, in which one of the responses becomes active, inhibiting activity in the complement of its own source-set, and thus preventing the occurrence of any alternative response. The response which happens to become dominant is initially random, but if the A-units are reinforced (i.e., if the active units are allowed to gain in value), then when the same stimulus is presented again at a later time, the same response will have a stronger tendency to recur, and learning can be said to have taken place.

为了分析方便，可以把系统对刺激的反应区分为两个阶段（图 3）。在优势前阶段，一部分 A 单元（图中以实心圆点表示）对刺激作出反应，但 R 单元仍不活动。这一阶段是短暂的，很快就让位于优势后阶段：其中一个反应变得活跃，抑制其自身源集补集中的活动，从而阻止任何其他反应出现。碰巧成为优势的反应起初是随机的；但如果 A 单元得到强化（即允许活动单元获得值），那么当同一刺激在以后再次呈现时，同一反应会有更强的再现倾向，这时就可以说发生了学习。

Fig. 3A. Predominant phase. Inhibitory connections are not shown. Solid black units are active.

Fig. 3B. Postdominant phase. Dominant subset suppresses rival sets. Inhibitory connections shown only for $R_1$.

Fig. 3. Phases of response to a stimulus.

图 3A. 优势前阶段。未画出抑制性连接。实心黑色单元为活动单元。

图 3B. 优势后阶段。优势子集压制竞争集合。只画出了 $R_1$ 的抑制性连接。

图 3. 对刺激作出反应的各阶段。

Analysis of the Predominant Phase

优势前阶段的分析

The perceptrons considered here will always assume a fixed threshold, $\theta$, for the activation of the A-units. Such a system will be called a fixed-threshold model, in contrast to a continuous transducer model, where the response of the A-unit is some continuous function of the impinging stimulus energy.

这里考虑的感知机总是假定 A 单元的激活阈值 $\theta$ 是固定的。这样的系统称为固定阈值模型，与之相对的是连续换能器模型，后者中 A 单元的反应是所受刺激能量的某个连续函数。

In order to predict the learning curves of a fixed-threshold perceptron, two variables have been found to be of primary importance. They are defined as follows:

$P_a$ = the expected proportion of A-units activated by a stimulus of a given size,

$P_c$ = the conditional probability that an A-unit which responds to a given stimulus, $S_1$, will also respond to another given stimulus, $S_2$.

It can be shown (Rosenblatt, 15) that as the size of the retina is increased, the number of S-points ($N_s$) quickly ceases to be a significant parameter, and the values of $P_a$ and $P_c$ approach the value that they would have for a retina with infinitely many points. For a large retina, therefore, the equations are as follows:

$$P_a = \sum_{e=\theta}^{x} \sum_{i=\theta}^{\min(y,\, e-\theta)} P(e,i) \tag{1}$$

where

$$P(e,i) = \binom{x}{e} R^e (1-R)^{x-e} \times \binom{y}{i} R^i (1-R)^{y-i}$$

and

$R$ = proportion of S-points activated by the stimulus

$x$ = number of excitatory connections to each A-unit

$y$ = number of inhibitory connections to each A-unit

$\theta$ = threshold of A-units.

(The quantities $e$ and $i$ are the excitatory and inhibitory components of the excitation received by the A-unit from the stimulus. If the algebraic sum $\alpha = e + i$ is equal to or greater than $\theta$, the A-unit is assumed to respond.)

为了预测固定阈值感知机的学习曲线，我们发现有两个变量最为重要。它们的定义如下：

$P_a$ = 给定大小的刺激所激活的 A 单元的期望比例；

$P_c$ = 对给定刺激 $S_1$ 作出反应的 A 单元，也对另一给定刺激 $S_2$ 作出反应的条件概率。

可以证明（Rosenblatt, 15），随着视网膜尺寸的增大，S 点的数目（$N_s$）很快就不再是一个重要参数，$P_a$ 和 $P_c$ 的值趋近于视网膜具有无穷多个点时的值。因此，对于大视网膜，方程如下：

$$P_a = \sum_{e=\theta}^{x} \sum_{i=\theta}^{\min(y,\, e-\theta)} P(e,i) \tag{1}$$

其中

$$P(e,i) = \binom{x}{e} R^e (1-R)^{x-e} \times \binom{y}{i} R^i (1-R)^{y-i}$$

并且

$R$ = 被刺激激活的 S 点的比例

$x$ = 每个 A 单元的兴奋性连接数

$y$ = 每个 A 单元的抑制性连接数

$\theta$ = A 单元的阈值。

（$e$ 和 $i$ 分别是 A 单元从刺激所接收的兴奋中的兴奋性成分和抑制性成分。如果代数和 $\alpha = e + i$ 等于或大于 $\theta$，就假定该 A 单元作出反应。）（译注：(1) 式内层求和的下限在原刊中印作 $i=\theta$，按原样保留。）

$$P_c = \frac{1}{P_a} \sum_{e=\theta}^{x} \sum_{i=e-\theta}^{y} \sum_{l_e=0}^{e} \sum_{l_i=0}^{i} \sum_{g_e=0}^{x-e} \sum_{g_i=0}^{y-i} P(e,i,l_e,l_i,g_e,g_i) \tag{2}$$

$$(e - i - l_e + l_i + g_e - g_i \ge \theta)$$

where

$$\begin{aligned} P(e,i,l_e,l_i,g_e,g_i) &= \binom{x}{e} R^e (1-R)^{x-e} \times \binom{y}{i} R^i (1-R)^{y-i} \\ &\quad \times \binom{e}{l_e} L^{l_e} (1-L)^{e-l_e} \times \binom{i}{l_i} L^{l_i} (1-L)^{i-l_i} \\ &\quad \times \binom{x-e}{g_e} G^{g_e} (1-G)^{x-e-g_e} \times \binom{y-i}{g_i} G^{g_i} (1-G)^{y-i-g_i} \end{aligned}$$

and

$L$ = proportion of the S-points illuminated by the first stimulus, $S_1$, which are not illuminated by $S_2$

$G$ = proportion of the residual S-set (left over from the first stimulus) which is included in the second stimulus ($S_2$).

The quantities $R$, $L$, and $G$ specify the two stimuli and their retinal overlap. $l_e$ and $l_i$ are, respectively, the numbers of excitatory and inhibitory origin points "lost" by the A-unit when stimulus $S_1$ is replaced by $S_2$; $g_e$ and $g_i$ are the numbers of excitatory and inhibitory origin points "gained" when stimulus $S_1$ is replaced by $S_2$. The summations in Equation 2 are between the limits indicated, subject to the side condition $e - i - l_e + l_i + g_e - g_i \ge \theta$.

$$P_c = \frac{1}{P_a} \sum_{e=\theta}^{x} \sum_{i=e-\theta}^{y} \sum_{l_e=0}^{e} \sum_{l_i=0}^{i} \sum_{g_e=0}^{x-e} \sum_{g_i=0}^{y-i} P(e,i,l_e,l_i,g_e,g_i) \tag{2}$$

$$(e - i - l_e + l_i + g_e - g_i \ge \theta)$$

其中

$$\begin{aligned} P(e,i,l_e,l_i,g_e,g_i) &= \binom{x}{e} R^e (1-R)^{x-e} \times \binom{y}{i} R^i (1-R)^{y-i} \\ &\quad \times \binom{e}{l_e} L^{l_e} (1-L)^{e-l_e} \times \binom{i}{l_i} L^{l_i} (1-L)^{i-l_i} \\ &\quad \times \binom{x-e}{g_e} G^{g_e} (1-G)^{x-e-g_e} \times \binom{y-i}{g_i} G^{g_i} (1-G)^{y-i-g_i} \end{aligned}$$

并且

$L$ = 被第一个刺激 $S_1$ 照亮、但未被 $S_2$ 照亮的 S 点所占的比例

$G$ = （第一个刺激之外）剩余 S 点集合中被包含在第二个刺激（$S_2$）中的比例。

$R$、$L$ 和 $G$ 这三个量确定了两个刺激及其在视网膜上的重叠。$l_e$ 和 $l_i$ 分别是刺激 $S_1$ 被 $S_2$ 取代时，A 单元“失去”的兴奋性和抑制性起源点数目；$g_e$ 和 $g_i$ 分别是刺激 $S_1$ 被 $S_2$ 取代时“获得”的兴奋性和抑制性起源点数目。方程 2 中的求和在所示上下限之间进行，并服从附加条件 $e - i - l_e + l_i + g_e - g_i \ge \theta$。（译注：(2) 式第二个求和的下限原刊印作 $i=e-\theta$，按原样保留；原式分两行排印，此处合并为一行。）

Some of the most important characteristics of $P_a$ are illustrated in Fig. 4, which shows $P_a$ as a function of the retinal area illuminated ($R$). Note that $P_a$ can be reduced in magnitude by either increasing the threshold, $\theta$, or by increasing the proportion of inhibitory connections ($y$). A comparison of Fig. 4b and 4c shows that if the excitation is about equal to the inhibition, the curves for $P_a$ as a function of $R$ are flattened out, so that there is little variation in $P_a$ for stimuli of different sizes. This fact is of great importance for systems which require $P_a$ to be close to an optimum value in order to perform properly.

$P_a$ 的一些最重要特性见图 4，图中给出 $P_a$ 作为被照亮视网膜面积（$R$）的函数。注意，提高阈值 $\theta$ 或提高抑制性连接的比例（$y$），都可以减小 $P_a$ 的数值。比较图 4b 和 4c 可见，如果兴奋与抑制大致相等，$P_a$ 关于 $R$ 的曲线就会变平，因而对于不同大小的刺激，$P_a$ 变化很小。对于需要 $P_a$ 接近某个最佳值才能正常工作的系统，这一事实非常重要。

The behavior of $P_c$ is illustrated in Fig. 5 and 6. The curves in Fig. 5 can be compared with those for $P_a$ in Fig. 4. Note that as the threshold is increased, there is an even sharper reduction in the value of $P_c$ than was the case with $P_a$. $P_c$ also decreases as the proportion of inhibitory connections increases, as does $P_a$. Fig. 5, which is calculated for nonoverlapping stimuli, illustrates the fact that $P_c$ remains greater than zero even when the stimuli are completely disjunct, and illuminate no retinal points in common. In Fig. 6, the effect of varying amounts of overlap between the stimuli is shown. In all cases, the value of $P_c$ goes to unity as the stimuli approach perfect identity. For smaller stimuli (broken line curves), the value of $P_c$ is lower than for large stimuli. Similarly, the value is less for high thresholds than for low thresholds. The minimum value of $P_c$ will be equal to

$$P_{c\,\min} = (1-L)^x (1-G)^y. \tag{3}$$

In Fig. 6, $P_{c\,\min}$ corresponds to the curve for $\theta = 10$. Note that under these conditions the probability that the A-unit responds to both stimuli ($P_c$) is practically zero, except for stimuli which are quite close to identity. This condition can be of considerable help in discrimination learning.

$P_c$ 的行为见图 5 和图 6。图 5 中的曲线可以与图 4 中 $P_a$ 的曲线相比较。注意，随着阈值提高，$P_c$ 值的下降比 $P_a$ 更加剧烈。与 $P_a$ 一样，$P_c$ 也随抑制性连接比例的增加而减小。图 5 是针对不重叠刺激计算的，它表明即使两个刺激完全不相交、没有共同照亮任何视网膜点，$P_c$ 仍然大于零。图 6 给出了刺激间不同重叠程度的影响。在所有情形下，当两个刺激趋于完全相同时，$P_c$ 的值都趋于 1。对于较小的刺激（虚线曲线），$P_c$ 的值低于大刺激。同样，阈值高时该值比阈值低时小。$P_c$ 的最小值等于

$$P_{c\,\min} = (1-L)^x (1-G)^y. \tag{3}$$

在图 6 中，$P_{c\,\min}$ 对应于 $\theta = 10$ 的曲线。注意，在这些条件下，除非两个刺激非常接近相同，A 单元同时对两个刺激作出反应的概率（$P_c$）实际上为零。这一条件对辨别学习可能大有帮助。

Fig. 4. $P_a$ as function of retinal area illuminated. [Panels: (a) effect of inhibitory-excitatory mixture, $\theta = 1$; (b) variation with $\theta$ for $X = 10$, $Y = 0$; (c) variation with M and $\theta$ for mixtures about 50% inhibitory; solid lines are for $X = 5$, $Y = 5$.]

图 4. $P_a$ 作为被照亮视网膜面积的函数。［分图：(a) 抑制—兴奋混合比例的影响，$\theta = 1$；(b) 当 $X = 10$、$Y = 0$ 时随 $\theta$ 的变化；(c) 对约 50% 为抑制性的混合，随 M 和 $\theta$ 的变化；实线对应 $X = 5$、$Y = 5$。（M 按原图保留。）］

Fig. 5. $P_c$ as a function of $R$, for nonoverlapping stimuli.

Fig. 6. $P_c$ as a function of $C$. $X = 10$, $Y = 0$. Solid lines: $R = .5$; broken lines: $R = .2$.

图 5. 不重叠刺激下，$P_c$ 作为 $R$ 的函数。

图 6. $P_c$ 作为 $C$ 的函数。$X = 10$，$Y = 0$。实线：$R = .5$；虚线：$R = .2$。

Mathematical Analysis of Learning in the Perceptron

感知机学习的数学分析

The response of the perceptron in the predominant phase, where some fraction of the A-units (scattered throughout the system) responds to the stimulus, quickly gives way to the postdominant response, in which activity is limited to a single source-set, the other sets being suppressed. Two possible systems have been studied for the determination of the "dominant" response, in the postdominant phase. In one (the mean-discriminating system, or $\mu$-system), the response whose inputs have the greatest mean value responds first, gaining a slight advantage over the others, so that it quickly becomes dominant. In the second case (the sum-discriminating system, or $\Sigma$-system), the response whose inputs have the greatest net value gains an advantage. In most cases, systems which respond to mean values have an advantage over systems which respond to sums, since the means are less influenced by random variations in $P_a$ from one source-set to another. In the case of the $\gamma$-system (see Table 1), however, the performance of the $\mu$-system and $\Sigma$-system become identical.

在优势前阶段，有一部分（散布于整个系统中的）A 单元对刺激作出反应；感知机的这种反应很快让位于优势后反应，此时活动局限于单个源集，其他集合受到压制。为了确定优势后阶段的“优势”反应，我们研究了两种可能的系统。在一种系统中（均值辨别系统，或 $\mu$ 系统），输入平均值最大的反应最先作出反应，取得相对于其他反应的微弱优势，从而很快成为优势反应。在第二种情形中（总和辨别系统，或 $\Sigma$ 系统），输入净值最大的反应取得优势。在大多数情形下，对均值作出反应的系统优于对总和作出反应的系统，因为均值受各源集之间 $P_a$ 随机波动的影响较小。不过，在 $\gamma$ 系统（见表 1）的情形下，$\mu$ 系统和 $\Sigma$ 系统的表现变得完全相同。

We have indicated that the perceptron is expected to learn, or to form associations, as a result of the changes in value that occur as a result of the activity of the association cells. In evaluating this learning, one of two types of hypothetical experiments can be considered. In the first case, the perceptron is exposed to some series of stimulus patterns (which might be presented in random positions on the retina) and is "forced" to give the desired response in each case. (This forcing of responses is assumed to be a prerogative of the experimenter. In experiments intended to evaluate trial-and-error learning, with more sophisticated perceptrons, the experimenter does not force the system to respond in the desired fashion, but merely applies positive reinforcement when the response happens to be correct, and negative reinforcement when the response is wrong.) In evaluating the learning which has taken place during this "learning series," the perceptron is assumed to be "frozen" in its current condition, no further value changes being allowed, and the same series of stimuli is presented again in precisely the same fashion, so that the stimuli fall on identical positions on the retina. The probability that the perceptron will show a bias towards the "correct" response (the one which has been previously reinforced during the learning series) in preference to any given alternative response is called $P_r$, the probability of correct choice of response between two alternatives.

我们已经指出，由于联想细胞的活动引起值的变化，可以预期感知机会学习，即形成联想。在评估这种学习时，可以考虑两类假想实验中的一种。在第一类实验中，让感知机接触一系列刺激模式（它们可以呈现在视网膜上的随机位置），并在每种情况下“强制”它给出所需的反应。（这种对反应的强制被假定为实验者的特权。在用更复杂的感知机评估试错学习的实验中，实验者并不强制系统以所需方式作出反应，而只是在反应恰好正确时施加正强化，在反应错误时施加负强化。）在评估这一“学习序列”中所发生的学习时，假定感知机被“冻结”在当前状态，不再允许值发生变化，然后以完全相同的方式再次呈现同一系列刺激，使刺激落在视网膜上完全相同的位置。感知机表现出偏向“正确”反应（即在学习序列中曾被强化的反应）、而非任一给定其他反应的概率，记为 $P_r$，即在两个备选反应之间正确选择反应的概率。

In the second type of experiment, a learning series is presented exactly as before, but instead of evaluating the perceptron's performance using the same series of stimuli which were shown before, a new series is presented, in which stimuli may be drawn from the same classes that were previously experienced, but are not necessarily identical. This new test series is assumed to be composed of stimuli projected onto random retinal positions, which are chosen independently of the positions selected for the learning series. The stimuli of the test series may also differ in size or rotational position from the stimuli which were previously experienced. In this case, we are interested in the probability that the perceptron will give the correct response for the class of stimuli which is represented, regardless of whether the particular stimulus has been seen before or not. This probability is called $P_g$, the probability of correct generalization. As with $P_r$, $P_g$ is actually the probability that a bias will be found in favor of the proper response rather than any one alternative; only one pair of responses at a time is considered, and the fact that the response bias is correct in one pair does not mean that there may not be other pairs in which the bias favors the wrong response. The probability that the correct response will be preferred over all alternatives is designated $P_R$ or $P_G$.

在第二类实验中，学习序列的呈现与前面完全一样；但评估感知机的表现时，不再使用先前呈现过的同一系列刺激，而是呈现一个新的系列，其中的刺激可以取自先前经历过的同一些类别，但不一定完全相同。假定这一新的测试系列由投射到随机视网膜位置上的刺激组成，这些位置的选取与学习序列所选位置相互独立。测试系列中的刺激在大小或旋转位置上也可以与先前经历过的刺激不同。在这种情形下，我们关心的是：无论某个特定刺激以前是否见过，感知机对所代表的那类刺激给出正确反应的概率。这一概率记为 $P_g$，即正确泛化的概率。与 $P_r$ 一样，$P_g$ 实际上是发现偏向正确反应、而非任一个其他反应的偏差的概率；每次只考虑一对反应，在一对反应中偏差正确，并不意味着不存在偏差偏向错误反应的其他反应对。正确反应优于所有其他反应的概率记为 $P_R$ 或 $P_G$。

In all cases investigated, a single general equation gives a close approximation to $P_r$ and $P_g$, if the appropriate constants are substituted. This equation is of the form:

$$P = P(N_{a_r} > 0) \cdot \phi(Z) \tag{4}$$

where

$$P(N_{a_r} > 0) = 1 - (1 - P_a)^{N_e}$$

$\phi(Z)$ = normal curve integral from $-\infty$ to $Z$

and

$$Z = \frac{c_1 n_{s_r} + c_2}{\sqrt{c_3 n_{s_r}^{\,2} + c_4 n_{s_r}}}.$$

If $R_1$ is the "correct" response, and $R_2$ is the alternative response under consideration, Equation 4 is the probability that $R_1$ will be preferred over $R_2$ after $n_{s_r}$ stimuli have been shown for each of the two responses, during the learning period. $N_e$ is the number of "effective" A-units in each source-set; that is, the number of A-units in either source-set which are not connected in common to both responses. Those units which are connected in common contribute equally to both sides of the value balance, and consequently do not affect the net bias towards one response or the other. $N_{a_r}$ is the number of active units in a source-set, which respond to the test stimulus, $S_t$. $P(N_{a_r} > 0)$ is the probability that at least one of the $N_e$ effective units in the source-set of the correct response (designated, by convention, as the $R_1$ response) will be activated by the test stimulus, $S_t$.

在所研究的一切情形中，只要代入适当的常数，一个通用方程就能很好地近似 $P_r$ 和 $P_g$。该方程的形式为：

$$P = P(N_{a_r} > 0) \cdot \phi(Z) \tag{4}$$

其中

$$P(N_{a_r} > 0) = 1 - (1 - P_a)^{N_e}$$

$\phi(Z)$ = 从 $-\infty$ 到 $Z$ 的正态曲线积分

并且

$$Z = \frac{c_1 n_{s_r} + c_2}{\sqrt{c_3 n_{s_r}^{\,2} + c_4 n_{s_r}}}.$$

若 $R_1$ 是“正确”反应，$R_2$ 是所考虑的另一反应，那么方程 4 就是：在学习期间为两个反应各呈现 $n_{s_r}$ 个刺激之后，$R_1$ 优于 $R_2$ 的概率。$N_e$ 是每个源集中“有效”A 单元的数目，即任一源集中不同时连接到两个反应的 A 单元的数目。那些共同连接的单元对值天平的两边贡献相等，因此不影响偏向某一反应的净偏差。$N_{a_r}$ 是一个源集中对测试刺激 $S_t$ 作出反应的活动单元数。$P(N_{a_r} > 0)$ 是正确反应（按约定记为 $R_1$ 反应）的源集中 $N_e$ 个有效单元里至少有一个被测试刺激 $S_t$ 激活的概率。

In the case of $P_g$, the constant $c_2$ is always equal to zero, the other three constants being the same as for $P_r$. The values of the four constants depend on the parameters of the physical nerve net (the perceptron) and also on the organization of the stimulus environment.

在 $P_g$ 的情形下，常数 $c_2$ 总是等于零，其他三个常数与 $P_r$ 的相同。这四个常数的值取决于物理神经网络（感知机）的参数，也取决于刺激环境的组织。

The simplest cases to analyze are those in which the perceptron is shown stimuli drawn from an "ideal environment," consisting of randomly placed points of illumination, where there is no attempt to classify stimuli according to intrinsic similarity. Thus, in a typical learning experiment, we might show the perceptron 1,000 stimuli made up of random collections of illuminated retinal points, and we might arbitrarily reinforce $R_1$ as the "correct" response for the first 500 of these, and $R_2$ for the remaining 500. This environment is "ideal" only in the sense that we speak of an ideal gas in physics; it is a convenient artifact for purposes of analysis, and does not lead to the best performance from the perceptron. In the ideal environment situation, the constant $c_1$ is always equal to zero, so that, in the case of $P_g$ (where $c_2$ is also zero), the value of $Z$ will be zero, and $P_g$ can never be any better than the random expectation of 0.5. The evaluation of $P_r$ for these conditions, however, throws some interesting light on the differences between the alpha, beta, and gamma systems (Table 1).

最容易分析的情形是：给感知机呈现的刺激取自一个“理想环境”，它由随机放置的光点组成，不试图按内在相似性对刺激分类。例如，在一个典型的学习实验中，我们可以给感知机呈现 1,000 个由随机的被照亮视网膜点集合构成的刺激，并任意地对其中前 500 个把 $R_1$ 作为“正确”反应来强化，对其余 500 个强化 $R_2$。这个环境之所以称为“理想的”，只是在物理学中说理想气体那种意义上；它是便于分析的人为设定，并不能使感知机达到最佳表现。在理想环境的情形下，常数 $c_1$ 总是等于零，因此对于 $P_g$（其中 $c_2$ 也为零），$Z$ 的值为零，$P_g$ 永远不可能好于随机期望值 0.5。不过，在这些条件下对 $P_r$ 的评估，为 alpha、beta 和 gamma 系统（表 1）之间的差异提供了一些有趣的启示。

First consider the alpha system, which has the simplest dynamics of the three. In this system, whenever an A-unit is active for one unit of time, it gains one unit of value. We will assume an experiment, initially, in which $n_{s_r}$ (the number of stimuli associated to each response) is constant for all responses. In this case, for the sum system,

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= (1-P_a)N_e \\ c_3 &= 2P_a\omega \\ c_4 &\approx 0 \end{aligned}\right\} \tag{5}$$

where $\omega$ = the fraction of responses connected to each A-unit. If the source-sets are disjunct, $\omega = 1/N_R$, where $N_R$ is the number of responses in the system. For the $\mu$-system,

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= (1-P_a)N_e \\ c_3 &= 0 \\ c_4 &= 2\omega \end{aligned}\right\} \tag{6}$$

The reduction of $c_3$ to zero gives the $\mu$-system a definite advantage over the $\Sigma$-system. Typical learning curves for these systems are compared in Fig. 7 and 8. Figure 9 shows the effect of variations in $P_a$ upon the performance of the system.

先考虑 alpha 系统，它是三者中动力学最简单的。在这一系统中，A 单元每活动一个时间单位，就获得一个单位的值。我们首先假定这样一个实验：其中 $n_{s_r}$（与每个反应相联结的刺激数）对所有反应都是常数。在这种情况下，对于总和系统，

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= (1-P_a)N_e \\ c_3 &= 2P_a\omega \\ c_4 &\approx 0 \end{aligned}\right\} \tag{5}$$

其中 $\omega$ = 连接到每个 A 单元的反应所占的比例。如果各源集互不相交，则 $\omega = 1/N_R$，其中 $N_R$ 是系统中反应的数目。对于 $\mu$ 系统，

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= (1-P_a)N_e \\ c_3 &= 0 \\ c_4 &= 2\omega \end{aligned}\right\} \tag{6}$$

$c_3$ 降为零，使 $\mu$ 系统相对于 $\Sigma$ 系统具有明确的优势。图 7 和图 8 比较了这些系统的典型学习曲线。图 9 给出了 $P_a$ 的变化对系统表现的影响。

Fig. 7. $P_{r(\Sigma)}$ as function of $\omega n_{s_r}$, for discrete subsets. ($\omega_c = 0$, $P_a = .005$. Ideal environment assumed.)

图 7. 离散子集情形下，$P_{r(\Sigma)}$ 作为 $\omega n_{s_r}$ 的函数。（$\omega_c = 0$，$P_a = .005$。假定为理想环境。）

If $n_{s_r}$, instead of being fixed, is treated as a random variable, so that the number of stimuli associated to each response is drawn separately from some distribution, then the performance of the $\alpha$-system is considerably poorer than the above equations indicate. Under these conditions, the constants for the $\mu$-system are

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= 1 - P_a \\ c_3 &= 2P_a^{\,2} q^2 \left[\frac{(\omega N_R - 1)^2}{N_R - 2} + 1\right] \\ c_4 &= \frac{2(1-P_a)N_R}{(1-\omega_c)N_A} \end{aligned}\right\} \tag{7}$$

where

$q$ = ratio of $\sigma_{n_{s_r}}$ to $\bar{n}_{s_r}$

$N_R$ = number of responses in the system

$N_A$ = number of A-units in the system

$\omega_c$ = proportion of A-units common to $R_1$ and $R_2$.

For this equation (and any others in which $n_{s_r}$ is treated as a random variable), it is necessary to define $n_{s_r}$ in Equation 4 as the expected value of this variable, over the set of all responses.

如果 $n_{s_r}$ 不是固定的，而是被当作随机变量，即与每个反应相联结的刺激数分别从某个分布中抽取，那么 $\alpha$ 系统的表现会比上述方程所示的差得多。在这些条件下，$\mu$ 系统的常数为

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= 1 - P_a \\ c_3 &= 2P_a^{\,2} q^2 \left[\frac{(\omega N_R - 1)^2}{N_R - 2} + 1\right] \\ c_4 &= \frac{2(1-P_a)N_R}{(1-\omega_c)N_A} \end{aligned}\right\} \tag{7}$$

其中

$q$ = $\sigma_{n_{s_r}}$ 与 $\bar{n}_{s_r}$ 之比

$N_R$ = 系统中反应的数目

$N_A$ = 系统中 A 单元的数目

$\omega_c$ = $R_1$ 与 $R_2$ 共有的 A 单元所占比例。

对于这个方程（以及任何把 $n_{s_r}$ 当作随机变量的方程），必须把方程 4 中的 $n_{s_r}$ 定义为该变量在所有反应上的期望值。

Fig. 8. $P_{r(\mu)}$ as function of $\omega n_{s_r}$. (For $P_a = .07$, $\omega_c = 0$. Ideal environment assumed.)

图 8. $P_{r(\mu)}$ 作为 $\omega n_{s_r}$ 的函数。（$P_a = .07$，$\omega_c = 0$。假定为理想环境。）

For the $\beta$-system, there is an even greater deficit in performance, due to the fact that the net value continues to grow regardless of what happens to the system. The large net values of the subsets activated by a stimulus tend to amplify small statistical differences, causing an unreliable performance. The constants in this case (again for the $\mu$-system) are

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= (1-P_a)N_e \\ c_3 &= 2(P_a N_e q \omega N_R^{\,2})^2 \\ c_4 &= 2(1-P_a)\omega N_R N_e \end{aligned}\right\} \tag{8}$$

In both the alpha and beta systems, performance will be poorer for the sum-discriminating model than for the mean-discriminating case. In the gamma-system, however, it can be shown that $P_{r(\Sigma)} = P_{r(\mu)}$; i.e., it makes no difference in performance whether the $\Sigma$-system or $\mu$-system is used. Moreover, the constants for the $\gamma$-system, with variable $n_{s_r}$, are identical to the constants for the alpha $\mu$-system, with $n_{s_r}$ fixed (Equation 6). The performance of the three systems is compared in Fig. 10, which clearly demonstrates the advantage of the $\gamma$-system.

对于 $\beta$ 系统，表现的亏损更大，因为无论系统发生什么，净值都会持续增长。被刺激激活的子集具有很大的净值，这倾向于放大微小的统计差异，造成不可靠的表现。这种情形下的常数（同样针对 $\mu$ 系统）为

$$\left.\begin{aligned} c_1 &= 0 \\ c_2 &= (1-P_a)N_e \\ c_3 &= 2(P_a N_e q \omega N_R^{\,2})^2 \\ c_4 &= 2(1-P_a)\omega N_R N_e \end{aligned}\right\} \tag{8}$$

在 alpha 和 beta 系统中，总和辨别模型的表现都会比均值辨别的情形差。然而在 gamma 系统中，可以证明 $P_{r(\Sigma)} = P_{r(\mu)}$；也就是说，使用 $\Sigma$ 系统还是 $\mu$ 系统，对表现没有影响。此外，$n_{s_r}$ 可变时 $\gamma$ 系统的常数，与 $n_{s_r}$ 固定时 alpha $\mu$ 系统的常数（方程 6）完全相同。图 10 比较了三种系统的表现，清楚地显示了 $\gamma$ 系统的优势。

Fig. 9. $P_{r(\mu)}$ as function of $P_a$. (For $n_{s_r} = 1{,}000$, $\omega_c = 0$. Ideal environment assumed.)

Fig. 10. Comparison of $\alpha$, $\beta$, and $\gamma$ systems, for variable $n_{s_r}$ ($N_R = 100$, $\sigma_{n_{s_r}} = .5\bar{n}_{s_r}$, $N_A = 10{,}000$, $P_a = .07$, $\omega = .2$).

图 9. $P_{r(\mu)}$ 作为 $P_a$ 的函数。（$n_{s_r} = 1{,}000$，$\omega_c = 0$。假定为理想环境。）

图 10. $n_{s_r}$ 可变时 $\alpha$、$\beta$、$\gamma$ 三种系统的比较（$N_R = 100$，$\sigma_{n_{s_r}} = .5\bar{n}_{s_r}$，$N_A = 10{,}000$，$P_a = .07$，$\omega = .2$）。

Let us now replace the "ideal environment" assumptions with a model for a "differentiated environment," in which several distinguishable classes of stimuli are present (such as squares, circles, and triangles, or the letters of the alphabet). If we then design an experiment in which the stimuli associated to each response are drawn from a different class, then the learning curves of the perceptron are drastically altered. The most important difference is that the constant $c_1$ (the coefficient of $n_{s_r}$ in the numerator of $Z$) is no longer equal to zero, so that Equation 4 now has a nonrandom asymptote. Moreover, in the form for $P_g$ (the probability of correct generalization), where $c_2 = 0$, the quantity $Z$ remains greater than zero, and $P_g$ actually approaches the same asymptote as $P_r$. Thus the equation for the perceptron's performance after infinite experience with each class of stimuli is identical for $P_r$ and $P_g$:

$$P_{r\infty} = P_{g\infty} = \left[1 - (1-P_a)^{N_e}\right] \times \phi\!\left(\frac{c_1}{\sqrt{c_3}}\right) \tag{9}$$

This means that in the limit it makes no difference whether the perceptron has seen a particular test stimulus before or not; if the stimuli are drawn from a differentiated environment, the performance will be equally good in either case.

现在我们用“分化环境”模型来代替“理想环境”假设。在分化环境中存在若干可区分的刺激类别（如正方形、圆形和三角形，或字母表中的字母）。如果我们设计这样一个实验，使与每个反应相联结的刺激取自不同的类别，那么感知机的学习曲线就会发生剧烈变化。最重要的区别在于，常数 $c_1$（$Z$ 的分子中 $n_{s_r}$ 的系数）不再等于零，因此方程 4 现在具有一个非随机的渐近值。而且，在 $P_g$（正确泛化的概率）的形式中，虽然 $c_2 = 0$，$Z$ 仍大于零，$P_g$ 实际上趋近于与 $P_r$ 相同的渐近值。因此，在对每类刺激都有无限经验之后，感知机表现的方程对 $P_r$ 和 $P_g$ 是相同的：

$$P_{r\infty} = P_{g\infty} = \left[1 - (1-P_a)^{N_e}\right] \times \phi\!\left(\frac{c_1}{\sqrt{c_3}}\right) \tag{9}$$

这意味着，在极限情况下，感知机以前是否见过某个特定测试刺激并无区别；如果刺激取自分化环境，两种情况下的表现同样好。

In order to evaluate the performance of the system in a differentiated environment, it is necessary to define the quantity $P_{c\alpha\beta}$. This quantity is interpreted as the expected value of $P_c$ between pairs of stimuli drawn at random from classes $\alpha$ and $\beta$. In particular, $P_{c11}$ is the expected value of $P_c$ between members of the same class, and $P_{c12}$ is the expected value of $P_c$ between an $S_1$ stimulus drawn from Class 1 and an $S_2$ stimulus drawn from Class 2. $P_{c1x}$ is the expected value of $P_c$ between members of Class 1 and stimuli drawn at random from all other classes in the environment.

为了评估系统在分化环境中的表现，需要定义量 $P_{c\alpha\beta}$。它被解释为从类 $\alpha$ 和类 $\beta$ 中随机抽取的刺激对之间 $P_c$ 的期望值。特别地，$P_{c11}$ 是同类成员之间 $P_c$ 的期望值，$P_{c12}$ 是取自类 1 的刺激 $S_1$ 与取自类 2 的刺激 $S_2$ 之间 $P_c$ 的期望值。$P_{c1x}$ 是类 1 的成员与从环境中所有其他类别随机抽取的刺激之间 $P_c$ 的期望值。

If $P_{c11} > P_a > P_{c12}$, the limiting performance of the perceptron ($P_{g\infty}$) will be better than chance, and learning of some response, $R_1$, as the proper "generalization response" for members of Class 1 should eventually occur. If the above inequality is not met, then improvement over chance performance may not occur, and the Class 2 response is likely to occur instead. It can be shown (15) that for most simple geometrical forms, which we ordinarily regard as "similar," the required inequality can be met, if the parameters of the system are properly chosen.

如果 $P_{c11} > P_a > P_{c12}$，感知机的极限表现（$P_{g\infty}$）将好于随机，最终应能学会把某个反应 $R_1$ 作为类 1 成员的适当“泛化反应”。如果上述不等式不满足，表现可能无法超过随机水平，而且很可能出现类 2 的反应。可以证明（15），对于我们通常认为“相似”的大多数简单几何形状，只要系统参数选择得当，所需的不等式就能满足。

The equation for $P_r$, for the sum-discriminating version of an $\alpha$-perceptron, in a differentiated environment where $n_{s_r}$ is fixed for all responses, will have the following expressions for the four coefficients:

$$\left.\begin{aligned} c_1 &= P_a N_e (P_{c11} - P_{c12}) \\ c_2 &= P_a N_e (1 - P_{c11}) \\ c_3 &= \sum_{r=1,2} P_a (1-P_a) N_e \\ &\quad \times \big[P_{c\,1r}^{2} + \sigma_3^2(P_{c1r}) + \sigma_j^2(P_{c1r}) + (\omega N_R - 1)^2 \\ &\quad \times (P_{c1x} + \sigma_s^2(P_{c1x}) + \sigma_j^2(P_{crx})) + 2(\omega N_R - 1) \\ &\quad (P_{c1r} P_{c1x})\big] + P_a^{\,2} N_e^{\,2} \\ &\quad \times \big[\sigma_s^2(P_{c1r}) + (\omega N_R - 1)^2 \\ &\quad \times \sigma_s^2(P_{c1x}) + 2(\omega N_R - 1)\epsilon\big] \\ c_4 &= \sum_{r=1,2} P_a N_e \big[P_{c1r} - P_{c1r}^{\,2} \\ &\quad - \sigma_s^2(P_{c1r}) - \sigma_j^2(P_{c1r}) \\ &\quad + (\omega N_R - 1)(P_{c1x} - P_{c1x}^{\,2} \\ &\quad - \sigma_j^2(P_{c1x}))\big] \end{aligned}\right\} \tag{10}$$

where

$\sigma_s^2(P_{c1r})$ and $\sigma_s^2(P_{c1x})$ represent the variance of $P_{c1r}$ and $P_{c1x}$ measured over the set of possible test stimuli, $S_t$, and

$\sigma_j^2(P_{c1r})$ and $\sigma_j^2(P_{c1x})$ represent the variance of $P_{c1r}$ and $P_{c1x}$ measured over the set of all A-units, $a_j$.

$\epsilon$ = covariance of $P_{c1r}P_{c1x}$, which is assumed to be negligible.

对于 $\alpha$ 感知机的总和辨别版本，在 $n_{s_r}$ 对所有反应都固定的分化环境中，$P_r$ 方程的四个系数表达式如下：

$$\left.\begin{aligned} c_1 &= P_a N_e (P_{c11} - P_{c12}) \\ c_2 &= P_a N_e (1 - P_{c11}) \\ c_3 &= \sum_{r=1,2} P_a (1-P_a) N_e \\ &\quad \times \big[P_{c\,1r}^{2} + \sigma_3^2(P_{c1r}) + \sigma_j^2(P_{c1r}) + (\omega N_R - 1)^2 \\ &\quad \times (P_{c1x} + \sigma_s^2(P_{c1x}) + \sigma_j^2(P_{crx})) + 2(\omega N_R - 1) \\ &\quad (P_{c1r} P_{c1x})\big] + P_a^{\,2} N_e^{\,2} \\ &\quad \times \big[\sigma_s^2(P_{c1r}) + (\omega N_R - 1)^2 \\ &\quad \times \sigma_s^2(P_{c1x}) + 2(\omega N_R - 1)\epsilon\big] \\ c_4 &= \sum_{r=1,2} P_a N_e \big[P_{c1r} - P_{c1r}^{\,2} \\ &\quad - \sigma_s^2(P_{c1r}) - \sigma_j^2(P_{c1r}) \\ &\quad + (\omega N_R - 1)(P_{c1x} - P_{c1x}^{\,2} \\ &\quad - \sigma_j^2(P_{c1x}))\big] \end{aligned}\right\} \tag{10}$$

其中

$\sigma_s^2(P_{c1r})$ 和 $\sigma_s^2(P_{c1x})$ 表示 $P_{c1r}$ 和 $P_{c1x}$ 在所有可能测试刺激 $S_t$ 上测得的方差；

$\sigma_j^2(P_{c1r})$ 和 $\sigma_j^2(P_{c1x})$ 表示 $P_{c1r}$ 和 $P_{c1x}$ 在所有 A 单元 $a_j$ 上测得的方差；

$\epsilon$ = $P_{c1r}P_{c1x}$ 的协方差，假定可以忽略。

（译注：(10) 式中 $P_{c\,1r}^{2}$ 原刊印作 $P_c{}^2{}_{1r}$；$\sigma_3^2(P_{c1r})$、$\sigma_j^2(P_{crx})$ 以及方括号的配对均按原刊照录，未作推断性修改。）