Overview
An Outline of a Theory of Semantic Information is a technical research report written jointly by Rudolf Carnap and Yehoshua Bar-Hillel and issued by the Research Laboratory of Electronics at the Massachusetts Institute of Technology. It is not a general treatise or a textbook but a compact programmatic paper: its purpose is to define, within the framework of Carnap’s inductive logic, a precise and quantitative concept of the semantic information carried by a declarative sentence — an “outline” of a theory rather than the finished theory.
The problem the report sets
The report opens from a deliberate contrast with the mathematical theory of communication. Shannon’s measure, the authors argue, quantifies the transmission of signals through a channel; it is concerned with how much a receiver’s uncertainty about which message will arrive is reduced, and it remains entirely indifferent to what the messages mean. Two messages of equal length and equal statistical rarity count as equally informative on that account, even if one is a tautology and the other a substantial factual claim. Carnap and Bar-Hillel propose to supply the missing semantic counterpart: a measure of how much content a sentence has, derived not from signal statistics but from the logical relations among sentences and from logical (inductive) probability.
The framework they exploit is one Carnap had already developed: the analysis of a formalized language by means of state descriptions and a logical probability measure defined over them. The report is thus an application and extension of the apparatus of Logical Foundations of Probability to the specific problem of measuring information, and it presupposes the distinction, central to Carnap’s programme, between purely logical (L-) concepts — L-truth, L-falsity, L-implication, L-equivalence, and degrees of confirmation — and empirical ones.
The model language and state descriptions
To make the definitions concrete the authors introduce a deliberately simplified language system, containing a finite number n of individual constants (“things”) and a finite number π of primitive one-place predicates. The atomic sentences of this language attribute a predicate or its negation to an individual; molecular sentences are built from these with the ordinary connectives. The basic semantic units are state descriptions: maximal conjunctions that, for every individual in the universe and every predicate in the vocabulary, state whether that predicate applies. A state description therefore describes one complete possible state of the whole finite universe, and every sentence of the language is logically equivalent to a disjunction of some set of state descriptions — its range.
The illustrative example used is a tiny universe of three individuals with two pairs of opposed predicates, roughly “young/old” and “male/female,” so that the reader can see the full space of possible state descriptions enumerated and can follow the arithmetic of the definitions on a small scale. The authors are explicit that this choice is expository: the aim is to display the machinery, not to model any actual empirical subject matter.
Content and amount of information
On this basis two notions are separated. The content of a sentence is the class of state descriptions that the sentence rules out — those possible states of the universe in which it would be false. Content so understood is a set-theoretic object, and comparisons of content can be made by inclusion: one sentence has more content than another when the class of state descriptions it excludes properly contains the other’s.
To obtain a numerical measure, the authors invoke a logical probability function m defined on state descriptions and extended additively to arbitrary sentences by summing over their ranges. Two magnitudes are then distinguished:
- a content measure, defined as one minus the logical probability of the sentence, i.e. the logical probability of the class of state descriptions it excludes;
- an amount of information, defined as a decreasing function of the sentence’s logical probability — the reciprocal of that probability, or equivalently its negative logarithm, so that the measure behaves additively.
The properties of these measures are then examined. Tautologies come out with zero content and zero information, as they should. Logically equivalent sentences receive identical values; L-implication is reflected in an ordering of content; and the measures are additive in the appropriate cases — the amount of information of the conjunction of two inductively independent sentences is the sum of their separate amounts. Various possible choices of the underlying m-function are canvassed, and the authors stress that the numerical values depend both on the vocabulary of the language chosen and on the particular logical probability measure adopted: the qualitative ordering of contents is more robust than any particular number.
The report is also candid about the difficulties. Because amount of information varies inversely with logical probability, the most improbable sentences are the most informative, and a self-contradictory sentence — having logical probability zero — would receive a maximal or undefined value. The authors acknowledge this awkward consequence rather than resolving it, and the point was later taken up in the literature as the “Bar-Hillel–Carnap paradox.” Related problems are raised for the information carried by scientific laws, whose universal form cannot be captured by a finite enumeration of state descriptions over a finite universe, and for the contrast between the amount of information a sentence carries and its value or utility for a given inquiry — a pragmatic dimension the report marks off as a separate question rather than treating.
Scope and significance within the text
The report confines itself to this programme: the motivation from communication theory, the construction of the model language, the definitions of content, content measure, and amount of information, the statement and proof of their formal properties, illustrative calculations in the sample language, and a discussion of open problems. It closes with the case for pursuing a semantic theory of information as a branch of inductive logic rather than of engineering, and with the suggestion that the apparatus could be extended to richer languages and applied to the analysis of the informativeness of hypotheses in the methodology of science. It does not develop a full theory of meaning, does not treat questions, commands, or non-declarative discourse, and does not offer an account of the transmission of information between agents.
The report’s influence lies in having fixed the vocabulary — state descriptions, ranges, content, semantic information, the inverse relation between informativeness and probability — that later work in formal epistemology and the philosophy of information would either build on or argue against.
Further themes and elaborations
The report defines two distinct quantitative measures on the same foundation. The content measure of a sentence is the logical probability of its content — equivalently, one minus the logical probability of the sentence itself, since the content is the set of state descriptions the sentence excludes. This measure is bounded between zero and one and behaves like a probability on contents: the content measures of a sentence and its negation sum to one, and the content of a tautology is zero. The amount of information is the logarithmic measure already described. The report presents both measures and examines their respective properties, without insisting that one is uniquely correct. The content measure is natural when information is thought of as the proportion of logical space ruled out; the logarithmic measure is natural when information is thought of as an additive quantity. The qualitative ordering of sentences by informativeness is common to both.
The framework is extended to relative or conditional information. Given a hypothesis h and evidence e, the amount of information of h relative to e is defined as the negative logarithm of the conditional logical probability of h given e — that is, of the degree of confirmation c(h, e). This relativized measure allows the theory to express how much information a piece of evidence carries about a hypothesis, and it connects the semantic theory of information directly to the apparatus of inductive logic. The report notes that this relative information can be negative, since evidence may lower the probability of a hypothesis, and it treats this as a feature of the formalism rather than a defect.
The report is explicit that it is offering an explication of the concept of information in Carnap’s technical sense: a precise replacement for a vague pre-theoretic notion, to be judged by its fruitfulness rather than by its fidelity to ordinary usage. It does not claim that content, as defined, captures everything that might be meant by “meaning” or “information.” Content is a formal surrogate — the set of possibilities a sentence excludes — and the report treats it as adequate for the purposes of measuring informativeness while leaving other dimensions of meaning aside. This methodological framing is important: the report is not proposing a definition of information in the ordinary-language sense, but a formal reconstruction of one aspect of it.
The report draws out the set-theoretic consequences of its definitions. The content of a conjunction is the union of the contents of its conjuncts, and the content of a disjunction is the intersection of the contents of its disjuncts. These relations mirror the logical relations among sentences and give the theory a clean algebraic structure. The report also notes that the content measures of a sentence and its negation sum to one, even though the sentences themselves are not jointly exhaustive in any stronger sense. These properties are stated and proved for the model language, and they serve as the basis for the illustrative calculations.
The report extends the definitions from single sentences to conjunctions and, by extension, to sets of sentences. The content of a conjunction is the union of the contents of its conjuncts, so a body of claims has a content that grows as further claims are conjoined. This allows the theory to represent the informativeness of a theory or a set of hypotheses, not just of individual sentences, and it prepares the ground for the application to the methodology of science mentioned in the report’s closing sections.
