Dennis Snell: See DATA, CDATA, RCDATA, and PCDATA oh my!

HTML and XML are markup languages based on plaintext files. This means that any given character could be part of a syntax form (a tag, a comment, a character reference, etc…) or it could be representing itself the way it reads in the file literally.

<tag>&middot; Text node</tag>

Whenever a character might be ambiguous, both languages require explicit indication of the intent of the character. In HTML this occurs via escaping, while XML allows escaping or wrapping the content in a marked section, specifically a CDATA section.

<tag>
<![CDATA[<tag>· Text node</tag>]]>

These terms confuse me at times, especially since CDATA and CDATA sections are distinct forms of the same content, and it’s easy to conflate each term. This post is here to disambiguate the terms, their meanings, and why they exist.

The punchline comes at the end, but the story is hopefully worth the read.

Markup and mixed content

One of the first jobs of a parser for any plaintext-oriented format is to determine if the next input character represents real text or is part of a syntax form that carries special meaning. If it’s a syntax form we would call it markup, but if the characters are part of real text meant for display or rendering or reading then we call it data.

Anything that is not syntax is data.

The interpretation of the next character depends on the region of the document in which it’s parsed. While the rules for syntax forms are complicated¹, this post will focus on the data forms.

PCDATA — “parsed character data”

May form: tags, comments, sections, character references, literal text.

Characters in this region could be data or could form the start of a new markup element. It’s “parsed” because it needs parsing before determining what it represents.

The HTML specification renames this to Data, which is simpler and a bit harder to search for. In XML, however, it’s used in a document-type definition (DTD). When an element may contain content — text — its data model must include #PCDATA. Otherwise the only characters allowable within that element are other elements, comments, and whitespace. XML documents are required to be valid SGML documents, so its own specification adopts the terminology from SGML’s.

Those who have worked with DTDs might note that elements in XML may contain #PCDATA while attributes contain CDATA instead. First of all, the # is there only to make it explicit that PCDATA is referring to the reserved keyword, rather than a <pcdata> element. Secondly, there’s a good reason for this, which is that attributes can only contain text — they can’t contain other elements of markup. If an attribute value could contain a <span> element, for example, then the attribute value would need to be #PCDATA instead, but this is prevented by design.

PCDATA actually contains more than just literal text and elements. In addition to comments, processing instructions, and other node-like syntax, one important feature of PCDATA is the character reference. These make it possible to represent characters that would conflate with syntax (such as ‘<’ — <) or which might be cumbersome to enter on a keyboard (such as ‘§’ — §). When parsing, each character in these sequences neither creates an element nor displays as the text itself; rather, the entire sequence is parsed and translates into the character it refers to.

HTML pre-specifies a fixed set of named character references, but any Unicode code point may be referenced by its decimal or hexadecimal numeric index. While XML also allows referencing code points by their index², it only pre-specifies the five named characters which correspond to its main markup introducers: <, >, &, ', and ". In XML, any additional named character references are created through the DTD by defining entities.

CDATA — “character data”

May form: [character references], literal text.

If a character isn’t markup, then it’s character data, which means that it’s representing its literal self or it’s part of a character reference. Once the parser has entered this region it will not create markup elements.

CDATA is the most confusable kind of character data; this is because there are many kinds of CDATA that share the same name:

• XML attributes may contain CDATA, where character references are decoded.
• XML CDATA sections only contain CDATA, but character references are not decoded.
• HTML kind of has the same CDATA sections, but only in foreign elements (inlined SVG and MathML elements).
• SGML elements may be declared to have a CDATA content model, in which case all content until the appropriate closing tag is to be parsed as character data, where character references are not decoded.

CDATA sections contain only literal text

Many people are familiar with CDATA sections, but it took me far longer to understand them than my intuition led on. They are the vestige of SGML “marked regions” which tell the parser to handle a specific range of bytes in a special way. The CDATA section is one of those, which tells the parser to completely turn off until it reaches ]]>.

<![CDATA[literal characters only in here]]>

It had other marked sections, however, which served different purposes.

<![IGNORE[everything in here is ignored; it doesn’t exist.]]>
<![INCLUDE[in here things <em>do</em> exist as normal.]]>
<![RCDATA[read on to learn about RCDATA!]]>

The IGNORE and INCLUDE sections may seem strange, since SGML already has comments, and INCLUDE effectively does nothing, but the sections can be marked by replaced entities, making for conditional inclusion which can be overwritten via command-line arguments when invoking the SGML parser.

<!ENTITY % review-only "IGNORE">
...
<![%review-only;[
<aside>
Add `-Dreview-only=INCLUDE` when building drafts.
This note won’t appear otherwise.
</aside>
]]>

XML only retained CDATA sections from SGML, while HTML never included them. They are useful because they are so easy to parse. All characters inside of them are to be treated as literal text, up until the first occurrence of the terminating ]]>. Unlike elements, the marked sections do not nest.

There are no CDATA sections in HTML

The Internet is full of discussions about the use of CDATA sections in HTML, but there are no such things, mostly. HTML itself is an amalgam of pure HTML and embedded SVG and MathML. Content inside of those embedded SVG and MathML elements is parsed differently, and within this “foreign content” there are CDATA section nodes.

When something which look like a CDATA section appears in an HTML document, it’s transformed into a “bogus” HTML comment and considered a snippet of malformed markup. To make things more confusing, the parsing rules differ inside an HTML document for these regions depending on whether they are found within HTML elements or foreign elements.

• When a real CDATA section appears within SVG and MathML, it parses as in XML or SGML — everything is literal text until the nearest ]]>.
• When a malformed CDATA look-alike appears in an HTML element, it gets special treatment — the parser only turns off until the nearest >. This means that these sections end even without a closing ]]>, and when they do, all of their contained content disappears from the page.

That small difference confuses naïve parsers and is a regular source of bugs.

<div><![CDATA[There are no tags in here.]]></div>
<svg><text><![CDATA[<none> here either.]]></text></svg>
<div><![CDATA[But there <em>are</em> tags in here]]></div>
     the section ends here ╯   ╰ start of a real end tag
The following is the equivalent markup to the third line.
<div><!--But there <em-->are</em> tags in here]]></div>

SGML contains CDATA regions outside of marked CDATA sections

SGML made it possible to define more kinds of content than XML does for a given element. For example, an element in SGML can be declared to have a CDATA content model, in which case the element itself behaves like a CDATA section. All characters after the opening tag are treated as literal text until the parser finds the nearest appropriate end tag³. XML rejected this ability because it increases the complexity of the parser and requires that every document also contains a full DTD when parsing. For example, if an element were declared to have CDATA content, then a <at> b would represent that literal string; on the other hand, if it were declared like any other normal element, it would have three children: “a ”, the <at> opening tag, and “ b”.

<!ELEMENT verbatim - - CDATA>
...
<verbatim>
There are <no> tags in here, because this is CDATA,
but you wouldn’t know without reading the DTD,
overcomplicating the demands on the parser.
</verbatim>

These kinds of elements do exist in HTML, though a few were modified when HTML5 was standardized in 2008. Inside of the elements, the parser essentially turns off, which makes them easy to parse and can help avoid the need to extensively escape content. These elements are, of course, <script> and <style>⁴.

Were it not for the CDATA declared content model, every angle bracket and ampersand would have to be escaped in included JavaScript and CSS. In XHTML this was required, because it had no CDATA declared content model (since it was XML)⁵.

All text in XML is CDATA

Herein lies the most-confusing aspect of discussing CDATA — XML contains CDATA sections as well as CDATA as normal text. After parsing there is no distinction between <tag> and <![CDATA[<tag>]]> in the parsed content.

Many XML generators (or serializers) provide two mechanisms for creating text content: one wraps text in a CDATA section and leaves the text as it came (apart from avoiding including the terminating sequence); the other escapes syntax characters instead. While there are times where it would be appropriate to intentionally pick one over the other, a good library design would at least offer a third mechanism (if not only providing this third mechanism) which simply produces CDATA, itself determining when to wrap and when to escape⁶, and whether or not to produce chunks of wrapped text interspersed with chunks of escaped text.

The real difference between these two kinds of CDATA is purely presentational in the source document, as the XML snippet below only contains one text node, not two. Creating CDATA does not imply creating a CDATA section!

<rule><![CDATA[#X13<d&r>]]> (&pp;4 &ss;3.11)</rule>

RCDATA — “replaceable character data”

May form: character references, literal text.

There’s one more confusing designation for characters in the HTML and XML input streams: RCDATA. RCDATA is almost identical to CDATA, except that in contexts where CDATA does not decode character references and entities, RCDATA will decode them into CDATA. This is confusing, because in the context of an XML attribute, the CDATA designation in a DTD automatically implies that character references are decoded, unlike the CDATA sections in content.

To this end there are no RCDATA attributes, since character references are always decoded inside attribute values. The RCDATA declaration is like the SGML CDATA content declaration: all characters following the opening tag for this element will be treated as text until the nearest matching closing tag (the difference being only that character references are recognized and decoded).

It’s worth remembering that XML rejected the CDATA content type because of how it complicates parsing, and it also rejected the RCDATA type. On the other hand, RCDATA was incorporated into HTML, but statically so. HTML has no configurable DTD, but in its specification two elements contain RCDATA content:

• TITLE
• TEXTAREA

While it’s easy to comprehend the way that <textarea> works, and that’s probably because we are used to entering text into one on a web page, the behavior of <title> is consistently confused in all manner of programming languages, platforms, and HTML-parsing code.

The TITLE element only contains character data — it cannot contain other markup. The parsing is among the easiest sections of an HTML document to parse: once the <title> opening tag is detected, the parser can capture everything until the nearest </title> closing tag. Everything it captured is literal text, after decoding character references.

<!-- the title is "<title>" -->
<title><title></title>
<!-- equivalent HTML -->
<title>&lt;title&gt;</title>

This complicates content management systems like WordPress which allow posts to have HTML in their post titles, because a page can show richly-formatted article titles which cannot be represented in the browser tab’s label, and care must be taken to extract the plaintext content from that HTML before display in those contexts.

Coda

HTML and XML both speak about different kinds of characters in their source documents and content models, which traces from the complicated ways that SGML documents could be constructed. SGML’s complexity almost always stems from the central idea that computers should do extra work to remove the hassle for humans to enter structured content in plaintext documents.

HTML, inspired by SGML, adopted some of the names and mechanisms for parsing those regions of text in distinct ways, but codified a single parsing standard independent of SGML. When XML was later developed, it was meant to form a simplified subset of SGML. This subset flipped the tradeoffs, leaning on humans performing extra work to remove the hassle for computers to parse structure in plaintext documents. For these text forms, this meant rejecting a few of the constructs while retaining others.

This is also another demonstration of how balanced tags are not enough to have well-behaved HTML with a naïve parser. A well-formed XML document may be parsed with a terse PERL script and regular expression, but HTML relies heavily on the context in which characters are found. Any HTML parser must know the special rules for each kind of element’s content model.

In summary

• When it’s unclear whether a character forms text or markup, that is PCDATA. Once parsed, there is no PCDATA anymore; it’s either a form of DATA or MARKUP.
• All text nodes in HTML are “DATA.”
• “CDATA” just means “character data” and means that after parsing, the content is text. It does not indicate whether character references are to be decoded or not; that comes from the region in the document, based on its context.
• There are no CDATA sections in HTML⁷.
• All text nodes in XML are CDATA, but only after being parsed.
• CDATA sections offer a convenient way to avoid escaping, but are indistinguishable from the equivalent escaped text.
• HTML contains two special RCDATA elements which only and always contain a single text node child: <title> and <textarea>. Everything until the closing tag will be parsed as text, even if it looks like markup.

This post is already long and still over-simplifies the picture. SGML is a rich and robust specification and includes NDATA and SDATA, HTML includes a latching PLAINTEXT parsing mode in which the rest of the entire document is parsed as literal character data, and there are other surprising goodies in how entities interact with the character mode.

Thanks for making it through to the end, or jumping directly here if you couldn’t wait.

1. As an example, each part of a tag — its name, attribute names, attribute values — carries its own parsing rules. The same is true for comments, DOCTYPE declarations, and every other syntax form. ︎
2. XML only allows character references to the characters in its “character set,” which is almost all Unicode code points, but excludes some control characters and U+FFFE and U+FFFF. ︎
3. Because SGML was designed to minimize the amount of necessary syntax, it’s not necessary to have a full end tag for an open element, but that’s a simple-enough model to understand the concept. ︎
4. The <style> element is straightforward, but the <script> element has its own complicated modification of the CDATA content model. It’s mostly CDATA, but makes it possible to escape the closing tag so that very old pages won’t break. HTML also applies this parsing mode for the <iframe>, <noembed>, <noframes>, and <noscript> elements (as well as for the deprecated <xmp> element), but these nominally should have no content inside of them (or shouldn’t be used); applying the CDATA content model prevents creating other elements as their children. ︎
5. Frustratingly, in XHTML one must escape JavaScript and CSS in the page to avoid parsing failure, while in HTML one must not. This alone makes for a complicated stage in any reliable HTML/XHTML converter. ︎
6. Wrapping a language like HTML inside a CDATA section is a convenient way to represent the HTML visually and retain the ability to easily modify it, but entities present a problem. The serializer must either pre-translate the entity into its resolved character content, losing the macro-like behavior and its name; or leave the entity in place, thus nullifying it because it will not be recognized as an entity on parse. However, in such a situation, a serializer is free to terminate the CDATA section, append the entity, and open a new one to continue. ︎
7. As mentioned in the discussion about CDATA, embedded SVG and MathML elements can contain CDATA sections, but these are not technically HTML elements. ︎