And please welcome Cocoa Fire Frameworks Engineer, Douglas Davidson. Good afternoon, everyone. Thank you all for coming. My name is Douglas Davidson. I'm here to talk to you about the Cocoa Text system. The text system, in some senses, is at the heart of Cocoa. Almost everything in the app kit makes use of text in one way or another and generally uses the text system to do so. The Cocoa text system is very powerful and very flexible. It has lots of features, and there are quite a few new features for Tiger.

Doug Davidson So what I'd like to do today is, first of all, go over briefly what we have that's new for Tiger. And then I'm going to dive into some of the inner workings of the text system so that by the end of the talk, I can really give you a detailed description of some of the most significant new features and how they work.

So what do we have that's new for Tiger? Those of you who were here last year may remember I talked about, among other things, our support for Word documents, and that was pretty well received. But Some people would ask me why we didn't really handle tables in Word documents, and I had to tell them the Cocoa Text system doesn't support tables. So, when I got my task list for Tiger, you can guess what the first thing on it was. Support tables in the Cocoa Text system. So, I'm glad to say that for Tiger, the Cocoa Text system will support tables.

We thought about this a bit though, and we realized that while tables are important for RTF and Word documents and so forth, what they're really critical for is HTML. So for Tiger, we've also put a lot of effort into our HTML support using that table support. And in addition, entirely new for Tiger, the Cocoa Text system will now export HTML.

So let me talk about this. Now, I've been speaking of this as text table support for short, because everyone understands that. But what we really have here is something that's a lot more general. What we have is a flexible, extensible mechanism for sizing and positioning text blocks of all sorts, the first application of which is to the representation of tables from RTF or Word documents or HTML or HTML and CSS.

Now, when I get to the end of the talk, I'm going to be describing how this works in great detail. I just want to say that it is a very general mechanism of text blocks, and our intent is to use it for all sorts of complex layout we might encounter.

For HTML import, you may recall that in Panther, we have two different kinds of HTML import. There is an older style that has some support for sophisticated layout, but that is rather out of date and limited in the kinds of HTML that it can parse. And then there's a second kind, new for Panther, that used WebKit for parsing and so could handle arbitrary HTML, but was rather limited in its layout support. So for Tiger, all of that is gone. We have a completely new HTML import mechanism. It uses WebKit for parsing, so it can handle any HTML you're likely to find. And it uses the new text block and text table mechanism for complex layout support.

For HTML export, whenever you talk about generating HTML, the question always comes up, "What kind of HTML is it that you're going to generate?" So when we were planning this feature, we took a little informal survey of some of the groups at Apple that we thought might want to use it. And the results were unanimous. Every group wanted a different kind of HTML.

So we decided we'd just have to satisfy all of them. And what we have is a mechanism that gives you control over this. It allows you to specify exactly which HTML tags should be generated. This gives you a lot of control. You can generate HTML. You can generate XHTML, strict or transitional.

Of course, it's automatically going to be valid and well-formed. You can use CSS in a variety of ways, or you can use no CSS. You could generate a complete HTML document or just a fragment and all sorts of other options. Take a look at the release notes. I think you're going to like it.

Now, we're only about halfway through the Tiger development cycle, so these things are not finished by any means, but if we'll go over to the demo machine, I'll give you a brief look at some of the things that are working. So, I have a little demo application here, so let me just type into it.

Thank you, Doug. Thank you, Doug. So, I'm going to go ahead and save this. Maybe make this bold. And let me go and save it. And I have several -- I've added several options here for saving this. I could save it as rich text or HTML, Word format. We've also added support for saving this Word's XML format, WordML.

Let me save it as HTML. Let me choose to save it as HTML 4.01 transitional. For CSS, I'm going to choose to use CSS inline for the encoding. Well, let me leave it as UTF-8 because, you know, that's the right thing to do. And let me give it a name and save it out. Let's see what kind of thing we generated. Open it to show the source. And we have an HTML document.

Let me bring up, I have a little Excel spreadsheet here. Select it, copy, paste it in here, and I get a real table. Let me just select that and bump up the size a little so you can see it better. Let me make it bold. Well, that's working pretty well so far. Let me try something a little harder.

So in Safari, I brought up a possibly recognizable website. Let's just take a look at the source for that. This thing usually has about 50 tables nested four or five deep. So let me take this and save it. I'll save it using WebKit's new Web Archive format, which of course we now support. And then let me go back and see if I can open that up.

"Select it all, maybe make it bold, and then save it back out. Yes, override it." And so here it is. Let me see if I can open it back up in Safari. And there it is. I can see a few glitches that I'll have to work on, but recognizably the same website, only all bold.

So, let me emphasize that this is an import and export mechanism. The HTML that we're producing is rather different from the HTML that went in. You'll notice, for one thing, it's valid and well-formed and nicely formatted, which the original certainly wasn't. So while I'm here, there's one more thing I wanted to show you.

will be joining us in a few minutes. Maybe I want to find all of the tigers in it. So I could just find the tigers, but Now we have the possibility to select all of them as a multiple discontinuous selection. Maybe make them a little bigger. Make it bold, italic, underline, and so forth.

Okay, let's go back to the slides and I can talk a little about that. Multiple selection, how does this work? Well, there are a variety of methods in NSTextView that act upon or take a selected range. So now for Tiger, all those will have a counterpart that works on selected ranges, which is going to be an array of range values. The precise details you'll find in the release notes as to exactly what they take.

What about compatibility? Any method that isn't multiple selection savvy will still continue to work just fine on the selected range, which will just be the first selected sub-range if there is a multiple selection. And this turns out to be a pretty good default. Works just fine in most cases. In fact, there are still some methods in the C that you have. There are still some methods in the app kit that haven't been updated. And they continue to work just fine operating on the first selected sub-range.

So what else is new for Tiger? Well, now that we're reading and writing so many different formats for import and export, we've stopped adding a new method for each format. Instead, we provided a set of generic methods that take the document type as a parameter, an option, or attribute.

And some new document attributes specifically for HTML export, including those that provide the control that I mentioned over the kind of HTML that's being generated, the tags and the encoding. And finally, we have a new document option for HTML import, the Text Size Multiplier. This corresponds to the little buttons in Safari that, say, make the text size larger or smaller.

What else do we have? Text lists. I'm going to be talking more about text lists later on. Let me say that our intent is that the text list feature should be used to enable NS TextView to automatically generate the markers for text lists, that is, the numbers of the bullets. That is not implemented in the seed that you have. What we have currently is that we are using this for designating lists in HTML import and HTML export.

We have some additions to NS paragraph style. First of all, this is where the text blocks and tables and text lists are attached to the text, as I'll be talking about later on. We also have the hyphenation factor. Previously, this was set up only globally per layout manager.

Now, we actually allow you to set it individually on an individual paragraph. We have another factor now, the tightening factor for truncation. If you request truncation with ellipses, we will first try to tighten up the text before truncating it with ellipses, and this controls how much we will try to do that.

And also, just for support of HTML, we have a header level on Paragraphs that allows us to distinguish the text that is an HTML header from ordinary paragraph text. String Drawing We have some new methods for string drawing. They are much like the old ones, but they have additional options. One common request is to be able to do string drawing with the origin at the baseline. That is now the default with these new methods.

We also have the option of measuring using either the typographic bounds as usual or the actual image bounds of the glyphs. Previously, the primary method for specifying a font has been by name, by the PostScript name. For Tiger, we are promoting NSFontDescriptor as the standard means for specifying a font.

It has a lot more capabilities. It can specify by name, by size, by traits, by the stylistic type of the font. It can even group together multiple fonts to serve as a cascade. If you went to the font talk earlier this week, there was a considerable discussion about NSFontDescriptor.

There have been some changes to NS-Typesetter. We're deprecating now NS-Simple Horizontal Typesetter, although it will still be there for backward compatibility. We have taken a lot of the functionality from NS-ATS Typesetter that wasn't really ATS-specific and moved it up into NS-Typesetter. This makes it easier to create a whole new custom typesetting engine and hook it up into the text system.

In addition, we've added one new method here for the typesetter when it wishes to obtain a rectangle for a line. I won't read out the method name. We're not trying here to compete for the longest method name. I think Bitmap Binge Wrap has that all sewn up. But actually, all of these parameters are really necessary and useful, as we will see when we come to the table layout.

And finally, some more miscellaneous additions. We have a couple of new responder methods that are implemented in NSTextView. One to insert a line break, that is, a line break as opposed to a paragraph break. And one to insert a container break, usually a page break. We have a new delegate method in NSTextView. It allows the delegate to control any changes to the typing attributes. This is in addition to our previous notification.

NSLayoutManager now has a getter and setter for its glyph generator. There are a number of new convenience methods for manipulating the base writing direction for right-to-left and left-to-right text on attributed strings in TextView. And finally, we have a number of new panels that are not yet implemented in the new seed, but are planned for inspecting and modifying links, lists, and tables in text, and they have methods for bringing them forward.

So, we've seen what's new. Now I'm going to move into the heart of the talk. My aim is, by the end of the talk, I can discuss in detail exactly how the new block and table mechanism works. But in order to get there, first I'm going to talk, I'm going to review a bit about how the layout process works in the text system.

And then I'm going to discuss some of the existing classes that are most relevant and analogous to the new mechanism. Fortunately, these are classes that I haven't really discussed in detail in previous years, NSTextContainer and NSTextAttachment. will review the NS Text List class, and then I'll get to the text blocks and tables.

So, to review, let's talk about the major players in the text system. At the model level, the main class is NSTextStorage, which stores the text of the document. Direct subclass of NSMutableAttributedString, so it holds the characters and their attributes. Then we have the classes that typically serve as values for attributes, and as font for fonts, and as paragraph style for paragraph-level attributes, and as text attachment for attached files. And there's NSTextContainer, which models the geometry of a region within which text is to be laid out, typically a page.

At the controller level, the central class of the text system, NSLayoutManager, that manages the whole process and calls on a couple of other classes, NSLiftGenerator and NSTypeSetter, to do some work for it. And at the view level, there's a visible face of the text system and its text view.

Now, the job of the text system really is to go from the characters and their attributes in the text storage to the bits that you see on the screen. And to do this, to make this happen, there are four basic processes that occur. First of all, attribute fixing.

This is where the text storage does this and makes sure that the attributes are consistent. That is, that the fonts can actually represent the characters to which they're attached. And the paragraph level attributes actually apply to whole paragraphs. Then comes glyph generation. This is controlled by the Layout Manager. The Layout Manager calls on the glyph generator to do this. This is where the characters and their fonts are converted into a sequence of glyphs.

Then comes layout, again controlled by the Layout Manager, and calls upon the typesetter to do it. The typesetter takes these glyphs and assembles them into lines positioned on the page. And then comes Display, again, done by the Layout Manager. It takes these glyphs and their positions, and it sends them on down to Quartz to be rendered on a screen.

We have a little diagram of how this occurs. On the one side, we have the glyphs that are generated by the glyph generator. And as they are assembled into lines by the typesetter within the geometry that's determined by the text container, they are then displayed in the text view, which sits in the window.

I want to focus particularly on layout because that's the most important process in the new text block and table mechanism. And the way that this occurs in detail is that the layout manager decides that a particular range of lists needs to be laid out. Then it calls on the typesetter and asks it to lay them out.

The typesetter then contacts the text container. Remember, the text container models the geometry of the region within which the text is being laid out. So the typesetter calls on the text container to ask it about that geometry, to determine the rectangles for the lines of text on the page. Then the typesetter takes its glyphs and it fills up these lines with the glyphs. It figures out exactly where each glyph should go in each line.

And then the typesetter calls back to the layout manager to tell the layout manager what it has done, where each line sits on the page, which glyphs go in it, and exactly where they go in each line. And the layout manager stores this information because it will need it later on for display and interaction and all other purposes.

So let me talk a little bit more about NSTextContainer. It models the geometry of the region within which text is to be laid out, typically a page, although it could be other things. The stock NSTextContainer object that you'll get from the app kit represents just a simple rectangle.

You can create a custom NS Text Container subclass that can represent essentially an arbitrary region. It will still have a rectangle that is this bounding rectangle, but within that, it gets to control where the text will be laid out. And the way that works is that, as I said, the typesetter, when it's laying out, calls on the text container, and it asks the text container, it proposes a rectangle for a line, and the text container gets to modify that, and return whatever rectangle for the line it likes.

Here's a little diagram of that, showing a possible custom text container with some of the rectangles for the lines that it might have returned to the typesetter. That's a conceptual overview. Let's go back over to the demo machine and see if I can show it to you in action. So let me bring up -- let's see this and start it and run it.

This is just some text, but I have set a custom text container here on this text view. And this text container has a parameter, and as I modify the parameter, the region within which the text is laid out will change. So it will go one way or the other. Let me just select it all so you can see what the actual line rects look like. So I modify that. Let's take a look at the code.

There's really only one main method that has to be implemented here. This class has some parameters, but they just control what happens in this method. The typesetter is going to propose a line fragment rectangle, and then we are going to get to modify it. And in this case, what we do is we trim it based on this sinusoidal curve that is the shape of this text container. So we figure out where we are on our sine wave, and in one case, we're going to trim it on the left side.

And the other case, we just trim it on the right side, and then we just return our modified rectangle. And that's all there is to it. There is one more little detail. We do have to implement this method to notify the Layout Manager that we are not just a simple rectangular text container, so that it can turn off certain optimizations. And that is all that it takes, really, to implement a custom text container. Let's go back to the slides.

We talk about another class for text attachments. In TextEdit, you may have dragged in an image or some other file into the text, and it gets attached to the text at a specific location, and it's represented by an image or some icon that's drawn inline in the text.

And the class, the way that this appears in the text storage is as a special character, the attachment character, with a special attribute, the attachment attribute. And the value of the attribute is an instance of the class NSTextAttachment. NSTextAttachment does two things. First of all, it represents the contents of the attached file as an NSFileWrapper. And second, it has to provide some sort of visual representation to be drawn inline in the text.

And to do that drawing, it uses a cell, an NSTextAttachment cell, to be specific. Now, by default, if you're dragging something into TextEdit, the NSTextAttachment will automatically generate an appropriate cell and image or other representation, depending on the contents of the file. But as developers, you don't have to let it do that. You can assign your text attachment a custom text attachment cell, either one of a standard class with perhaps a custom image attached to it.

If you were at the tips and tricks talk the other day, I gave an example of that. Or you can give it an instance of a custom subclass of NSTextAttachment cell. With a custom subclass of NSTextAttachment cell, you can do essentially arbitrary drawing inline in the text. How does this work? Well, during glyph generation, the glyph generator will notice, it'll come across the attachment and not try to generate a glyph because there's no glyph to represent it. It just puts in a null glyph as a placeholder for it in the glyph stream. Then at layout time, a typesetter will come across it. Notice that there is an attachment there.

What it needs to know about it is what space that attachment will take up in the text so that it can position it in the line. So it actually calls to the text attachment cell, passing it as arguments, the position in the text and line fragment and so forth, where it is being laid out. And the text attachment cell gets to decide how big it should be there and return that to the typesetter. And the typesetter then reserves that space for it. during layout.

Then when it comes along to display time, the Layout Manager, as I said, the Layout Manager handles the display. The Layout Manager notices that it's not an ordinary glyph, it's a text attachment, and it calls on the text attachment cell to do the drawing in the space that was reserved for it.

So that a custom text attachment cell can do more or less arbitrary drawing in an arbitrary space inline in the text. But there's also one more thing, and that is that the text view will notice where there is a text attachment, and it will give the text attachment cell the opportunity to handle mouse clicks in that region. So you can even do some interaction with a custom text attachment cell. If we go back to the demo machine, I want to give a tiny little demo.

So this is the same application, but if I click on this box, I will put in some custom text attachments. And these are just here. This example is just a horizontal line. I put a few of them in here.

That's pretty short. As I said, the methods that you have to implement are, first of all, when the typesetter asks the text detachment cell how big it is, here we're returning a rectangle whose size and position is just determined by the size of the line fragment within which we're being laid out, just a third of it. And then when it comes time to draw, we get a chance to do our own drawing, called upon by the layout manager. And here I'm just doing something very simple, setting black and filling that rect.

We're gonna talk about a new class, NSTextList. The basic principle of our text list support is that all of the text of the list, including the marker, the bullet or numbering, will appear in the text as usual. The NSTextList object itself will appear as an attribute on the text, and it will do primarily two things. First of all, it will specify which portions of the text lie within which part of the list, of which list, and it will also determine what the formatting of the markers will be.

And as I said, our intent is that this should be used by NS TextView for automatic generation of markers, be they bullets or numbers or what have you, that is not yet implemented in your seed. Currently, we use this for specifying lists for HTML import and HTML export.

Now, how does this actually appear in the text? The text list, the NSTextList object is intended to be a paragraph-level attribute, so it appears as part of NSParagraphStyle. But Lists can be nested. So a given region of text may not just be in one list, it may be in multiple lists. So NSParagraphStyle doesn't have just one NSTextList, it has an array of NSTextLists listed in order from outermost to the innermost.

That's how we specify which portion of the text lies in which lists. And we have a couple of convenience methods now in NSIntributedString for determining the entire range of a particular list or the location in a list of a particular item. And the text list, NSTextList object itself, has some methods for specifying the format of its markers. I'm not going to describe that now. You can look at the release notes. It's fairly standard. We support a number of basic list formatting options. with possibly more to come in the future.

Now here I have a little diagram to show how this works. Here we have two nested lists, an outer list and an inner list. Some of the text is only in the outer list. In that case, the text lists array just has the outer list in it. And some of the text is in both lists, the outer and the inner. And for those, the text lists array would have the outer list first, then the inner list.

So now we have all the ingredients to understand how our text block and text table support works. I've said that NSTextContainer represents the geometry within which the text is laid out, and it's quite general and flexible at doing that. But it's essentially static. It doesn't depend on the text that's being laid out in it. The point of the new mechanism and the new class, NSTextBlock, is to allow the text, attributes on the text, to affect how it is laid out.

So, an NSText block is going to be attached to the text much like an NSText list is, as an attribute, and it's going to participate in the layout process somewhat as an NSText attachment does. The contents, the text in the text blocks, will all appear in the text as normal, and it will all be laid out perfectly normally and laid out and displayed with one little change, and that is that the text block is allowed to affect the line rectangles within which the text is being laid out.

So, the way that the text blocks appear in the text is very similar to that in which text lists appear, because blocks, again, are going to be paragraph-level attributes, so they appear on the paragraph style, and they, too, can be nested. So, a given section of text can be in more than one block.

And so, in its paragraph style, in addition to the text lists array, it also has an array of text blocks, again, in order from outermost to innermost. And again, we have a couple of convenience methods on this attributed string for determining the entire range of a particular block or table. I'll get to tables in a minute.

So how is this treated when it comes to layout? During the layout process, we are going to define two rectangles for a given text block. The first one is the layout rectangle. That is the rectangle within which the text of the block is actually going to be laid out.

And then there is the bounds rectangle, which is a rectangle around the text of the block that also includes space for margin, borders, padding, any additional region around the text that is not available for the layout of other text. The layout rect for a block has to be determined just as layout for the block is starting, immediately before the block is laid out.

And the way this happens is that the typesetter notices that it's starting to lay out a block of text, and it calls on its text block and asks it what its layout rect should be. It passes in the location at which it's starting to lay out text and a containing rectangle, which for the outermost rectangle, would be a bounding rect for the text container, but for inner blocks, would be the rect of the immediately enclosing block.

And then the bounds rect has to be determined at the end of layout for the block immediately, as soon as the block has finished being laid out, because the bounds rect generally is affected by the length of the text in the block. And again, the typesetter notices that it's finished laying out a block, and calls on the block, and asks it for its bounds rect.

And the NSText block gets to determine that rect and pass it back to the typesetter. And the typesetter then, as usual, calls out to the layout manager and tells the layout manager what it has found out, stores these two rects for the block in the layout manager, and the layout manager keeps that information and maintains it.

Here's a little diagram. So when this block is being laid out, it's laid out in this layout rectangle, which is typically a long rectangle when we're allowing the block to be of arbitrary height. Then after the block has been laid out, then the bounds rectangle is sort of wrapped around the text with any extra space needed for, say, margins, border, padding, whatever the block wants to put there.

Then when it comes time for display, again, the Layout Manager manages the display. The Layout Manager actually has two methods for display, one to draw background and one to draw the glyphs. So when its method is called for drawing background, There's one addition, and that is that it notices if there are text blocks to be drawn, and it calls upon those text blocks in order from outermost to innermost and asks them to do any drawing they need to do.

And in that case, they can draw backgrounds, they can draw borders, any sort of decoration that they need. And then the usual glyph background is drawn on top of that. And then when the Layout Manager's method for drawing glyphs is called, the glyphs are simply drawn on top of all this background and show up in the right places.

Now, all that is a general text block mechanism. There is also a specific version for text tables. And we have a subclass of NSTextBlock called NSTextTableBlock. NSTextTableBlock represents a block that appears as a single cell within a table. The distinguishing feature of the table layout is that the different cells in the table have to coordinate their layout with each other. So there is another object, NSTextTable, that represents the table as a whole. And each text table block, each cell, has a reference to the table as a whole.

And when it comes time for layout or for display, as I've said, the layout manager calls upon the block, in this case, the NSTextTableBlock. And NSTextTableBlock is the block that represents the table as a whole. And when NSTextTableBlock as a subclass does something special, it passes those requests on to its NSTextTable, which coordinates them with all the other cells of the table and returns the appropriate results or does the appropriate display.

Now, all this is the general mechanism. The standard text block and text tables and text table block classes in the kit have a number of parameters that determine how they actually do size and position themselves. These are things like margin, border, and padding widths on each side, content width, and so forth.

They can be specified either as an absolute point value or as a percentage of the enclosing block. And text blocks will draw a background color if specified. They can also specify border colors, if you like, a separate one for each side. You can look at the details. They're all in the release notes. It should be not terribly surprising.

What I want to emphasize here is that these do not limit you. The mechanism is very general. If you have a custom subclass of an as-text block, you can do any sort of sizing and positioning and decoration that you like. So let's go back to the demo machine and I'll give a little demonstration of this.

Now, I said that the custom, that the standard text block class in the kit will do a background color. It doesn't, at least not yet, support, for example, background images. But there's nothing to prevent you from writing your own text block class that will do that. And I've written this tiny little sample here that does a background image.

And in addition, it does some custom sizing. It sizes the block to the size of the image. Now, let me turn that on. Notice that here is the text up here in the block. And I can select it, make it bigger, make it bigger if we like, and so forth.

The methods that we need to implement for this are, first of all, when we start the layout of the block, we need to get the layout rect. And the typesetter will call upon this custom subclass and ask it for the layout rect. And here all that I'm doing is determining the layout rect based upon -- determining its width based upon the size of the image. And to be nice, I've made it centered horizontally in the containing rect, so there's a little bit of calculation here.

But it's pretty straightforward. Then when we're called upon to generate the bounds rect -- What I'm, although I'm doing here, is trying to make the bounce rect equal to the size, defined by the size of the image. Of course, it might be that our container is not quite big enough for the image, or it might be that it's bigger and we need to center it. So there's a little bit of calculation here to determine the bounce rect in those cases, and we return that.

And then when it comes time to draw, All that we're going to do is just draw this background image. But again, the rect might be smaller than we actually wanted, or it might be bigger than we actually wanted. So we're going to adjust for that trim or center one way or the other. And then so there's a little more calculation here. But again, quite straightforward. And then we just draw the image in the rect.

That is what is necessary to make a custom subclass of NS Text Block. Now let's go back to the slides. and finish up. That's what I have to present to you today. If you want to see more, the first place to go is the release notes, which has detailed descriptions of all this.

For the pre-Tiger features, there is extensive documentation. Actually some of the nice diagrams I showed you here were taken directly from that documentation, and the samples that I showed you here should be available for download on the ADC website, the disk image for this session. and contact person, Matthew Formica and [email protected].