Hi, welcome to session 129. My name is Richard Williamson and I'm a Safari Web Kit engineer. This is advanced Web Kit development. So I'm going to talk a little bit about how you can use Web Kit, which is, as you all know, the technology behind Safari, and now a lot of other applications that ship with OS X.

And really what I want to talk about is Web Kit, but not just as it's used in browsing. Web Kit's not just about browsing. So to set the stage for this, I'd like to talk about a couple of examples of browser-hosted applications and then see how they compare to desktop applications.

So one common technique used for writing a browser hosted application is to have server generated content that changes via form interaction with web pages. Pretty primitive, but a lot of pages actually quite successful with this technique. A lot of banking sites, for example, use this kind of approach.

More recently, applications have started to put more interactivity into the client using JavaScript and XMLHttpRequest, and really only interacting with the server for the application logic. And this is a much better approach to building browser-hosted applications. A great example of this is maps.google.com. On this site, you can actually interact with the map without having to load a new web page. And the interaction happens using XMLHttpRequest.

And this is now being called Ajax. Go figure why it's being called Ajax, but it is. So these are two different approaches that are being used for browser hosted applications and there's many others. These are just a couple of examples. And the point I want to make is that the same technologies that are used for browser-hosted applications can be used in desktop applications.

And let me give you a couple of examples. So an application can embed Web Kit and the client code can generate content. A great example of this is dictionary.app that ships with OS X. The content area here is rendered with Web Kit and the form area above the content area is Cocoa, which is the GUI toolkit Cocoa. And as the user enters data in the form, the content area updates.

Another example that's analogous to the browser hosted application is a user interface that is using Web Kit to render content using JavaScript to do dynamic HTML and XML HTTP requests to interact with server application logic. A couple of the widgets that we ship do this. The stocks widget and the weather widget.

What I'm advocating here is a new approach to building your desktop applications. That approach is to use Web Kit for the layout of your user interfaces. Why do this? Well, it's easy to iterate on the interface. It's very easy to leverage rich media types. And HTML and modern CSS provide you with very powerful layout capabilities. and very importantly, a lot of designers out there, H.I. designers, have existing skill sets in designing HTML and CSS content. So that can be leveraged now in your desktop applications. And lastly, it's very easy to style the interfaces.

So you might be saying, but why, really? There's a lot of existing techniques that I have for building application interfaces, lots of GUI toolkits. Cocoa is great. Why should I do this? You might also say, "InnoHTML really isn't capable enough. It doesn't have the kind of techniques that I need to use in my application. It just doesn't cut it." I can't access native code. I really want to access my existing code base that has my application logic.

If I really do want to extend HTML, writing a plug-in is really difficult. The API is just too hard to access. and JavaScript. JavaScript isn't like C or Objective C. It's too slow. It doesn't cut it. That's not true. And I'm going to show you why it's not true.

How do you do this? You use Web Kit. In those areas where Web Kit isn't appropriate or isn't capable out of the box for fulfilling your needs, you can extend and enhance it by writing custom plug-ins, by using the powerful DOM API, by extending JavaScript, and by accessing native code if necessary.

So what I'm going to talk about specifically in this session is three technologies in Web Kit. The Web Kit plugin API, scripting using Objective-C and JavaScript, and the Objective-C DOM bindings to the actual DOM document model. And because this is an advanced session, I'm not going to spend a lot of time showing you code on slides. We're actually going to go through an example of building an application from scratch. And I'll touch on each of these areas in that example.

Before we actually get to coding, I'm going to show you just a few slides to give you an overview of the technologies that we're going to use and the APIs we're going to use. First, Web Kit plugins. You may have already heard about Web Kit plugins in other sessions, both this year and last year. They're very easy to implement. They're a standard Cocoa bundle. And the key object in a Web Kit plugin is an NSView. It's just like any other NSView. And it's very easy to script these plugins.

The API really couldn't be more simple. To implement a plugin, it's really one method on the principal class of your plugin. And that's plugin view with arguments. And the arguments that are passed to this method are the attributes that are specified on the embed tag that specifies the plugin on your web page. Here's an example of some of the attributes that are passed or could be passed.

In addition to that one method which you have to implement, there are some additional methods that you can optionally implement. There are lifecycle methods. Plugin initialize, start, stop, and destroy. And initialize is called when the plugin is first created. Start and stop may be called multiple times as the plugin is suspended or resumed. And destroy is called when the plugin is actually destroyed, when the page goes away.

So in addition to those lifecycle methods, there's one additional method here for scripting, object for web script. And that one little innocuous method is actually really powerful. And I'll go into some depth about how we can use that method to script a plug-in. And then finally, there's a method that will let you alter the appearance of your plugin content based upon whether or not the plugin is selected.

And also in this upcoming demo, we're going to touch on this area. And I assume you all know how to use Xcode. So I'm not actually going to show you the entire interface for these two different areas. You can look it up in the Xcode documentation. It's quite comprehensive.

There's really two areas. There's accessing JavaScript objects from Objective C. And JavaScript objects in Objective C are represented by what's called a Web Script object. And this is really a proxy for the JavaScript object, but as it exists in Objective C. And then the other side of this is if you have an existing Objective C object and you want to expose it to JavaScript, and to do that, all you have to do is implement an informal protocol. And then you have the Web Scripting protocol on your existing Objective C class. And this can be any Objective C class. So again, the complete documentation for this is in Web Script object.h. Okay.

So let's move to some code. So what I'd like to do is write a little application that is a picture frame application. And I'd like to show several images in a picture frame. I'm just going to use these nine pictures here. And What I'd like to do is inside the Web content, use Core Image to make transitions between these images. So this will be completely inside the Web page, leveraging Core Image.

So the first thing I need to do is create the HTML that's going to represent the user interface for this application. So what I have here is a little template that's relatively simple, and all it's doing right now is loading one of those images. Pretty straightforward. The first thing I'd like to do is use CSS to style this page. So let's add a little bit of CSS. And I have two external style sheets that I'm going to use.

Let me just show you those style sheets. All this style sheet is doing is -- specifying a gray background color for the border cells that surround the image. Let's go ahead and load that in Safari again and see what it looks like. A little bit more interesting, but still not that exciting. So what I'd really like to do is use a series of images to create something that looks more like a real picture frame. So let's go ahead and do that.

And to do that, all I have to do is change the style. And what I have here is a series of images that represent

is going to show you the image that we actually want to show. So to make that change, all I'm going to do is point to a different style sheet.

will save that. And that style sheet looks pretty much identical to the previous style sheet, except instead of using a background color, we're now using a background image. Let's go ahead and load that. Now we have something that looks a little better. It looks like a real picture frame. Okay. So now what I'd like to do, though, is provide a control that will let me transition between the different images that I'd like to show. So let's do that with a bit more JavaScript.

In fact, I'm going to show you a little bit more of What we're going to do is just add a reference to an existing piece of JavaScript that will is going to show you how to use a div effectively as a button. Let's go ahead and do that and see what it looks like. What I've done here is add a div with a mouse down handler, and that mouse down handler is just going to change the source of the image element. Up to now, this is all just basic HTML stuff.

We haven't really accessed the power of Web Kit, but that's coming up. So the other thing I wanted to do is I only want this control to show up as I move over the page. So this is done using mouse over and mouse out handlers. And now as I click, it's going to -- on a timer, it's going to update the images.

So standard HTML, CSS, and JavaScript. I haven't done anything special with Web Kit. So in looking at Core Image, what I wanted to do is try and leverage some of these sophisticated transitions that are available. And I actually just went out and looked at some of the sample code that is available for Core Image, and I've leveraged that in this demo.

So the first thing we need to do is because I know HTML doesn't support this directly, I'm going to use a plug-in. And where we now specify an image element, that's going to change to be--

One interesting thing here is the type attribute which specifies the MIME type for my custom plug-in.

So if we try and load this right now, it's not going to work. Let's go ahead and try and load it. Actually, let me make sure I didn't install the plug-in. Actually, okay. I didn't paste it into the file. said it can't find the plugin. Let's go ahead and write the plugin.

This is Xcode 2.1, so I have to upgrade. Okay. So what I've done, let me just first of all talk about the view class. This is a standard Cocoa view class.

is the heart of the custom code. This is no different than any other NSVU. In fact, I pretty much took this verbatim from the Core Image demo that does a similar kind of thing.

And there were a few additional methods on the view to start and stop a transition. But the plug-in code is the interesting code that we care about for this particular demo. And you'll see here, plug-in view with arguments is the method that I have to implement on my plug-in. And all it does is create a transition image view.

That's the class that is going to do the transition between the images. It does a couple of additional things. It gets the width and the height of the plug-in. and it gets the source attribute, which lets you specify an initial image to show in the plugin. And then it does a little bit of logic to deal with relative URLs.

and then there's a couple of the lifecycle methods that we've implemented, plugin start and plugin stop.

And there's a pause and resume method which just call Web Plugin Stop and Stop, just a nicer name for those same methods. Okay. So that's the core of the plug-in. We've implemented the API that we have to implement.

But now what I'd like to do is from JavaScript, I'd like to control the actions of the plug-in via script code. So to do that, I have to export methods. And by default, the object that you export from the plug-in, which we do -- I'm sorry, : This little innocuous method that I mentioned earlier, object for WebScript, you're just going to export self.

And by default, none of the methods or properties of your object are exported. This is because methods like Dialog don't make a lot of sense to export, and it also could be a potential security risk to export methods which

A method to set the source image, that's the beginning image for the transition. A method to set the target image for the transition. A method to set the particular transition number. And a method to perform the transition. And then two other methods to stop and start the transition.

Additionally, we'd like the names of these methods in JavaScript to be reasonable. We want the API to look good. And in Objective C, we have these funny colons that are used in names, and those don't make a lot of sense in JavaScript. So what this method does is replace those Objective C names with names which are more palatable for JavaScript. And in particular, you can see things like set transition has a colon in Objective C. In JavaScript, it's just going to be called set transition. And we have no properties that we're going to export.

So we're going to exclude all of the properties from this object from visibility in Objective C.

is the developer of the core image functionality to set up the transitions and actually generate an image for a particular transition. Like I said, this is pretty much verbatim from the developer examples for core image.

So that's pretty much it. We've implemented We've implemented the essential plug-in functionality, and we've implemented the scripting functionality. Let's go ahead and build this. and install it. Now, to install a plugin, they just go in any of the internet plugins directories and slash library internet plugins or tilde library internet plugins. And it built successfully. I'm going to copy this from my build directory into Tilda Library Internet Plugins.

And now if I reload this page, I have to actually restart Safari. Let's go to the history item to bring up that page. Now instead of the missing plug-in icon, we actually have the first image. And now when I start the transitions, I can Oh, actually, nothing happened.

Why did nothing happen? Well, I haven't actually changed the JavaScript to perform the transition and to set the current transition that we want to use. Let's go ahead and go back to our HTML and take a look at what we'll need to do that. So I have a different JavaScript file that I'm going to be using here that's going to be imported into the HTML file, frame transition.JavaScript. Let's take a look at what that does.

So on my button mouse down handler, remember, this is the mouse down handler of the div from our Web page. Let's take a look back at that. is the unmoused . This is the piece of JavaScript that will be called when we select that play button. is doing is standard JavaScript stuff to get the button image.

And here we have some code that's going to leverage our custom scripting on the plugin. It's going to find the plugin element.

And then it's going to do some manipulations to change the current image that's being displayed. We're just going to implement a counter into an OA. And then it's going to do some manipulations to change the current image that's being displayed. We're just going to implement a counter into an OA. will show you how to use the new DOM binding.

And then we're going to set a transition number. And we're just going to increment that transition number so that each time we display a new image, we'll use a different transition. And then we'll perform the transition. is going to show you how to use the DOM binding. That's pretty much all there is to it. And then we'll repeat this on a three-second timer. So every three seconds, we're going to update the image. Now let's take a look at what that looks like.

Oops. Let me leave that running. So actually, this is great, but it's still inside a browser. I talked about using custom applications and not using the browser to host my application. So what will it take to transform this into an application? Let's do that next. Because one of the things I'd like to do is use the Open Panel to select a set of images that are shown in that picture frame.

So these are the six lines of code it takes to transform that web page into an application. That's it. What we're doing is getting the HTML document, getting the web frame, which I'll show you in a minute, in the nib file, and loading the request into the web frame to load the particular HTML page. This nib file really couldn't be more simple. at Gatekeeper asking me if I want to open Interface Builder. Yes, I do. So this is just a simple window with a single web view in it. Let's go ahead and build and run this.

Now we have the same web page in an application. And you'll notice the same kinds of effects that we expect to work still work. So when I mouse over and mouse out, the control appears and disappears. Now we have the same web page in an application. And you'll notice the same kinds of effects that we expect to work still work.

What I'd like to do next is leverage some of the capabilities of Cocoa, like the Open Panel, something very straightforward. So how do we do that? Well, Back in Interface Builder, I'm simply-- I think if you were to do an application more completely, you'd want to use the NS Document architecture. But for now, I'm just going to have a single window. And I'm going to connect the File Open menu item to my app controller.

You can see that there is an existing connection between the open menu item and the app controller, and it's just going to send open to the app controller. For some reason, Interface Builder isn't letting me draw this line. That looks like a bug. So anyway, there's another source code file for the app controller here that actually implements the open method.

And this is the method that will be sent from that open menu item to my app controller. And this is standard Cocoa code. It's going to open up an open panel. It's going to run it, and then it's going to pull out a set of URLs that represent my selection.

So that's all pretty standard except for this line of code down here. JS window call web script method, set images with arguments and an array of strings that we've received from the open panel. And what this is doing is actually sending a JavaScript method, set images to the window object.

And we stored that window object using this delegate method, window script object available. And this is one of the frame load delegate methods that you can implement on your frame load delegate object, which is in this case the app controller. So let's go ahead and take a look at the HTML now and the corresponding set images method.

is going to show you how to use the DOM binding. This is an object that we created in Objective-C and we're passing to JavaScript and it's going to represent the array of URL strings that were selected in the open panel. And all we do is we update the this hard coded list of images. Pretty straightforward. Let's go ahead and run this again and see how this works. I'm going to bring up the open panel. I happen to have a set of images here from my daughter's recent second birthday.

Let's take a look at those. So what we've seen is how to create a user interface using Safari really as a development tool to iterate on the design of my user interface, how to turn that into an application, and then how to leverage Cocoa functionality. And how it really is possible to use Objective-C and JavaScript in very interactive ways. You can really leverage the capabilities of Objective-C, but still have the flexibility of JavaScript.

There's one additional thing I want to show you, which is an alternate deployment option you now have. You've got a piece of code and some HTML that can be deployed as a widget. So what does it take to turn this into a widget? Well, pretty much nothing. Let's go ahead and look at the widget version of this.

is going to show you how to use the DOM binding. This is pretty much exactly the same HTML code in JavaScript as the application, except I shrank it a little bit just to be more like a widget, not take up so much screen real estate. I'm actually going to move this onto my desktop. and you guys know about that special hack? Yeah, it's pretty great. Okay. There's a default you can write called com.apple.dashboard.devmode. And it allows you -- you set that to yes, and it allows you to run your widgets on the desktop outside of the dashboard layer.

So that's great, but one of the things I'd like to do in this widget is change the images, and there's no open dialogue that's accessible to me from the widget. : So how can I do that? Well, I'm not going to show you a lot of detail because the session following this is actually going to use the same widget and talk about how you can use some of the dashboard capabilities to implement what I'm going to show you. But let's just pull up

Okay. Back to slides. So what we've seen is really how it is possible to leverage these technologies in ways that use the Web Kit for your presentation, but still access native code and the power of Cocoa.

Okay. Back to slides. So what we've seen is really how it is possible to leverage these technologies in ways that use the Web Kit for your presentation, but still access native code and the power of Cocoa. This is an advanced session, so I'm sure you all know about the JavaScript DOM API. This is the same API, but available to you from Objective-C.

The API is a W3 standard. You can go to the W3 site and look at the API. And I've given you an example here of the IDL definition of one of the W3C methods. And it's binding to C and it's binding to Objective C. And there's one thing I want to note, and that is Objective C has a funny syntax, as we all know. It's got the colons. And just as we had an issue mapping Objective C names to JavaScript, there's an issue mapping IDL names to Objective C.

So what we chose to do is use the base name of the IDL method as the first component of the Objective C name. So in this case, get named item NS, which stands for namespace, is the IDL definition. And the Objective C method for this is get named item NS.

And then for each of the parameters that are associated with the IDL definition, we have a colon. So in this case, there are...

So why would you use this API versus the JavaScript DOM API? Well, excuse me, it's really a matter of convenience. And another possible reason is performance.

JavaScript is plenty fast for doing DHTML, but if you want to do something like iterate over every node in a complex document, it might be more appropriate to use Objective C. In fact, what I'm going to do now is show you a demo of doing exactly the same thing in JavaScript versus Objective-C, and we'll time it and see what the difference is.

So what I decided to do is go out on the web and look for a large document. In fact, I decided to go and find a Shakespeare play. In this case, I'm going to use the Comedy of Errors. And this page is the entire play in one large HTML file. So far, it's pretty zippy loading it. It's a huge file.

: This is somewhat of a contrived example, but what I like to do in JavaScript is iterate over every node in the DOM, a computer hash of the text nodes in the document. And do it from JavaScript. and the hash is really stupid. It's just going to add up the value of each of the character codes. Not a very good hash, but it illustrates the point. Let me pull up what it is. I saved that HTML file and I added a little bit of JavaScript to it. So let me pull that up.

So you can see what I did is I added a reference to hash text.js to the top of the HTML file. Let's take a look at what that does. So there's an unload handler that gets called. And that unload handler iterates or traverses over all of the nodes in the document. And it calls this hash function, which I implemented in JavaScript to very simply, as I said, just add up all of the character codes in the document.

And there are other ways you could possibly implement this using the Tree Walker or the Node Iterator JavaScript objects. This is just one example of how you can iterate over a document. And the other thing I've done is I've recorded the start and end times to compute the total time it takes to do this. And I've added up the number of text nodes that we add, and I'm going to report the cumulative hash value in an alert.

Let's load this in JavaScript -- I mean, in Safari and see --

: So that took two seconds to iterate over the entire document and compute a cumulative hash. Not bad. There were almost 6,000 nodes in the document. But we can probably do better if we use native code to do that.

In fact, we can do better. Let's take a look now at is going to load that page. And this is very familiar to you now from the previous example. : We have a single WebView and we're just going to load that Comedy of Errors HTML page into our application.

And now In our didFinishLoad method, we have essentially exactly the same algorithm that we had in JavaScript. But instead of being implemented in JavaScript, it's implemented in Objective-C. Let's take a look at the Traverse Next Node method. And let's compare that to the JavaScript method. There might not be enough screen real estate to do this.

You can see side by side, the code is almost identical. The only thing that's different is the syntax. And similarly, the hash function is essentially identical, except for the syntax differences. Okay, so now what I want to do is report the results from this application using a Cocoa Alert panel. And do you think there will be a difference in performance? Let's see.

2 seconds to .04 seconds. The same number of text nodes in the same cumulative hash. So if there are any concerns about JavaScript speed being not sufficient, and you shouldn't really have concerns, because for the most part, it's more than capable and more than fast enough to do most of the kinds of things you want to do for an interactive interface. But if you choose to implement computationally intensive algorithms like this in JavaScript and are too slow, you now have an alternative.

You can do the same thing in Objective C and bind that into JavaScript. So in fact, what I wanted to do is -- I'm will show you how you can take this same Objective-C code and reference it from your JavaScript code by exporting the Objective-C functionality in much the same way that we exported functionality from the plug-in to JavaScript. We're now just going to export a piece of Objective-C code to JavaScript.

So the way we do this is, again, to leverage a delegate method, Windows Script object available, And this will let us set additional -- well, let us access the window object from JavaScript from Objective C. And in this case, all we're doing is we're adding self as a new property of the window object, and we're going to call it hash.

So now this object will appear in JavaScript under the name hash, and it's going to implement is going to show you how to use the new DOM binding. We're going to export in exactly the same way that we exported methods from the plug-in. Now from this class, you can see we're exporting the selector hash and we're going to rename it for visibility in JavaScript as hash without the colon.

And then let's go ahead and take a look at the I'm going to show you the HTML now that we're going to use, because what I need to do is report from JavaScript is going to show you how to use the hash value. We actually want to have JavaScript to generate this hash and report it. So let's take a look at that code.

is going to do is instead of all of that other code that we had previously in JavaScript, it's just going to call hash. is a developer at Microsoft who has been working on a new This is what we would do by default. Hash.hash. Because we're exporting a hash object, and that hash object has a hash method. So it's a little bit awkward, but let's just live with that for one second and run this and see what the value is.

: One other thing I want to mention before we run it, though, is this is calling alert to report the results. And in Safari, the Web UI delegate by default will put is going to show you how to implement the UI delegate. I haven't implemented the UI delegate, so the message isn't going to get shown anywhere.

We're going to implement one is the UI delegate method. Run JavaScript alert panel with message. We're going to log it to the console. This means when I run the application in Xcode, which we'll do now, the console will report the results. So the results showed up in this alert because that was the alert that we generated from the Objective-C code. But you'll also see in the console is reporting the same numbers.

So this is JavaScript code calling into Objective C to compute the hash and then reporting via the Web UI delegate the results. So let's just do one final thing here, and that is clean up this messy API. I really don't want to have to call hash.hash to compute the hash of a particular node.

I just want to call hash. So how do we do that? We do that by implementing one of the web scripting methods that I mentioned earlier. These are documented in web script object.h. And that's invoke default method with arguments. So if you export an Objective-C object to JavaScript and you try and operate on it as a function, which is what you're essentially doing. : So, in this JavaScript code, we consider that to be a default method, and this method will be called in JavaScript.

So what we're doing here in the default method is the same thing that we do in the hash method, which is to call hash. So let's go ahead and run that, and you'll see exactly the same results that we saw before.

So what you've learned today is how to write a Web Kit plugin to leverage really powerful capabilities. Core image really gives you a lot of powerful rendering capabilities, and it is as easy as what I've shown you to leverage that.

Richard Williamson You've seen scripting between Objective-C and JavaScript and how it really is easy to mix and match code between JavaScript and Objective-C as necessary. And you've seen the powerful capabilities that we've exposed via the DOM Objective-C bindings. Richard Williamson So that's pretty much all I have to show you today. For more information, You can go to the WWDC site. You're familiar with that.

There's a couple of sessions I want to mention that are of interest. The upcoming Safari for Web designer session will talk more about the HTML and JavaScript that you can leverage inside of Safari that will let you create great websites. And then there's another really good session for a lot of folks, and that's the hands-on advanced dashboard widgets. So if you're interested in how we made this application into a widget and some of the details of that, I would recommend going to that session. And if you have any particular questions after the conference, you can contact Mark Malone.