Configure player

Close

WWDC Index does not host video files

If you have access to video files, you can configure a URL pattern to be used in a video player.

URL pattern

preview

Use any of these variables in your URL pattern, the pattern is stored in your browsers' local storage.

$id
ID of session: tech-talks-2011-extra-1
$eventId
ID of event: tech-talks
$eventContentId
ID of session without event part: 2011-extra-1
$eventShortId
Shortened ID of event: tech-talks
$year
Year of session: 2011
$extension
Extension of original filename: m4v
$filenameAlmostEvery
Filename from "(Almost) Every..." gist: [2011] [Tech Talk China] Buildin...

China Tech Talks #1

Building and Optimizing Websites for iOS

2011 • 37:30

Get an overview of the technologies available for Safari on iOS, and learn how to leverage newly-added features such as accelerometer and gyroscope support.

Speaker: Vicki Murley

Unlisted on Apple Developer site

Transcript

This transcript was generated using Whisper, it has known transcription errors. We are working on an improved version.

Hi, I'm Vicki Murley, the Safari Technologies Evangelist at Apple. The number of users browsing the web with Safari on iOS is growing every day. In this video, I'm going to give you an overview of the available technologies which you can use to deliver an awesome experience to iOS users visiting your website. Since this is an overview, I won't be going into all the details of every available technology, but along the way I'll point you to additional, in-depth resources so that you can get more information on each technology as needed.

So let's get started.

[Transcript missing]

In fact, WebKit is used as the layout engine in Safari on Mac OS X, Windows, and iOS. Besides being blazingly fast and providing a small memory footprint, which is great for mobile devices, another one of WebKit's strengths is that it has great support for HTML5.

So HTML5 is the technology that you can use to deliver rich web experiences in Safari on all platforms, including iOS. These days, the phrase "HTML5" has kind of a dual meaning. First of all, the term "HTML5" refers to a formal specification from the W3C. This is a specification that's used by web browser implementers to determine how different tags and technologies should behave. There are a lot of technologies in the HTML5 specification that are supported by Safari on iOS, including web sockets, geolocation, enclosed captions, and full screen mode for video.

So that's kind of the first meaning of the term HTML5, referring to that W3C specification. Lately, the term HTML5 is also being used as sort of a buzzword or catchphrase to refer to a collection of technologies that web developers can use to deliver rich internet applications and rich web page experiences. In that collection, we of course have the HTML5 specification. We also have CSS3 for adding advanced styling and animation.

And finally, we have JavaScript to tie it all together. JavaScript is our interface to many of the elements in the HTML5. We can use it to change CSS styles, and it gives us some additional features such as touch and gesture detection in web pages. I'm going to touch on each of these and how you can use them in Safari on iOS, but let me start with HTML5. The HTML5 specification is very large, so today I'm just going to touch on two key features of this spec: audio and video elements and offline data storage.

Let's get started talking about the HTML5 audio and video elements. These are standard HTML elements that you can use to embed media in web pages. Why should you use HTML to embed audio and video in web pages? First of all, these elements are the best way to deliver media in web pages on iOS.

If you want to display video or play audio in your website in Safari on iOS, this is the way to do it. Secondly, HTML is also a great way to deliver media because of built-in optimizations. For example, since Safari is responsible for playing the media, and we've created Safari, we can optimize video playback by decoding it in hardware to extend battery life, which is great for users.

The HTML5 media elements in Safari also include default controls that are customized to each platform. So you can see here on the desktop, we have default controls that appear when a user hovers over the video. They're pretty small, so that users can see plenty of the video, but they're still big enough to be clickable with a mouse pointer. On iPad, the default built-in controls are slightly bigger so that they're tappable with a finger.

Finally, you should use HTML5 audio and video elements because they're standard web technology. There's no new programming languages to learn, and they behave just like other web page elements in the sense that they can be styled with CSS and controlled with JavaScript. So when you combine HTML5 audio and video elements with CSS and JavaScript, what specifically is possible? Well, here are some of the things that you can do. First of all, you can create your own custom controls that match the look and feel of your web page. There's a JavaScript API that accompanies these elements, which you can use to play, pause, and query the state of your media.

This includes JavaScript to trigger full screen mode. You can also use standard DOM events to optimize the experience. Events are emitted when an error is detected and as progress is made, so you can do things like put up an error message or display a progress indicator to really optimize and provide feedback to users. And finally, as I mentioned, these elements behave just like other web page elements, so you can style them with CSS. You can easily add a drop shadow to your video, change the opacity, add a reflection to your video, mask the video with an image.

And with a lot of the CSS transforms that are available in CSS3, you could slide the video across the screen, scale it to different sizes, tilt it, and even flip it around in 3D space. space. Another great reason to use HTML5 media elements is that it's easy to get started. I'm going to talk more about HTML5 video later on, but right now I'd like to show you how easy it is to get started.

Here, with just a few simple lines of code, we've embedded video in a web page. We have an opening tag, which specifies the width and height. We've added the controls attribute, which tells Safari that we'd like to use the default built-in controls. And we have a source attribute that's pointing to our media. Between the opening and closing tags is our fallback content. This is the content that's displayed in browsers that don't implement the HTML5 video tag. In this case, our fallback content is a line of text, but it could really be anything: text, an image, even plugin content.

So this is looking pretty good. We have some video displaying in our web page, but the video on our page, the screen is black. That's because what we're seeing here is the very first frame of the video. And in this case, the first frame of this video is dark. The video content fades in after a second or so.

So even though we have our video showing in our web page, we want this default appearance to change slightly. To do that, we can add a poster attribute. So here we've added the poster attribute, and we're just pointing to an image file. And that displays this image instead of the first frame of the video. From here, we can add some CSS styling to our video. With just one line of CSS, we can add a drop shadow to our video. So now it's looking even better.

We can also use just a little bit of JavaScript to add a customized button. So here in my HTML, I've added an additional element, my play/pause button. And I'm specifying that when the user lifts their finger from the screen, we should toggle the state of this play/pause button.

In this toggle play/pause function, we're just doing a couple simple things. We're grabbing the video with the query selector function, and assigning it to a variable. And we're also grabbing that button so that we can change the look and feel of it. Next, we're just checking the state of the video to see if it's paused, and if it is, we're calling the play function on that video. We're also changing the class name on our button to change the look and feel.

And then if the video isn't paused, then we're calling the pause function on that video, and again, changing the look and feel of the button. And the result is going to look something like this. As you can see, we tap on the play button to begin playing the video, hit it again to pause the video, and as we tap to play and pause, the video responds accordingly.

So now let's move on to another key part of the HTML5 specification: offline storage. So when I talk about HTML5 offline storage, I'm referring to a collection of technologies that you can use to create web applications that will work even when users don't have an active internet connection.

So why should you add offline storage capabilities to your web application? Well, the first and most obvious reason is that users will always be able to access your web app. Secondly, if you add this functionality, your web app will have improved performance. If you're loading resources locally instead of over the network, your web application will load faster, and if you're loading data from a local data storage, your user interface will be more responsive.

There are a lot of flexible options available. There's key-value pair storage for simple string storage. And for structured data, there's a queryable SQL database storage API. Also, it's simple to get started. So let me show you what you can do with the HTML5 Offline Application Cache. This technology lets you store resources for your web application locally on the user's device. So all the HTML, CSS, and JavaScript files, all of your images, all of those can be stored locally right on your user's iPhone, iPod Touch, or iPad.

Your web app is going to always be accessible. Once a user visits it once, the resources are downloaded, and from that point forward, users can always access your web app just by revisiting that URL in Safari. So the first thing we're going to do is set the manifest attribute on the HTML tag to point to this simple text file. I've named it my-offline-cache.app-cache. Thank you.

So this file, my-offline-cache.appcache, is going to look something like this. It begins, the first line of this file is cache manifest, that's mandatory. Lines in this file that begin with a hash mark are comments, and those are ignored. And from there, you really just list the URLs of the resources that are required by your web application. So you can use absolute URLs, or URLs that are relative to the current page.

You can also use headers to define an online whitelist. So here, we're saying that we always want this forecast.cgi file to be fetched from the network, never the cache. And you can also use these headers to switch from the network back to the cache, like so. We always want this rainy.png image to be fetched from the cache. So once you have created your manifest file, you need to make one more change, and that is to serve the manifest file from your server with a text cache manifest mime type.

If you have used an offline application cache before on earlier versions of iOS, you should know that the cache size has increased to 50 megabytes in iOS 4.2, making this feature even more usable for more people. So now you've got all of the resources for your web application, all the HTML, CSS, and JavaScript files, all of your images. All of that is now cached locally on the user's device. So what about the data that is generated by your web application? People entering their username, saving the state of a game, etc.

Well, for storing dynamic data, there's two types of storage. The first is the Web Storage API. This is used for simple key-value pair storage for strings. There's two types of web storage. There's session storage, which is something that could replace cookies for session tracking, and there's one session per browser window or tab.

The second type of web storage is local storage. This is persistent, it's written to disk, and it's global to the domain. So storing items in web storage is really straightforward. It's just standard JavaScript. So you can store an item in web storage by calling a function. Here we're setting the item with the key of screen name and the value "Vicki Murley." You can index into an array or set a property to set items in web storage. Retrieving items from web storage is also really simple. Again, you can call a function. Here we're calling getItem to get our item that has the key screen name. You can index into an array or access a property.

So here we're setting and retrieving items from session storage, but the API for local storage is exactly the same. So that's a type of storage that you can use for storing simple strings of dynamic data. If you want to store structured data, you can use SQL database storage.

This API allows you to execute SQL transactions directly from JavaScript. So you can query and manipulate the data in your web application, or do things like add search to your web application. You start by calling the Open Database function with four parameters. The first is required, last three are optional. The first parameter is the name of our database. In this case, we have a to-do list example, so the name of our database is "to-do". That's followed by a version number, a display string, and a size in bytes.

From here, we can execute SQL transactions. We start by just building up a string of SQL, and then we pass that to the execute SQL function, and our SQL is executed. So we've learned about a few different technologies that you can use to create offline web applications. What's possible when you combine these technologies? Well, we have a few examples available at the developer.apple.com Safari demos website. The first is this offline calendar.

It uses an SQL database to store all of the events in the calendar, which allows us to create a filtered search, which searches through all of the events in the calendar, and returns matching events as the user types new characters into the search field. This calendar also uses an HTML5 offline application cache, so that it can be used to create a search engine for the application. This allows us to create a search engine for the application, so that it's fully functional even if a user doesn't have an active connection to the internet.

We also have a simple checkers application that demonstrates offline storage capabilities. This demo uses web storage, so simple key-value pairs of strings to store the state of the game and high scores. It also uses an HTML5 offline application cache, so that it's fully functional without an active network connection.

This application feels even more like a native application because it uses a home screen icon, and it also runs in full screen mode when it's launched from that home screen icon. So if you want to add offline storage capabilities to your web application, what tools are available to you to use? Well, as I mentioned, WebKit is the basis of Safari on all platforms. So you can actually use the Web Inspector, which is available in Safari on Mac OS X and Windows, to add offline functionality to your web application for iOS.

It has some really great features. You can inspect all of the values for local storage. And if you click on the name of a local database, you can even execute SQL transactions interactively. So that's it for HTML5. Now let's move on to talking about CSS, primarily CSS effects. So when I say CSS effects, I'm referring to a family of CSS properties that you can use to create engaging user interfaces and add visual styling and animation.

So why use CSS? Well, CSS will let you deliver rich animated interfaces on iOS. It's also simple to use. It's just standard web technology, so if you've ever written CSS, you know how to use CSS effects. Graceful degradation is built right in. A web browser will simply ignore any CSS property that it doesn't understand.

So you can add an animation and create an enhanced interface in Safari that will still be functional in other browsers. Finally, Performance Many of the CSS animations in Safari on iOS are hardware-accelerated. So even though JavaScript is really fast in Safari on iOS, a hardware-accelerated animation will always perform better.

So this is why CSS is so great. What can you do with CSS? Well, you can translate, rotate, scale, or skew any web page element in 2D or even 3D space. You can animate state changes that would otherwise be instantaneous, and we'll see an example of that in a moment. You can create advanced effects and rich animations with just a few lines of code. And you can embed custom fonts in web pages.

Prior to iOS 4.2, only SVG fonts were supported as custom fonts. But in iOS 4.2 and later, TrueType fonts and OpenType fonts are also supported. So speaking of fonts, let's take a look at a text example. So here we have a line of text, "Hello World," with some CSS properties that you've probably seen before, that you're used to seeing. We have a color, a font family, and a font size. This text looks pretty good, but we can take this text a step further by adding a text stroke around the text.

We do that by adding the text stroke property with a value of two pixels, and we want our stroke color to be white. This looks pretty cool, but we can make it even better by using a custom font for this text. To do that, I use the font at font face rule. I specify the name of my custom font. In this case, I'm naming it after the type of font that I found online. It's called "lobster," and I point the source value to the URL of my font file.

I also change my font family CSS property to use my custom font as the first font in the list. So this text is looking pretty cool, but we can take it even another step further by making the red color a red gradient. To do that, we're going to add a background image to this element using a WebKit gradient. This is a linear gradient that starts in the upper left-hand corner, and goes down to the lower left-hand corner, and the color interpolates from white to our red color.

So here you can see we have the gradient that's showing up kind of behind our text at the moment, but we want that gradient to be on the text. So what we do, we set the color of this text to transparent so that the gradient can show through, and then we clip the background to the text using the background clip property with the value of text.

So this text looks pretty good. Now let's say that we want to make this element appear on screen. So one way that we could do this is by setting the opacity value, changing it from 0 to 1 to make this text appear. So if we do that, change the value of the opacity property in JavaScript from 0 to 1, our text is just going to pop onto the screen.

But maybe what we want instead is for our text to kind of fade onto the screen gradually, over time. To do that, it's very easy to add this animation using CSS. Here we have our Hello class, which is a class on our text, and we're adding three CSS properties.

The transition property, which tells Safari which CSS property we want to animate, that's the opacity property. The transition duration property, we want this animation to last three seconds. Finally, a transition timing function. There are several built-in timing functions: easeIn, easeOut, easeInOut, and linear. Here we're specifying a timing function of easeIn.

[Transcript missing]

Well, the answer is no. The rule is actually that it's sort of one touch per container is required to start playback within that container. This means that calling play from an onload handler will not start media playback. This also means that autoplay does not work in Safari on iOS. However, after a user starts playback, subsequent calls to the play function will work for that container.

So what does this mean if I want to play two videos back to back, one after another? What steps will I follow? Well, the first thing you're going to do is add an event listener on your video for the ended event. Once that event is emitted, you should reset the source attribute for that video element to your second media source.

Once that's done, you can call the play function on that video element to start playing the second media file. And of course, you should remove the ended event listener. So what does this mean if I want to play two videos back to back, one after another? What steps will I follow? Well, the first thing you're going to do is add an event listener on your video for the ended event. Once that event is emitted, you should reset the source attribute for that video element to your second media source.

Once that's done, you can call the play function on that video element to start playing the second media file. And of course, you should remove the ended event listener from the video so that you don't repeat the process all over again when your second media file finishes playing. For bonus points, you should also display progress as this is happening. You can show a progress indicator until you receive a can play or can play through event.

And you should always listen for the error event to report problems to your users. For more information on HTML5 audio and video, again, I would encourage you to visit the Safari Dev Center at developer.apple.com. Two key pieces of documentation are the Safari HTML5 audio and video guide and the HTML media element class reference. And also, there are several pieces of sample code that will be helpful to you when you're implementing this technology. So that's it for some additional information on HTML5 video.

Let's move on to talk about what's new in Safari on iOS. So this new feature is extremely straightforward. Basically, you're just going to invoke the printing dialog by calling the print function. Doing so will bring up a platform-specific UI specific to either iPhone or iPad. On iPhone, a panel slides up so that users can choose their printer options. And on iPad, a popover appears aligned to the action button. Very straightforward. Just call the print function to automatically invoke AirPrint. Adding support for AirPlay on HTML5 video is also very simple.

All you have to do is set the AirPlay attribute on your HTML5 video to allow. So you can see that I'm calling it the AirPlay attribute, but then in my code snippet, you see xWebKitAirPlay = allow. You may be wondering what that prefix xWebKit actually means. This is following the W3C recommended syntax for a vendor-specific, HTML attribute.

So in this case, we have the X, the vendor is WebKit, and the feature is AirPlay. If you want to know more about this recommended syntax, you can visit this URL. Support for AirPlay is really easy to add, and it can be enabled in the default video controls. So now let's move on to talking about accelerometer and gyroscope.

What can you use them for? Well, one thing that you can use them for is playing games. You might imagine having a boxing game where you could tilt the phone right and left to throw punches, and maybe move the phone to the side to avoid a punch. You can also use the accelerometer and gyroscope to view a panoramic photo.

There's this great native application on iOS called 360 Panorama, and it allows you to create a panoramic photo and then view that photo from within a web page. So as you stand holding the phone and turn your body to the right and left, you can view this entire panoramic scene.

The accelerometer and gyroscope can also be used for interacting with data. So here we have a chart of age structure in Germany. And as I tilt the iPad away from me, the years go up and the data on the graph and in the chart adjusts accordingly. As I tilt the iPad towards me, the years go down, and again, the chart and the graph adjust. So how do I begin accessing accelerometer and gyroscope data in my web page in Safari on iOS? Well, this follows the W3C device orientation specification. And this information is exposed through two types of DOM events.

[Transcript missing]

Alpha is the rotation around gravity. Beta is the tilt forwards and backwards. And Gamma is the tilt left and right. Let's take a closer look at these orientation values. So if I have an iPhone laying flat on a table, the X-axis is going to go directly across the phone, Y-axis goes up and down, and the Z-axis goes straight through the phone, pointing up.

That means for the alpha value, that's degrees rotated around the z-axis. So this is going to report movement such as this. An iPhone kind of spinning around like a compass. The beta value reports the degrees rotated around the x-axis. So if we're looking at an iPhone from the side, the x-axis is now pointing towards you. So the beta value describes motion like this.

The gamma value is degrees rotated around the y-axis, so if we're looking down at an iPhone from the top or directly from the bottom, the gamma value is going to report motion like this. So that's device orientation. What about device motion? How is the device moving? Well, to get device motion information, again, it's pretty straightforward. You're going to add an EventListener to the window object. And then once that event is emitted, you can check one of three vectors on the event for motion.

Acceleration describes the acceleration force applied to the device with X, Y, and Z properties. Acceleration including gravity is like acceleration, but with the Earth's gravity included. And rotation rate describes the angular velocity around the vectors described with alpha, beta, and gamma properties. Acceleration is measured in units of g, which is units of Earth's gravity. So a body at rest has 0 g in the x direction, 0 g in the y direction, and 1 g in the z direction, up and down. So any deviation from an X, Y, Z value of 0, 0, 1 means that the device is moving.

When you're listening for the device motion event, you should know that the accelerometer will emit several events per second. So you're going to have to filter this data for best results. The best way to do this is to use a combination of high-pass and low-pass filters to create an average value of a range of events over time.

So remember, if you're trying to decide whether to use the accelerometer or gyroscope, remember that you should listen for the device orientation event to use the gyroscope wherever possible. The gyroscope is available on iPhone 4 and the latest iPod Touch. Listen for the device motion event to use the accelerometer whenever necessary. This is appropriate for older iPhones and older iPod Touch models, and also for iPad.

So how do you detect whether or not the gyroscope is available? Well, the gyroscope is available if acceleration and rotation rate on your device motion event are not null. Acceleration, including gravity, on the device motion event is guaranteed to work on all devices. For more information on the accelerometer and gyroscope, visit the Safari Dev Center, in particular the Device Orientation Event Class Reference and the Device Motion Event Class Reference.

So that covers new features in Safari on iOS. So that's it! I've covered an overview of the available technologies in Safari on iOS, provided more information on HTML5 video, and introduced new technologies which you can use to add additional features to your web application or website. With this information, you're ready to enhance and optimize your web content and deliver the best possible experience to Safari on iOS users.