Distribution • iOS, macOS, tvOS • 48:54
Gain an understanding of the pros and cons of server- versus app-based receipt validation, and how to implement this critical functionality for your In-App Purchases. Get the latest news, and valuable tips for keeping subscribers and managing subscriptions. Learn about a major improvement to In-App Purchases: Server-to-Server Notifications. And finally, get detailed instructions for testing with the In-App Purchase sandbox.
Speaker: Pete Hare
Unlisted on Apple Developer site
Downloads from Apple
Transcript
This transcript has potential transcription errors. We are working on an improved version.
[ Clapping ]
My name's Pete Hare and I'm an engineer on the App Store team here at Apple. Now you're probably here because you're in one of a couple of different groups. Maybe you're here to find out a bit more about integrating in-app purchases into your application and you want to know a bit more about how to do it securely and reliably. Or maybe you're here to find out about subscriptions and you want to understand the process of maintaining a subscription state across multiple platforms in a server environment, that kind of thing.
Well whichever group you land in, in-app purchases really represents a new level of trust between you and your users. You see as soon a user is handing over money in exchange for digital content or services, they're really entrusting you with the responsibility of delivering that in a reliable and secure manner.
So today we're going to talk about some techniques do that. Firstly, we're going to discuss receipt validation in greater detail. We talked on it a little bit this morning in the what's new session but we're going to go into some details around how to do this on the user's device.
We're also going to talk about maintaining a subscription state particularly on the server and how to update multiple different platforms based on that subscription state. Finally, we're also going to touch on development in a sandbox environment. So how you can use these technologies when you're developing without actually having to use your own money.
So I'm going to put up this diagram. This is what we looked at in this morning's session. This was the in-app purchase flow. So this is what you need to implement in your application in order to actually sell in-app purchases. In this particular session, we're just going to really focus on these last three points though which I'm going to refer to as processing transactions.
So this is when StoreKit delivers a transaction to you once a user's made a payment and it's up to you to then process that transaction and deliver the content to the user who's paid for it. So I'm going to expand out these three steps now and also introduce an extra layer to this which is to add your server into the mix.
So this is how we can think of it for this particular talk. The process is going to start with us receiving a transaction. And this arrives by a StoreKit on the user's device. Then you make a choice as to whether you want to validate the receipt on the user's device or up in the server.
From there, you can proceed to inspect the contents of that receipt and unlock content or update a subscription state based on that contents. And finally the last step in the process, to finish the transaction back down on the user's device. Now you'll notice that the first and last parts of this flow occur on the device level only. We don't allow transactions to come in at the server level. Now this is really important. Even if you're using a service side flow for receipt validation, you still need to make sure that you receive and finish transactions on the user's device.
Now this morning, we also discussed the four different types of in-app purchases. If you're dealing with consumable products or non-consumable products, it's quite likely that you'll do a flow like this, which is the on device flow. This is where you can inspect the receipt and unlock content without sending anything up to your server or requiring a network request.
If you're dealing with subscriptions, auto renewable subscriptions in particular, it's quite likely that you'll have a situation like this where you want to keep state on the server side. And you need to have a server being able to update multiple devices from there. You can, of course, do both of these techniques if you want. But either way, the process still starts at that same point which is to receive transaction on the device. So let's dive now into each of these steps to see what they look like.
This is right at the start of my application life cycle. We have the did finish launching with options delegate method on the application and here we have to register a transaction observer for the SK payment queue. So you need the transaction observer to be registered as early as possible in the application life cycle so that you can start to receive the updated transactions that come through StoreKit.
Here I'm just adding the app delegate itself as my payment observer but you might add a separate controller object to do this for you. It's really up to you. What's important is that it's happening right at the start of the application life cycle but once that's registered, you're ready to start receiving transactions through the updated transactions call back.
So this is the updated transactions call back in the transaction observer. And you receive an array of transactions that you can iterate through. You can check the transaction state on each of these. And you want to look for a transaction in the purchase state. So this is a transaction that StoreKit deems appropriate for you to go ahead and validate. And then unlock content for. So it's telling you that the user has actually handed over money.
So once you've got that transaction in a purchase state, you're ready to go ahead with the next step in this diagram. So let's firstly look at how to do this validation of the receipt on the user's device. Now what is the receipt? We talked about it a little bit this morning but for those of you who weren't there, the receipt's really just like a document.
Like a receipt you'd get at any department store. It's a proof of purchase that really is an authentic document that indicates they've bought what they say they've bought. So we've have this application receipt which is a trusted record of the app and any in-app purchases that a user's made in your application.
This particular receipt's stored on the device. And it's issued by the App Store. It's put there by StoreKit. Now it's assigned an verifiable document which means you can use certificates to be able to check that this document was actually issued by Apple and put there on the user's device to make sure it's authentic.
Finally it's for your app on that device only which means it can't be shared across devices or amongst other applications. So when it comes receipt validation, this is the process of ensuring that this document is an authentic document and not some phony one that's someone put there. There's two ways to do this. You can do this on the device, which we're about to look at now and of course, you can do this by sending it up to your server using the server-to-server validation.
Now I just want to point out here that if you're on the user's device, it's really important that you don't use online validation directly from the user's device because this isn't a secure way of checking at all. If you're going to be doing it directly on the device, you need to use these checks we're about to do now. So what does the receipt look like? Let's take a look at the actual document.
The receipt document. We just take all that purchase information about your application including all the in-app purchases and we wrap that around certificates and signatures in order to create a document that you can check for authenticity. So it's a document that's stored in the application bundle and we provide an API for you to actually get that document.
It's a single file. And it contains all the purchase data about the application and also the in-app purchases that have occurred. And it also contains a signature for you to be able to check for authenticity. To make sure it's issued by Apple. The document is based on a series of industry standards. So it's actually signed using the public key cryptographic standard 7, cryptographic container.
It's encoded using an ASN.1 data encoding. And don't let the acronyms, the daunting acronyms fool you. These are actually very public open standards and there's a lot of information available online for you to be able to actually do these things. One prebuilt technique that you might recognize is actual OpenSSL. So OpenSSL is a framework that not only provides the functionality for secure web traffic tunneling, it also includes functions to be able to read in the data encoding from an ASN.1 payload and also check the signing on a cryptographic container like this.
Of course, you can use a prebuilt solution like OpenSSL or you could even roll your own, you know, to read in this particular data. It's really up to you and your business as to which solution you use. You see when it comes to security and this particular type of check, it's not a binary choice. It's not secure or not secure. You know, security's a scale. So you have to think about how far down that scale you want to go in order to verify these purchases.
But either way, it starts by reading in this encrypted data for this document. So you do that by using the API I mentioned. It's the AppStoreReceiptURL API on the bend bundle. That gives you an url that you can just past through to a data object and you can read in that encrypted binary data to a receipt object. So now you've got that encrypted binary data in memory ready to act on.
A couple of tips if you do want to use OpenSSL. We're not going to go through the whole process of using OpenSSL in this talk but OpenSSL doesn't actually ship with iOS. You have to build it and include it in your app yourself. If you are building it, remember to build as your own static library for your application and not a dynamic library.
If it was a dynamic library, it's much easier for someone to come along and switch out that dynamic library with a phony one than say, you know, messing with the actual methods inside your own application binary. So a static library will mean that that binary data's wrapped up with your own application and it's much harder for someone to come along and switch out your OpenSSL instance.
When it comes to the actual certificate check, you can download the Apple Root certificate authority's certificate from the Apple site. And you can use that certificate to actually perform that check using OpenSSL to see that it is a verified document from Apple. If you are bundling in the app, just one note here that be mindful about the expiry date on the actual certificate that you're including with a bundled application.
There is plenty of documentation online for this. In fact, we had a session here a couple of years where we built OpenSSL here on stage just in a live demo and did these checks here on stage. So it's a much simple process than you might think. So I'd encourage you to check out those previous sessions on receipt validation to see how that might be done.
When it comes to prebuilt solutions, I'm sure some of you in here have tried integrating in-app purchases and probably maybe go on Get Hub and found a solution that's prebuilt that can do a lot of these checks for you. Just remember when you're downloading prebuilt solutions that convenience comes at a price.
And so reusing code like this brings with it any bugs and vulnerabilities. And that's especially important when it comes to transactional APIs like StoreKit. Can you imagine if every jeweler around the country used the same lock on their safe? And then a single exploit was found on that one particular lock.
Suddenly every jeweler around the country has their jewels vulnerable to the same single exploit. So it's important for you when you're doing this to know and own the risks as you build it. And remember that while you're building these integrations with StoreKit, it's your revenue stream. And so people who are building these prebuilt solutions are not as inclined to be worried about your money as you are.
When you're verifying the receipt -- the actual certificate used to sign the receipt, a couple of tips here. You don't need to actually check the expiry date of the certificate used to sign the receipt. What do I mean by that? Well, if you think about a receipt that's encrypted at a point in time, let's say two years ago and then someone signs it with a certificate that's valid at the time. Then that certificate expires a little while later.
Just because the certificate's now expired and we're in the future, doesn't make the receipt actually any less valid. What's really important here is not whether the certificate's still valid right now, it's really only important as to whether it was valid at the time the receipt was made. So if you're going to compare that date to anything, compare it to a purchase date of a transaction inside the receipt to make sure it was valid at the time of signing.
Let's take look inside the actual payload of the receipt. So this is the ASN.1 encoding that I mentioned. All this is, is really, a series of types and values. Much like dictionary. You can just think about it as keys and values like a dictionary would have. And you could read out these particular values based on the different types.
So now that you've verified that the actual document is signed using the right certificate from Apple, you need to verify that the application used to make this receipt is the one the user's running on. So how do we do that? To verify that this receipt's for this application, there's two particular types of attributes. Type 2 and 3, and they contain the bundle identifier that this receipt's for and also the bundle version that this receipt's for.
So what you need to do is compare these two particular attributes to hard coded ones inside your application. It's important to use hard coded values here because of the same principle I mentioned earlier. If it's an info plist file, it's much easier for someone to switch out a plist file with phony values to match a phony receipt than it is to switch out something to match hard coded values.
But if you check these two things against hard coded values and they match then great. You verified that the application is correct. The next step is to verify the device that the user's on and verify that this document matches. We do that using types 4 and 5. It's a similar process but this time, the one that we want to check is attribute number 5.
Now attribute number 5 is actually a SHA-1 hash of these three values. It's a SHA-1 hash of the bundle ID, the device ID, we provide APIs for the device ID. And the third one is an opaque value, which is really just the attribute in type number 4. Now the reason we do that, it's a little bit of cryptographic entropy. A bit of secret salt that allows that SHA-1 hash to change over time, even the bundle ID and the device ID aren't changing. So it just makes this process a little more secure.
And so accordingly, this SHA-1 hash is unique to this app on the device. And what you do is you create that SHA-1 hash using your own, you know, hard coded values. The same technique that we did before and you compare it to the one in type number 5. And if they match, then that's it. You verified that the device the user's on matches the one the receipt is for.
So now you've done those three checks. That's the process of validating the receipt on the device. You now know that this is a document that you can trust when you're reading out further information from it. So let's take a look at the next step which is to actually update state and inspect the contents of these in-app purchases inside the receipt.
Let's dive back into what the payload of the receipt contains. The receipt contains a specific type, type 17 for every transaction that occurs for this user on this device. Now in each type 17, the actual payload is another ASN.1 encoded container. And inside of this, there's a bunch of types and values that are associated just for this specific transaction in question. So we have things like a quantity, a product identifier, a transaction ID. So these are values that you can use to verify that a transaction exists in the real world.
One more to call out while I'm here is type 1708. This is particularly important if you're dealing with auto renewable subscriptions. This contains the expiry date for a particular transaction for a particular billing period. And we'll come back to talking about subscriptions in a bit. If you want to know more information about all the different types that are included here, I'd just encourage you to check out the Receipt Validation programming guide online. And we go through all the different types that are actually included in the receipt.
But now that you can read all these transactions, it's up to you to use those to verify the content that StoreKit's telling you the user's bought. So what you do is you take the transaction that's appeared through this updated transaction's call at the beginning of the process and all you have to do is compare all those values to the ones inside the receipt.
So you can use things like the transaction ID, the purchase date, the product identifier that it's saying the user bought, and if you can verify that there's a transaction that matches then great. You've got a proof -- a document proving that the user actually purchased it and money's actually changed hands. So you can trust the transaction that StoreKit's telling you.
When you're dealing with subscriptions a common question at this point is does my user have an active subscription. Well one thing just to note here. Just remember that a valid receipt is not the same thing as having a subscribed user. So there's a bit of confusion sometimes. If you can validate the receipt that doesn't mean that the user's actually paid anything.
You know every app has a receipt. It contains information about original application purchases even for free apps. It's the data inside the receipt. These transactions that's going to tell you about the subscription state of the user. Now how do we find out that subscription state? You could take those transactions and you want to group them together by the original transaction ID field. This field just contains that first transaction ID for a particular auto renewable subscription that a user used. You can kind of think of this as being like a subscription ID that you can use to reference and group together these transactions.
So you can grab those transactions and you want to look for the one that has the latest expiry date. Now this is an indication of the latest transaction that's taken place. So if you see an expiry date in the future, that's an indication that the user's in the middle of a billing period and they have an active subscription. Now if you find an expiry date in the past, that's an indication that there's been no transaction since then. So the user's subscription's actually lapsed.
Now if you do find this, you can do what we say -- what we call a receive refresh request. And this'll go fetch the latest copy of the receipt just as a double check and you can repeat those steps of receipt validation and check these steps that we just called above to see if any new transactions have appeared.
Now there is one caveat when you're maintaining a subscription state on the device like this. We talked about, you know, expiry dates. And we've talked about purchase dates. Well if you're doing this purely on the device, the only data you actually have to compare these to is the user system data.
So what's stopping the user from just winding their clock back and putting themselves into an active subscription period? Not a lot, unfortunately. So if this is a problem for you, it's probably likely that you're going to need to look at some kind of service side solution. Maybe look at receipt validation on your server or at least, some check to actually get timed -- time and date from your server to check it against.
When it comes to actually refreshing the receipt like I just mentioned, you can do this if the receipt doesn't exist or it's invalid or maybe you're searching for that extra transaction. It will require a network request because it goes and fetches a new receipt from the App Store.
And it will require sign-in from the user which means that you should be very careful about how often you do this and avoid continuous loops of validating and refreshing. If you're doing that example that we just spoke about where you're looking for that expiry date, make sure that if you don't find it that you don't just keep refreshing the receipt refresh request over and over again because it'll keep prompting the user to log in. So just issue one of these requests if you're going to do it and this is what it looks like in code. You create an SKReceiptRefreshRequest object. You set a delegate on it and you just kick it off using the start method.
On macOS if you're developing for the Mac, same principles apply. You can do this if the receipt's invalid. It'll require a network request and it will prompt the user to actually log in as well but in this case, it's a little different. The API what you do is you exit your application using the code 173. And that looks like this. So this'll exit your application.
This will trigger StoreKit in the background to go and download a new receipt to the Mac and then it'll prompt the user to log in and launch your app again. Now at this point I just want to touch on a couple of differences here between restoring transactions and refreshing the receipt. These can be sometimes be confused. These are two separate APIs.
So restoring completed transactions which is an API we looked at in the What's New in StoreKit talk this morning, this is an API on the SK payment queue whereas the receipt refresh request is its own instance that you create and you kick it off using the start method. And they accomplish slightly different things.
So restoring completed transactions causes all the completed transactions that have occurred for a user to appear back on that updated transactions call back for you to be able to process. Whereas the receipt refresh request is really just used to go and fetch that new receipt document. That encrypted binary code for you to be able to check the contents of.
And there's also a slight difference to actually what they include. So when you're restoring completed transactions, this restores only non-consumerable products and auto renewable subscription products. Whereas a receipt refresh request has both of those but it also includes also any non-renewing subscription entries in the receipt. So you'll notice that consumable products are absent from both of these types of requests.
If you're dealing with consumable product purchases, they're just going to appear both in the updated transactions and on the receipt at the time of purchase. So you kind of have that one chance to actually verify the consumable product and it won't be restored for either of these calls. Now one other tip for dealing with receipts is if you're looking to switch to subscriptions, maybe you've got a paid application and you want to switch it to being a subscription model, you can use this type 19 value in the application receipt.
This contains the original application version. So you can use this application version that a user originally downloaded as kind of a gate to know as to whether you need to provide a content based on a paid app or based on a subscription. You know it's not a great experience if you've paid for an application and then suddenly you lose access to that functionality you paid for if it's now a subscription model. So use type 19 as a bit of a gate to be able to supply that.
So once you've done this step, that's the process of checking the transaction and confirming subscription state on the device. You can then go ahead to finish the transaction. So you've made the content available to the user. You've updated that subscription state. When it comes to finishing the transaction, you have to remember to finish all transactions that come through this flow but only do it once you've made content available to the user.
So maybe you're downloading content associated with an in-app purchase. Make sure until that download's completely finished before you go ahead and finish the transaction. And this includes all auto renewable subscription transactions. So these renewable transactions that come in at the end of each billing period, you still need to finish all of these transactions and handle them. And if you don't, the payment actually stays on the payment queue and it'll keep reappearing in the updated transactions call back until you deal with it and call finish transactions.
We also have specific logic around subscription billing retry. So if you do have auto renewable subscriptions, it's important that you do finish these transactions so that our subscription billing retry logic can continue to try and charge user's credit cards if there's any kind of billing error along the way.
So this is quite important for our end to know the state of all these transactions. This is what the API though looks like. It's just one line of code. You can pass in that transaction object that we received at the start of the process to the finish transaction method on the SKPaymentQueue's default queue.
So that wraps up that device flow of validating the receipt and updating content on the user's device. Let's jump up now and look at how this works in a service side environment. And to do this, let's just walk through a bit of an example. Let's say I have a user here and it's using your application and you've got a server there powering your back end. The process here starts, of course, with receiving that transaction and the update transactions call back that we saw.
From there, you could read in that binary receipt data using the API mentioned. At this point it's still encoded so we haven't done all these checks yet with the certificate or anything. Instead of doing it on the device, what we can do is take that binary encoded receipt data and send it up to your server.
From here, you can just establish connection over to the App Store server by this verify receipt url. And you pass that binary data over to the App Store. Now the App Store does all that hard work of checking the certificates and verifying all this information. And it responds with a receipt validity status as to whether or not this is a valid document that you can trust.
So this is it again really in textual form, the response is in JSON. And it returns that status as to whether a receipt is valid or not but one point to note here, as I mentioned before, you shouldn't ever use this technique directly from the user's device. This is really only secure when you're doing it from your server to the App Store.
And that's the whole step of validating a receipt on your server. It's a little simpler than on the user's device. Now let's take a look at how you unlock content and inspect transactions based on this scenario. Let's look at this example again. So let's say that you've sent that binary data up to your server. You establish that connection to the App Store and you send over the binary data to the App Store.
Now not only does the App Store actually respond with the validity of the receipt, the response here actually includes a decoded version of the latest application receipt as well. So this is a decoded version in JSON that you can inspect and look at all those transactions. It's just the same consent that you see on the user's device when you decrypt it on the device but this time it's a JSON payload from the App Store and you can just inspect all those transactions. Make decisions about whether to unlock the content and then go ahead and finish that transaction back down on the user's device again.
This is particularly useful because you can then do things like updating state across other platforms that you might have associated with your server. So this is what that process looks like, again, just in text but the important point here to call out is that you have to remember to tell the device to still finish the transaction if you're using this technique.
So let's answer this question again of unlocking subscription features. So does my user have an active subscription. It's exactly the same flow, you know, here because we said that this receipt contains all the same information. So you can group the transactions that come back in this receipt by the original transaction ID. That's that subscription ID field. And all you need to do is find the transaction that has the latest expiry date. And if there's an expiry date in the future that's an indication that the user is in an active subscription.
If there's not an expiry date in the future, if it's in the past somewhere well that means that the subscription's unfortunately lapsed. And this is the latest copy of the receipt. So there's no way that you can really do a receipt refresh request from the server. You've just got the latest copy already.
So now we've unlocked content on the server based on this information. As I mentioned, you still need to remember to finish the transaction back down on the user's device again. You'll hear me say finish transactions a lot in this talk. It's a really important point that I want to drive home today.
So let's look specifically at subscriptions for a little while. These scenarios have applied to all in-app purchase types. Now let's really talk about maintaining subscription state and particularly using that service side flow. So in this example, again, we have an updated transactions call being given to the user's device. And the user can read in that binary receipt data into memory on the user's device. They send that information up to your server.
Now this time, we're going to do a little bit of a different technique. This is going to be a bit more of a real world scenario here. We're going to hold onto a copy of the binary receipt data on your server here and at the same time, we'll send a copy of that data over to the App Store to achieve the technique that we saw before.
So this will respond with the latest copy of the receipt and we can do that same technique of finishing a transaction and updating content across devices. Now in this example, we're dealing with an auto renewable subscription. So let's say that the user goes offline for a bit and stops using your app for a few days which is a shocking thought, I know.
But it can happen and if it does happen and then the user happens to have their subscription renewed in the background during this process. The credit card's charged. So a new transaction's taken place somewhere. And you don't know about it yet. So the user jumps on, I don't know your website.
And at this point, your server doesn't have any new information about the transaction that's taken place in the background. So in order for you to know this information from your service point of view, remember we're holding on to that binary receipt data on the server. You can treat that data just like a token. And you can actually send it back over to the App Store here by that same request.
And as I mentioned before, not only does this include the decoded receipt data, this is actually the latest copy of the application receipt. So this is latest copy is going to contain any new transactions that have occurred in the background. So you can find out about that transaction that's occurred and accordingly, give the user access to your website again.
From there, you might want to unlock content across multiple devices. But you do have to remember that when the user does pick up that original device again and come back online, that will still receive the transaction through the updated transactions call. And so you still need to handle this and all the way through to finishing the transaction. So what we'd suggest is maybe treat this as an opportunity to update that binary receipt data up on your server. You're probably associating that with a user's account for your particular application.
And then remember to finish the transaction back down on the device again. So even though your server already knew about this transaction, it's still important to complete this flow and finish that transaction based on what I said before. So for this technique, what we're really doing here is we're treating that receipt data much like a token. And we're using it to perform multiple requests by storing it on your server. It's the same binary data that can be used over and over again.
And it's quite useful for propagating subscription state across multiple devices and platforms but still remember that you have to process all the updated transaction calls and that means all the renewal transactions that come through for each billing period all the way through to finishing the transaction. Now, as you can imagine, over time if you're dealing with an auto renewable subscription particularly if it has a short billing period, this transaction receipt can grow quite large, you know? Every transaction that occurs for a subscription appears on the receipt.
So this document can grow quite large over time. And we've heard feedback that a lot of you only real care about the latest transaction. You know I keep on saying you check for the latest expiry date. Well a lot of you really only care about that particular transaction. So we're enhancing this endpoint today with a new query parameter that you can include which is to exclude old transactions from this endpoint.
If you set this to true, the verify receipt endpoint's just going to respond with the latest transaction for each subscription. So that's going to drastically reduce the payload of that request that's coming back from the verified receipt endpoint. And not only just save web traffic but it's also just going to mean saved processing time on your server because you don't have as many transactions to be looping through.
Now this doesn't sound like a lot but when you're dealing with thousands or millions of users like a lot of you do, this can actually save quite a lot of time. So we think this is going to be a great enhancement for those of you're dealing with this particular scenario. Now this technique of status polling --
[ Clapping ]
Thanks. This technique of status polling really fits as a bit of a server side tool. So you can think about it as sitting up here in this particular diagram but you'll notice I'm leaving that particular flow of updating and finishing transactions because it's still important that you handle all of these transactions as you come through in a user's device. So let's stick on the subject of subscriptions for a little bit.
Now we opened up auto renewable subscriptions to many more categories last year. And we've seen great uptake and heard great feedback from those of you who have implemented them. And we've tried to offer information about your users and the behavior of them via the way of things like iTunes Connect reports. You can find about how many expirys you had and some of the reasoning behind that but there's been a lot of questions that you've had that you haven't been able to answer until now.
Particularly about individual users. So what are these questions? Things like why did a particular user's subscription expire? Or will this user's subscription be renewed as the end of this billing period? Will this user be downgraded at the end of this billing period? Have they elected to change that subscription that they're subscribed to? Maybe they asked for a refund from AppleCare? What was the reason behind it? Did they have a problem or is just something that they wanted to do? Have they agreed to a price increase that I've put in place? Or will they just end their subscription at the end of this billing period? How can I know this ahead of time? Or just simply what kind of messaging do I need to tell my user about their subscription? How can I communicate to them effectively and you know, with things they need to know? Now why are all these questions really important? Obviously to provide a great experience for the user but can you think about why these questions are really important? Well, it's because all these questions are centered around the user's renewal of their subscription and so providing a seamless experience in these particularly scenarios, it's really paramount to reducing what we call subscription churn. Subscription churn is just a fancy way of saying losing subscribers. And the thing is when you lose subscribers that's immediately lost revenue for your business. And not only is it lost revenue for your business it's a lost acquisition cost for each one of those users as well.
Now we can think about subscription churn as two particular buckets. We can talk about the involuntary churn, which is the case that the user hasn't even elected to unsubscribe. Maybe their credit card just expired and it was an involuntary action. Now this is far too many users fall into this particular category. There's also voluntary subscription churn. In the case that a user elected to unsubscribe from your application. Maybe they asked for a refund from AppleCare or they turned off auto renew inside the settings.
So we really want to empower you with new tools to be able to address both of these buckets and we think we've got some great new ways to do that today. So we're announcing some new tools to help reduce this subscription churn. Now to go through them, let's walk through an example, again, just to illustrate it.
So let's say a user here is subscribed to your subscription service. It's a video subscription service for this example. You're using these techniques with status polling like we said before to poll the App Store to find out the latest subscription information about a user and they're actively subscribed. Everyone's happy. Then the user goes offline for a few days.
And during this particular time, their subscription's renewed again. This is a similar example to before, but, this time let's say that the user's credit card's expired. And the App Store wasn't able to charge them. So there was some kind of billing error that's happened and the user comes along and jumps on your website in order to keep on watching videos.
Now at this point your server does that status polling technique that we talked about but the App Store's going to inform it that there's been no transaction that's occurred because there was a billing error, right? So your server does the only thing that it can. It informs the website that subscription's actually expired and the user, the poor user's done nothing wrong. Their credit card just expired.
They go over and begrudgingly pull out their credit card. They go and update their credit card details in the App Store. The App Store based on the new credit card details is successfully able to actually charge that card. So the user's gone to this effort, everything looks good. They jump back on your website and then bah. This happens.
Now at this point, your server had no knowledge of the actual transaction that's occurred. The fact that that credit card was able to be charged again and so if this kind of thing happens to you as a user, go to all that effort. You update your credit card details and then you even see money being taken off your credit card and then you don't get an immediate update of your subscription state, you probably just unsubscribe or you ask for a refund straight away. Now this happens far too frequently. So to address that today we're introducing new server-to-server notifications from the App Store.
[ Clapping ]
So let's look at how that scenario plays out now. The user receives this information that's something happened. There's been a billing error. They go over and they update their credit card details. Now this time with the new notifications, as soon as the App Store is able to charge that credit card it sends a notification over to your server with the latest transaction that's occurred. And you can use the payload of that transaction to unblock the user immediately and give them access to using that website and you've got happy users. That's great. So this is going to be awesome for those of you who are dealing with subscriptions on the server side.
This is what it looks like in practice. You've got a status url in iTunes Connect that you can enter. That you can put url in for your own server. Your server does have to adhere the app transport security requirements but if it does, all it is an http post that gets sent to your server for key status changes for subscriptions. Now what are these events that we send them for. It happens for any initial purchase of a subscription.
If there's any subscription cancellations by AppleCare. So if a user gets a refund, you get notified of that. It happens for any subscription downgrades. So any time a user elects to downgrade their subscription, you can find out about that and update their subscription state on the server accordingly.
And also that example that we just saw, when there's a successful renewal or a re-purchase for an expired subscription so that you can unblock those users immediately and give them access to their subscription. Now the payload of the notification includes the latest transaction for the actual transaction in question that's taken place that you've missed out on.
So when you're doing this it means that you don't need to polling the verify receipt nearly as often as you used to. You still might need to call verify receipt using that polling but you can be a bit smarter about when you do it. Maybe an App Store notification wasn't able to reach your server.
So you still need to maybe use this to actually retrieve and poll information about the subscription state but as I said, you can a bit smarter about when you do it. Maybe you want to do it the day before or the day of a user's expiry instead of having to do it every five minutes. So these are coming later this year and we think it's going to drastically improve and reduce that subscription churn that you might be seeing.
So let's bring up these questions again that I asked just before. The new notifications is going to do a great job of reducing subscription churn. You know via that technique that we just saw but there's a lot of information here that we still don't have answers for particularly about users before they've expired. So in order to give you access to this specific information, today we're announcing new fields in the verify receipt endpoint.
Now these new fields are going to provide you with the information specifically about users along these key subscription events. So if they turn off auto renew in the settings, you can now know this ahead of time that they've elected to not be subscribed at the end of this billing period or maybe they cancel a receipt or refund at the end and they won't have their subscription continuing. So you can act on this information now ahead of time and make key business decisions about this information. So let's look at the new fields that we're including.
So this is that same request, the verify receipt endpoint. We're including now an auto renew status. So when a user elects to turn off auto renew for a subscription, you can find this out now ahead of time before they've actually expired their subscription. We're including an auto renew preference. So if a user elects to downgrade or change their subscription preference, you can now know how they're going to have -- what's going to happen at the end of that particular billing period.
We're including a price consent status. So if you're rolling a price increase to a bunch of users, now you can find out that I don't know let's look at an example. Maybe 80% of your users haven't agreed to a price increase and they're actually going to be unsubscribed at the end of the billing period. Previously you didn't know how many users were going to be rolled off. So you can make key business decisions based off that now. Maybe you'll decide to not go ahead with the price increase based on the fact that you're going to lose too many subscribers.
We're including a subscription billing retry flag. So if a user falls into the category of having some kind of billing error like we saw before and App Store is trying to recharge their credit card to get a successful transaction, you can see if a user's in that window. And we'll see a couple of examples about how to use these in just a moment. An expiration intent. So why a user actually expired. Was it because of a billing error or some other reason? You can now find this information out about a user.
We're also including an cancellation reason. So previously if a user got a refund from AppleCare, you were going to kind of blind as to why that was the case. Now you can know if the user had a problem with your app or an in-app purchase or if it was some personal reason the user had in order to get a refund.
So let's look at a couple of examples about how you can actually use these fields in order to address the subscription turn that we talked about. Firstly, let's address involuntary expiration. So this is the case that a user hasn't even elected to unsubscribe. So we think there's a lot we can do to address involuntary expiration. And so we're going to do what we can on our end to address this. And that includes expanding that retry window for billing retry to up to 60 days now.
So previously, we would only try for a period of a few hours to recharge a credit card if there was any kind of issues. Now we're expanding that right out to be up to 60 days. We think that's going to catch a lot of unsubscribes that occur. On your end, there's a couple of things that you can do to address these scenarios.
You can use that expiration intent field and the subscription retry flag to do a few key things. Now if there's anything that you can get from this talk, let it be these three points because if you can go ahead and implement these things that I'm about to outline, you'll drastically reduce your involuntary subscription turn. So firstly, you can use these two fields to provide messaging to your user. If you see that they've had a billing error, you can now tell them to go ahead and update their billing info in the app store.
Number 2, you can offer a downgraded or temporary experience to users who are in this retry window. So if you see that they're in billing retry window, maybe you give them access to browse your catalog or videos but you don't let them watch them. So give them some kind of temporary experience in the middle there.
And number three, use these new server notifications. Use the verify receipt endpoint or any other technique to unblock users as soon as their subscription renews. Now these sound like pretty obvious examples but if you can do these three things, there's a lot of involuntary expiration that you will save because of these three things.
So there's some things you can do for the voluntary case as well. So when a user has actually voluntarily elected to unsubscribe from your app, you can use this information to help you here as well. There's that expiration intent field that we're including now and you can use this as a way to offer messaging to those particular users. So let's say that a user canceled. You can maybe think well, let's apply some kind of win back.
Maybe you rolled out a price increase and the user expired because they didn't consent to it. Well now you can offer maybe an attractive downgrade option because you know that it was really about the price difference that caused the user to unsubscribe. So you can see how you can use just in these couple of examples, these fields to now, you know, make key business decisions and how you interact with your users and save subscription churn. These new fields are arriving later this year along with the server notifications and we think this is going to give a much better experience for all users for your subscriptions and for you developing it as well.
[ Clapping ]
They fit in this diagram as another set of tools that you can use for server side subscription management alongside the status polling that we talked about. But of course, you still need to be relying on this flow of updating and finishing transactions on a user's device. So while we're talking about subscriptions, I just want to touch on free trials for a moment.
Free trial is when a user can begin a subscription without actually paying for anything. And they are not billed until the free trial period is actually over. So previously, we had this pretty convoluted looking table and you'd have an associated free trial duration for a subscription length. And you had to choose which trials you could offer based on the subscription length that you were offering users.
Now we made some changes recently to make this much simpler. So you can now have any subscription length available for any free trial duration. So we think is a much better improvement and this includes two new free trial durations as well, which is the three-day trial and the two-week trial. These are two new trial durations that you can offer to your users.
So that's a bit about maintaining subscription state and talking about managing auto renewable subscriptions in the server. I'd like to touch now on developing in the sandbox environment. So how you can use these technologies while you're developing. So what is the sandbox? Well it's not to be confused with the sandbox on your application, on a user's device that guards access to application resources. This particular sandbox is the test environment that we offer for you to test out in-app purchases in.
And we select it based on the certificate that's actually used to sign your application. It's how StoreKit knows whether to be in a sandbox mode or not. So if you build and run your app from Xcode [phonetic]. It's signed using your developer certificate and StoreKit knows that it needs to be talking to the sandbox environment. Of course, if you distribute it using the App Store, well StoreKit knows that it needs to talk to the production environment based on the certificate used to sign your app.
And how do you know when you're actually working with app which environment you're in? When you make your in-app purchase inside your application, you get this payment sheet that we saw in this morning's session. And we have this indication here that you're in sandbox mode. So this is going to inform you that the payment that's about to take place is actually a sandbox payment and no real money is going to be changing hands.
So the key differences between the sandbox and the production environment, well the biggest one, of course, is that there's no real money being changed hands here. There's no actual charge happening. So you don't need to be charging your personal card 10-cent transactions all the time. There's a different endpoint as well when it comes to server-to-server validation. We provide a different url for that verify receipt endpoint. And you can also request expired and revoked certificates in the sandbox environment to be able to handle them a little differently.
The other thing that is different is a time contraction for auto renewable subscriptions. So instead of having to wait a full year for a subscription to renew to be able to test out that scenario, we can track these time intervals. So the rule of thumb is that one hour of time in the sandbox world equals one year of time in the real world. And this is how those relate for the various different subscription durations.
The other thing that we do when you're operating in a sandbox environment is that if you are subscribed to an auto renewable subscription, we only renew it after six times per eight-hour window. Then we let it expire. So you can handle the case where a user has actually let their subscription lapse and you can handle that in your application accordingly.
When it comes to setting up this test environment, it's done through iTunes Connect. You create test users in iTunes Connect and you can just those in-app purchase products that you've already got for sale and then you just build and sign your app using XCode. So it's built and signed using the developer certificate. And you just go ahead and buy products in your application. And when you're prompted to sign in, you'll be able to sign in using that test user that you made in iTunes Connect.
One note if you are developing for the Mac, you may need to launch your app once from Finder just in order to make sure that the receipt gets fetched. This is because that exit code that we mentioned earlier on in the talk where you exit 173 on a Mac, well XCode catches that if you're building and running straight from XCode the first time. You need to make sure that you launch the app once that binary from Finder so that StoreKit can catch that exit code and handle it appropriately.
In terms of using the sandbox from the server, I mentioned that we have a different endpoint for the verify receipt url. This is what it looks like in the development environment. You'll probably have your developer signed app probably talking to your test server and that test server can talk to the App Store sandbox by the different url. In production, you'll have something like this. Your production app will talk to your production server and that can talk to the production App Store server but there is one particularly case where there's a bit of a mismatch.
And that's when your app is in app review. And why is it a mismatch? It's because the app review team actually uses a sandbox user to be able to test your in-app purchases. So this is what that looks like. You have your production signed app talking to your production server but you need to be able to verify transactions for a sandbox user just to get through the app review scenario. So how do you handle this particular mismatch? Well we have a way of doing this.
Firstly, when you're in the production environment try the production App Store url first. Now if the receipt is for the sandbox, you'll receive a specific error code, 21 double 0 7. That's an indication that you need to then to try against the App Store sandbox instead. So you can leave that in your production environment and this will just mean that your app is able to sail through app review without any problems in this regard.
When it comes to the new server to server notifications, this is handled slightly differently. So we don't actually have a separate url in iTunes Connect for a sandbox -- or for your own test server. We handle this sandboxing by parameter in the actual payload of the notification. So there's an environment key that you can use that'll tell you whether or not the notification is for a sandbox subscription or whether it's for a production one.
So that's a bit about developing now in a sandbox. So we touched on a few things today. There's been a lot of information. Let's just go through again what we talked about. We talked about receipt validation in detail. How to do this on a user's device and how to check that document for authenticity and read out transactions from it.
We talked about maintaining a subscription state across a server environment. How to update devices and act on that verify receipt endpoint. We have new notifications that we're introducing that's going to unblock and save a lot of that involuntary subscription churn. Provide a much better experience for your users.
We're also introducing those new receipt fields. So you can make key decisions based on business information about your users and now message them much more directly. Remember those three easy steps to retain subscribers. Three steps to success. So if you do these three things, this will, I kid you not, it'll save you a ton of involuntary subscription churns.
So please go ahead and do these when these fields come out later this year. And finally we talked about sandbox development. Being able to use these technologies in the sandbox environment. So for more information on this session and to see the slides and follow any links, this is session 305. You can check it out on the developer website.
We did have some related sessions earlier this week. We're kind of at the end of the week now. But we are still here in the labs all afternoon this afternoon. So we have engineers from both the client and from the service side from all across the organization down in the StoreKit labs, both this afternoon and tomorrow afternoon. So please come and say hi and bombard us with questions. And we're happy to help you out and talk about how you're architecting your solutions. But until then, thanks for coming today. Thanks for your time. Enjoy your afternoon.
[ Clapping ]