Introduction

In this post, I’ll explain how to build Windows Azure Media Services (WAMS) async CORS enabled upload page. To be more specific, the page will:

1. Let the user select a video file to upload
2. Asynchronously upload a video file (split in chunks) directly to Azure Storage using AJAX calls to Azure REST API
3. When the file gets uploaded it will create and start a WAMS encoding job which will convert the original video into two new videos (using two different formats)
4. When the encoding finishes it will create public links for all three videos
5. Embed all three videos to the page

These steps will be further broken down into smaller, more detailed and precise steps which will explain the complete workflow of such a page and will hopefully give you a better understanding of everything that needs to be done to make it work. Since it would be a bit too much code to put into a blog post, I created a fully working demo app that implements the mentioned functionality which can be downloaded from github for you to follow along. The post will focus on the workflow and not on the actual implementation (code) itself. There will be plenty of new terms for you to get a grasp of, but bear with me, it will hopefully all be clear to you once you finished reading this post.

Why async cors enabled upload?

Since Microsoft recently enabled CORS on their REST API, we are now able to go around the server hosting our web application and upload the files directly into Azure Storage even if they’re on different domains. In my previous post about managing Azure CORS rules, I explained why this is good so I’ll just cite it here: “Imagine having a website hosted on your own server and allowing your users to upload large files to Azure Storage. Previously, you would need to upload and siphon all the data through your server and you would have to use Azure Storage SDK to do it. This means that processing time and bandwidth of your server are being wasted just for relaying data to its real destination." Well, we can now upload blobs directly from the web browser. Awesome.

Building the upload

1. Create an Azure Media Services Account

First we need to create an Azure Media Services Account. There is a nice post on azure website about how to do this. So go ahead, go to the Windows Azure website and create a new WAMS account (use an existing one or add a new Azure Storage account, whatever suits you best).

Once that’s done, select the newly created account and at the bottom of the page you’ll see a “Manage keys” link. Click on it and you’ll see a pop-up that looks like this:

It will hold your WAMS name and key which you’ll need to add to your application. You will need to get the same kind of name and key combination for the storage account as well. Basically, we’ll upload files to a storage account on behalf of our WAMS account. The two accounts are tied together very tightly. Here’s a quote from MSDN: “An Asset is a virtual entity that encompasses digital files (including video, audio, images, thumbnail collections, text tracks and closed caption files) and the metadata about them." When we upload a file through WAMS it will create a Media Service content item which will automatically create a new Azure Storage container associated to it which will hold our uploaded file (blob). These two items together will form an asset. Also - original video is one asset, a video encoded from the original one is a new asset and generated thumbnails would be yet another new asset.

2. Upload workflow

This brings us to the next point - when I say “upload through WAMS”, what I really mean by this is “upload through a Shared Access Signature (SAS)”. SAS is basically an URL which we will create for each file (a video in our case) and it will be used to make consecutive AJAX upload requests against the Azure REST API. Each SAS has its validity duration and a permission type like read, write, list and delete. These will all be defined in the backend portion of the code. (if you want to know more about SAS, i recommend you read this great post about Azure SAS).

Now that we know all this, it’s time to get down to business. The application I made is an ASP.NET MVC project which uses Windows Azure Media Services SDK. All the stuff I’ll talk about here is implemented in the application so you’ll be able to follow along easily. I also need to give a huge credit to Gaurav Mantri for his post about uploading large files to azure blob storage. Much of the core client-side code from the application is based on his code which I improved upon a bit, added some additional functionality and namespaced it.

One of the main improvements to his example is actually an enhancement he mentioned in his post. It’s on-demand SAS generation. This means we’ll be creating SAS on the server-side for every video file users try to upload. It will give you control over SAS creation (it’s duration and permission type) and it will keep your WAMS credentials safe since they will be stored on the server and regular users won’t be able to access them.

Here are the steps that need to be executed in our application for each file upload:

1. After the user gets to the web page and selects the file, make an AJAX request to our applications WebAPI service (the CreateAsset call), sending it selected files name
2. WAMSProvider Gets the CloudMediaContext using our WAMS account name and key credentials
3. WAMSProvider creates a new Asset for the selected file
4. WAMSProvider creates an AssetFile object for the selected file
5. WAMSProvider gets/creates an upload AccessPolicy which defines how long the SAS will be valid for (if you expect uploading of large files, set this to a few hours) and which which permissions will it give to the user (write in our case)
6. WAMSProvider creates a Locator (an Azure SDK object containing the actual REST API upload URI) based on created Asset, AssetFile and AccessPolicy
7. Return the SAS URI and the Asset ID to the client as an AJAX response to the request from step 1
8. Present the user with the upload button
9. On upload button press, split the file into multiple chunks using HTML5 splice() method, track them with blockIds and start sending AJAX PUT requests against Azure REST API using the SAS URI from step 7 (each AJAX PUT request sends both the chunk of blob data and its corresponding blockId to the Azure Storage)
10. When all the chunks are uploaded, create an XML object containing all the blockIds and make one final Azure Storage AJAX PUT request (the blockList commit) containing the created XML object 11.On successful blockList commit, make a second request to the WebAPI service of our application (to the PublishAsset call), containing the AssetId obtained in step 7
11. WAMSProvider revokes upload URI - after the upload is done, we don’t want anyone else to upload anything to our original asset
12. WAMSPRovider processes the video and publishes the newly encoded videos

This kind of async upload does not work with IE9 because it utilizes HTML5 FileReader, File and Blob API’s so please keep this in mind. I tried to implement a fallback option using various flash shims but I just couldn’t make it work. If you figure this one out, please drop a comment and let us know how this can be accomplished. :)

Enabling CORS

To upload files through an SAS URI, Cross-Origin Resource Sharing needs to be enabled over at Azure for your particular domain. If you don’t enable CORS, your upload PUT requests against Azure will be rejected. You can read a bit more about managing Azure CORS rules in my previous post. Normally you would have to set the rules up through your source-code, but since this is a one-time set-up, it makes no sense to keep this code in your project. That’s why we made a simple web app for managing Azure CORS rules which you can download over at github. It will make it easier for you to add/edit the rules.

These are the settings for the CORS rule which is needed for the upload to work:

• Allowed Origins: http://yourdomain.com, http://yourdevalias.local
• Allowed Methods: Put
• Allowed Headers: x-ms-*, content-type, accept
• Exposed Headers: x-ms-*
• Max Age In Seconds: 86400

You obviously need to use your own domain names / aliases as “Allowed Origins” and you could set a different “Max Age In Seconds” but the rest of the settings needs to be the same.

3. Encoding the original video and generating thumbnails

After the video has been fully uploaded, validated and the upload URI revoked, the original video can now be encoded into multiple new formats and thumbnails for it can be generated. At the moment of writing this post (19th of January, 2014.) it is not possible to create a custom encoding preset to be used to encode the video. This means that we can’t define our own custom video size, ratio, bitrate or coding standard. Instead one of the existing presets such as “H264 Broadband 720p” or “H264 Smooth Streaming 1080p” needs to be used.

To generate thumbnails on the other hand, one can define a point or multiple points in time for an asset for which thumbnails will be generated. The size of generated thumbnails can also be set. Thumbnail generation settings are saved within your project as an XML file which is then read through the application and used as a configuration parameter when creating a thumbnail generation task. This means you can create multiple thumbnail generation presets and create multiple thumbnail generation tasks using different settings. (Thumbnail generation is not covered in the demo app)

Now, to initiate the encoding process, an IJob entity needs to be created first. After that, encoding (and thumbnail generation) tasks are added to it and the job is submitted to Windows Azure Media Services for processing. After that, the only thing that can be done is to wait for the job to finish. If a larger video is uploaded, this may take quite a bit of time.

In the demo app this is covered in the ProcessVideo() method in WAMSProvider. There I added an execution progress Wait() task which will wait for the job to finish and continue with execution afterwards. This is OK for the demo app since it is simply a proof-of-concept. However this is not in any way a robust solution and it would most probably be a bad idea to use it in production since it would occupy a new thread for each submitted job and it would fail to continue from where it last was in case of some kind of failure (think of azure connectivity problems or the thread failing for whatever reason).

Checking WAMS jobs statuses

For production I would recommend you store all the important data such as assets and job ID’s and job statuses within a database. Since it is not really that important to publish a video the second it got encoded, you could create a web service call which you could then ping every 15 minutes or so to check upon the statuses of all the unfinished jobs. This can be done using a simple VB script such as this in combination with a scheduled Windows service:

Set oServerXML = CreateObject("Msxml2.ServerXMLHTTP")
oServerXML.setTimeouts 5000, 5000, 120000, 120000
oServerXML.Open "GET","http://YOUR_DOMAIN_HERE/PATH_TO_YOUR_CALL_HERE", False
oServerXML.setRequestHeader "Content-Type","application/x-www-form-urlencoded"
oServerXML.send ""
Set oServerXML = nothing

4. Publishing encoded videos (creating WAMS locators and obtaining public links)

After a job finishes and your web application is aware of it (either through waiting or checking) videos can be published / their public links which can be later embedded into HTML5 video tag can be obtained. To get a public link for accessing an asset blob, a WAMS locator needs to be created. Locators contain URI’s for accessing asset blobs. In case of the original video, setting its blob as a “primary file” for it’s corresponding asset will automatically create the locator. For all the encoded video files, locators need to be created manually using Locators.CreateLocator() Azure SDK method. Similarly to upload URI’s creation, each manually created locator requires an access policy. You can set it’s duration which defines for how long the public link will be accessible and you must set its access permission to at least “AccessPermissions.Read”. You can then further extract a full URI from a locator object using UrlBuilder and embed it in your website.

Final thoughts

There is quite a lot to learn and take care of while making an async WAMS CORS enabled upload page and I hope this post will aid you in doing that. There is a lot of information about WAMS out there but it’s mostly scattered around various blog posts and MSDN pages and it can sometimes be a bit tedious to bring it all together and just make something that works. That’s where this blog post and the demo app will hopefully give you a kickstart by providing you with a working code base which you can further build upon.

If there is anything unclear or if you have improvement suggestions for anything described here, please leave a comment and I’ll make sure to get back to you and to improve the demo app / blog post.