When you deploy your web role to Azure (or the local development fabric), it will be assigned a url which is different from the one assigned by VS.NET, and which usually has a port of 81.  This can cause problems in your Silverlight application if you call web services within the web role project.  This is because VS.NET assigned a port to your web services (such as 43039).  When deploying to Azure (or again the local development fabric), the app has a new (localhost:81 on the local dev fabric, and yet a different one in Azure), but the old ports (and a reference to localhost) remains in the proxy configuration(s).

So how can you solve this?  The best scenario that I have found is to reconfigure your proxy URLs in the start up of the silverlight application.  As an example:



What I'm doing here is calling this method during the startup of the Silverlight application object.  It uses the HtmlPage.Document.DocumentUri object to determine the url to the application, and truncates it to contain just the address and port of the page that the application is served from the web server.  The last two lines then create two proxy objects that I use in this app, and pass a new enpoint address object which is the base URI determined before appended with the asmx filename of the service.  Note that you'll also need to pass in a binding object too; I change some parameters on mine, but using just a default BasicHttpBinding object should do fine (and it has to be a basic HTTP binding object as that is all that is supported by Silverlight).

This works for all deployments of the Silverlight application, either running locally in VS.NET, the local development fabric, or in Azure.

I pushed my first set of F# code to Azure today.  Upon first run, I got an error that FSharp.Core could not be found.  I was pretty sure it was in the package that was deployed, so I set about a bit investigation.  The most appropriate link I found was the following:

http://code.msdn.microsoft.com/fsharpazure

Basically, you can run F# code in azure, but because F# libraries are stored locally in the GAC (and hence referenced in the GAC), and that since in Azure the the GAC F# assemblies to not allow partial trust the F# libraries are not found by default?

The solution?  It's in that article to a point.  My F# code was not an actual worker_role piece of code, so that template did not assist me.  My worker was a C# worker that was attempting to call an F# library.  Since I had the code, I put the --standalone value in the other flags section of the project properties (see the picture), recompiled, linked the new dll into the C# app, deployed it, and it ran fine!

Yesterday I was deploying a Silverlight application into the local azure fabric and was getting this error 2103 when the app started stating that I had an invalid xap.  After a bunch of searches that didn't get me the answer I needed, I found this post:

http://social.msdn.microsoft.com/Forums/en-US/windowsazure/thread/722cd6ab-3c6a-40db-913c-e406c7596289/

Basically, I was on Windows 7 build 7057 x64, and did not have the static content option enabled when IIS installed.  I turned that on and things all started to work just fine.

The two primary cloud service providers available at this time are Amazon and Microsoft.

Amazon Web Services (AWS)

• Elastic Compute Cloud (EC2)

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.

• SimpleDB

Amazon SimpleDB is a web service providing the core database functions of data indexing and querying. This service works in close conjunction with Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), collectively providing the ability to store, process and query data sets in the cloud, making web-scale computing easier and more cost-effective for developers.

• Simple Storage Service (S3)

Amazon S3 is storage for the Internet. It is designed to make web-scale computing easier for developers.

• CloudFront

Amazon CloudFront is a web service for content delivery. It integrates with other Amazon Web Services to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no commitments.

• Simple Queue Service (SQS)

Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers. By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available. Amazon SQS makes it easy to build an automated workflow, working in close conjunction with the Amazon Elastic Compute Cloud (Amazon EC2) and the other AWS infrastructure web services.

Microsoft Azure Services Platform

• Azure

The Azure™ Services Platform is designed to help developers quickly and easily create, deploy, manage, and distribute web services and applications on the Internet. Windows® Azure is a cloud services operating system that serves as the development, service hosting and service management environment for the Azure Services Platform. Windows Azure provides developers with on-demand compute and storage to host, scale, and manage web applications on the internet through Microsoft data centers.

• .NET Services

Microsoft .NET Services are a set of Microsoft-hosted, highly scalable, developer-oriented services that provide key building blocks required by many cloud-based and cloud-aware applications. Much like the .NET Framework provides higher-level class libraries that make developers more productive, .NET Services enables a developer to focus on their application logic rather than building and deploying their own cloud-based infrastructure services.

• Access Control

The Microsoft .NET Access Control Service provides an easy way to control web applications and services while integrating with standards-based identity providers, including enterprise directories and web identity systems such as Windows Live ID. Authorization decisions can be pulled out of the application and into a set of declarative rules that can transform incoming security claims into claims that applications understand.

• Service Bus

The Microsoft .NET Service Bus makes it easy to connect applications together over the Internet. Services that register on the Bus can easily be discovered and accessed, across any network topology. The Service Bus provides the familiar Enterprise Service Bus application pattern, while helping to solve some of the hard issues that arise when implementing this pattern across network, security, and organizational boundaries, at Internet-scale.

• Workflow

The Microsoft .NET Workflow Service is a high-scale host for running workflows in the cloud. It provides a set of activities optimized for sending, receiving, and manipulating HTTP and Service Bus messages; a set of hosted tools to deploy, manage and track the execution of workflow instances; and a set of management API’s. Workflows can be constructed using the familiar Visual Studio 2008 Workflow Designer.

• SQL Services

Microsoft® SQL Services delivers on the Microsoft Data Platform vision of extending the SQL Server capabilities to the cloud as web-based services, enabling you to store structured, semi-structured, and unstructured data. SQL Services will deliver a rich set of integrated services that you can use to perform relational queries, search, reporting, analytics, integration and synchronize data with mobile users, remote offices and business partners. Currently, SQL Services offers relational database service called Microsoft® SQL Data Services. Other services will be available in future.

• Live Services

Live Services is a set of building blocks within the Azure Services Platform for handling user data and application resources. Live Services provides developers with an easy on-ramp to build rich social applications and experiences, across a range of digital devices that can connect with one of the largest audiences on the Web.

• Sharepoint Services

In the future, developers will have access to functionality from Microsoft SharePoint® Services in the Azure Services Platform. With the flexibility to use familiar developer tools like Microsoft Visual Studio, developers will be able to rapidly build applications that use SharePoint capabilities as building blocks for their own applications. Developers can expect a breadth of SharePoint capabilities across the spectrum of on-premises and online applications using the Azure Services Platform.

• Dynamic CRM

In the future, developers will have access to Microsoft Dynamics CRM functionality in the Azure Services Platform. With the flexibility to use familiar developer tools like Visual Studio, developers will be able to rapidly build applications that use Microsoft Dynamics CRM capabilities as building blocks for their own applications. Developers can expect a breadth of Microsoft Dynamics CRM capabilities across the spectrum of on-premises and online applications using the Azure Services Platform.

While doing all of this blogging on cloud computing lately, it came to my realization that I should probably put together a template/outline for a complete set of posts to cover everything that I'd like to discuss, and to perhaps serve as an outline for a book or at least a series of white papers.  What I've come up with so far is below, and I'll modify this list as I go along refining the topics:

Working title: Building Cloud Applications with .NET, Azure, and Amazon Services

I.                    Introduction

1.       Overview

a.       What is cloud computing?

b.      What problems does cloud computing solve?

c.       Business Benefits of Cloud Computing

d.      Components of a cloud application

                                       i.      Table based storage

                                     ii.      File based storage

                                    iii.      Service bus

                                   iv.      Workflow

                                     v.      Web front ends

                                   vi.      Dynamic processing allocation

                                  vii.      Queue based messaging

                                viii.      Web and worker roles

                                   ix.      Mesh Services

                                     x.      Data services

                                   xi.      Geographic diversity

                                  xii.      Elastic addresses

e.      Where are we going in this book?

2.       Cloud Computing Architectural Patterns

a.       Overview

b.      Example Architectures

                                 i.      Processing Pipelines

                               ii.      Batch Processing Systems

                              iii.      Websites

c.       Characteristics of Cloud Systems

                                 i.      Loosely Coupled

                               ii.      Self scalable

                              iii.      Highly Parallel

                             iv.      Must be reboot and relaunch resilient

                               v.      Distributed storage and computing supporting failover

II.                  Cloud Services: Azure and Amazon

3.       Windows Azure Services

a.       Overview

b.      Comparison to reference architecture

c.       Components of Azure

                                 i.      Internet Service Bus (.NET Services)

                               ii.      Azure web applications

                              iii.      Mesh

                             iv.      SQL Data Services

4.       Amazon Services

a.       General overview

b.      S3

                                 i.      Buckets,

                               ii.      …

c.       EC2

                                 i.      AMI’s

                               ii.      Kernels

                              iii.      Security Groups

d.      SQS

e.      SimpleDB

5.       Comparison of Azure and Amazon Cloud Services

a.       Similarities and Differences

b.      Using the two together

III.                Programming the Cloud with .NET

6.       Web front ends

a.       ASP.Net applications for Azure

b.      ASP.NET on Amazon

7.       The Internet Service Bus: Connecting Desktops and Servers to the Cloud and to each other

a.       Overview of ISB

b.      Connecting to the Cloud to request Services

c.       Integrating with Partners through the ISB

8.       Storing and Retrieving Data in the Cloud

a.       Blob storage: Azure blobs and S3 storage

b.      Table data storage: Azure tables and Amazon SimpleDB

9.       Databases

a.       SQL Data Services

b.      Connecting to your SQL Server

c.       Amazon SimpleDB SQL

10.   Queuing: using Azure and Amazon Queues

11.   Workflow: Orchestrating the Cloud

a.       MS Workflow

b.      Self implemented workflow

12.   Mesh Services: Replicating Data

13.

IV.                Deploying, Operating and Optimizing Cloud Applications

13.   Application Packaging:

a.       Azure packages

b.      Amazon AMI’s

14.   Deployment to the Cloud:

a.       Provisioning Azure applications

b.      Deploying AMI’s at Amazon

15.   Geodiversity

a.       Using Azure Geodiversity

b.      Amazon Availability Zones

c.     Amazon Cloud Front

16.   Data partitioning

a.       Why Partition the Data?

b.      Partitioning Azure Table Data

c.       Similar for Amazon

17.   Monitoring

a.       Logging

b.      Self monitoring in the cloud

c.       Remote cloud monitoring

d.      Talking back to OSS systems

V.                  Closing

18.   Best Practices for Cloud Computing

a.       Storage

b.      Queueing

c.       Table data manipulation

d.      Start multiple instances at once

e.      Plan for failure

19.   Cloud Computing Futures

a.       Automatic Demand Based Allocation

b.      Better Monitoring

c.       Windows Live Services

d.      Advanced Amazon Services

                                 i.      Mechanical Turk

                               ii.      Payment Services

                              iii.      Search

My previous post showed how to create a simple asp.net application that runs on the local development fabric.  This post, as promised, will follow up and show how to move your application into the azure services cloud.

The first think you need to do is go into the azure services developer portal and create a new project.

Click on the 'Hosted Services' item, and you will be taken to a page that allows you to create a new project:


Next will take you to a form that allows you to specify the name of the service, but is really just the DNS name that will be assigned to the web app when in production:


When that is complete, you will be taken back to the main page which will have changed its look to be as follows:



The files I specified were a result of publishing from vs.net using the 'Publish' menu item.  This will create these packages, open the azure portal, and open an explorer folder where the files are located.

When pressing the deploy button, you will be presented with a dialog telling you the files are being copied:

This is the first post is a series of posts that I will be making about using the windows Azure cloud computing services.  This first post will be about making your first cloud service that runs locally on your computer.  Over the series of posts I'll explain moving the code into the actual azure cloud, adding storage to your application, using the cloud services as a bus between various distributed applications, providing scalability and reliability, and I'm sure other things that come up along the way.

The first this to do to get started is go to http://www.azure.com:

Click on try it now and download and install the SDKs:


While you are at it, I'd suggest registering for the services.  I did this at the PDC and got registered almost immediately.  I don't know how long it will take at this point, but in the worst case you can still develop and run locally.

Once the SDKs are installed, open visual studio and create a new web cloud service project / solution:


The solution will open and look as follows:


This is the service configuration file.  It is used by the cloud fabric to determine the roles and # of instances of each role to run.

There is another configuration file, the service definition file.  This file informs the fabric how external applications are to talk to the role.  In this case we are defining normal http access:


The project also has a web page added to it, and I've added a little bit of text to it:


You can then run this application with ctrl-f5.  Thing will appear a little different during the build and start up process.  You'll notice some messages about initializing the development storage system, which is basically the local cloud fabric and its data storage.  What's going on is the that application is being packaged and installed into the local fabric.  Once that is done, IE will open and you will see your application:



All in all, this does not seem too exciting as it's a real simple ASP.NET app.  The difference here is that it is running on a local development fabric, not just on a local copy of IIS.  If you look in your task bar, you will notice the following icon (it looks like a couple of azure colored gears):


Click on the icon and select the "show development fabric UI" menu item, and you will see the following user interface:


This UI is showing you the current applications being hosted in the cloud fabric on your local system.  This is a really cool tool, as when you run multiple instances, you will see multiple consoles.  When we actually get around to running .net console apps in the fabric, information that you write to the console will actually be shown here.  In a way this tool is a viewport into what can be many virtualized .net instances running on your system (or in the cloud).

One last thing in this lesson.  Click on service details, and you will get information about the service:


Notice that this information matches the information in the service configuration file that I showed before as part of the solution.

And voila, you are running in the cloud, albeit a cloud on your local system.  Very easy so far, but not too exciting.

In my next post, I'll show you how to move this into the actual cloud.  I have not actually done this yet, so it will be a discussion of my experience learning how to do it.

This used to be Biztalk services. Good to see that it has evolved into something more useful.

There is a diagram of the Internet Service Bus.  Need to get this.  Cant find it easy online.

Naming

• DNS has some practical constraints: latency, pollution, hosts not services, limited write
• Service Bus Naming System: r/w with access control, updates instantly, endpoints not machines
• Global naming structure: scheme://solution.servicebus.windows.net/name/...

Service Registry

• For service endpoints, not a general directory
• solution in name is a root of a hierarchy of feeds
• Looks like this also has an rss feed; nice!
• Interfaces by WCF ServiceRegistrySettings class
• Publishing is done with rest requests to access control, create WCF endpoint, then an Atom feed describing the endpoint (through an endpoint), and then publish.  Again, nice!
• For the CTP, these are ephemeral.  You need to recreate them all the time.  Will be sticky later.
  

Messaging

• WCF is the primary programming model (but you can do Java too)
• All kinds of bindings
• Supports, SOAP 1.1, 1.2, all WS-*, SSL, RM, Streaming, Full extensibility model, WebGet/WebInvoke, Metadata exchange
• Not supported: atomic transaction flow, protocol level transport authentication, webscripting behavior, direct tcp modes with RM of WS-* Sec.
  

Notes on bindings:

• NetOnewayRelayBinding is the most used.  Requires outbound port 828, although there are workarounds.  Routes from client to server one way through the fabric.  Just showed a nice demo of a laptop connect just on t-mobile edge receiving messages from a client on the local wifi; very nice.
• NetEventRelayBinding.  Similar to oneway, but allows two receivers.  Topic based model, eventing with multi-cast.
• NetTcpRelayBinding: should use for everything.  Highest throughput.  Introduces an intermediary.  Basically, this uses the bus to find the other node, and then direct connections between nodes are established through the relay.
  
• Hybrid mode of the previous.  Direct connect without relay, even with firewalls.  Predication algorithms used to determine which ports will next be open on NATs.  This is pretty cool as it will move between relayed and direct.
• WS|Basic|Web|HttpRelayBinding: Uses Http instead of sockets.  Connections are pooled.

Some links:

• http://msdn.microsoft.com/en-us/library/bb906065.aspx
• http://vasters.com/clemensv/PermaLink,guid,92d78bee-2cfd-4a29-95ab-c5abb9b905e7.aspx
• http://www.microsoft.com/azure/servicebus.mspx

Some pictures I found:

Boy, this was a waste of time.  Should have stayed in bed.  That is everyone else's opinion too as everyone is leaving early, like me.  It's a couple of guys from MSR rambling on things that no one cares about.  I'm out and over to the big hall.

• Cloud services have specific design considerations:  Always on, distributed state, large scale, and failure handling.
• Azure is an OS for the cloud: scale out, dynamic and on demand
• Azure manages services not just servers: tell is what you want and it will help automate the details
• Frees developers from many platform issues: allows concentrating on logic instead of platform

This session will show us how this is done.

Characteristics of cloud computing

• Scale out not up
• Add and remove capacity on demand
• Pay for what you use as you go
• Automation is key to reducing costs

Design coniderations

• Failure of any node is expected: each node is a cache and must be replicated
• No one-time install step: apps need to reinitialize on restarts, don't assume previous local state is available
• Configuration changes to due to load or failures: handle dynamic configuration changes
• Services are always running: rolling up grades/downgrades; must handle data schema changes
• Services are build on multiple nodes / roles: document service architecture and communications paths
• Services can grow very large: careful state management at scale is needed

benefits of adhering to a windows Azure design point

• Azure manages services not just severs: tell it what you want and it automates; system manges nodes, services and network
• Automates sevice live-cycle management: MDA, allocation deployment snd SLA
• Turns pool of physical resources into a shared fabric: pay for what you use; platform insures isolation

Azure Service Lifecycls

• Coding and modeling
• Provisioning
• Deployment
• Maintain goal state

Service Model Guides Automation

• Describes service as distributed entities: authored by service developer, and configured by service developer
• Logical description of the services: same model used for testing and prod, mapped to hardware at deployment time
• Powerful declarative composition language

Azure Service Model Elements

• Service
• Role
• Group
• Endpoint
• Channel
• Interface
• Configuration settings
  

Fault Domains

• Purpose: avoid single points of failure
• Unit of failure: ex: compute node, rack of machines
• System considers fault domains when allocating service roles: ex: dont put all roles in the same rack
• Service owner assigns number required by each role: ex: 10 front-ends across 2 fault domains

Update Domains

• ensure service stays up while updating
• unit of software/configuration update: ex: set of nodes to update
• Used when rolling forward or backward
• Developer assigns # required by each role

Dynamic Configuration Settings

• Purpose is to communicate settings to service roles; there is no registry for services
• Application configuraiton settings; declared by the developer; set by deployer
• System configuration settings: pre-declared, sample kinds for all roles (instance it, fault domain id, update domain id); assigned by the system
• Both cases, available at run-time via callbacks when values change

Azure Automation

• Fabric controller: maps declarative service specs to available resources; manages service life cycle starting from the bare metal; maintains system helth and SLA
• What's special about it?: MDA

Azure Push-Button Deployment

1. Allocate nodes, across fault domains, update domains
2. Place OS and role images on nodes
3. Configure settings
4. Start Roles
5. Configure load balancers
6. Maintain desiered # of roles: failed roles automaticall restarted; node failure results in new nodes automatically allocated

Managing running services

• Adding capacity: push-button; steps repeated to the running service
• Removing: pb, steps reversed
• Rolling service upgrades: pb, iterative\

Rapid reliable software provisioning

• image based multicast deployment (scalable and reliable)
  
• seperate OS and service images: images copid, not installed; same images used for physical machines and vms
  
• multiple images are cached

Monitoring and Events

• Log collection
• Alerts
• Usage metering
• Data available through portal

Service Isolation and Security

• Yours are isolated from other services: model is boundary of isolation; local node resources is temp storage; network end-points
• Isoolation using multiple mechanisms
• Automatic application of windows security patches (rolling OS image upgrades)

Axure is Highly Available

• Netowrk has redudancy: switches, lb, access routers
• Services deployed across fault domains (lb's route to active nodes only)
• Fabric controller state checkpointed:can roll back to previous chekpoints

Azure automates

• Provisioning and monitoring of hardware
• Hardware life cycle mmt
• Capacity planning
• Internal security measures

Roadmap

• PDC: automated service dployment; subset of service model - simple set of service template; can change # of isntances; simple upgrades / downgrades; automated service failure and recovery; hardware mgmt; managed code / asp.net; run in fixed-size VM instances; external virtual IP address per service; service network isolation enforcement
  
• 2009: expose more of underlying service model, native code; multiple data centers