Notes on SQL Azure Present and Future

by mheydt 18. November 2009 02:45 >
SQL Azure Database
  • Highly scaled out relational database as a service
  • Accessible via TDS
  • SOAP/REST/ADO.NET/EDM/HTTP/HTTPs
  • On premise access via SSRS
Customer value props:
  • Self provisioning and capacity on demand
  • Symmetry with on promesis database
  • Automatic high availability and fault-tolerance
  • Automated DB maintenance
Provisioning:
  • Account as zero or more servers, billing instrument
  • Each server contains one or more database
    • Metadata
    • Unit of security
    • Unit of geo-location
    • Logical grouping of DBs
  • Each database has standard objects: unit of consistency, users, tables, views, indices, etc..
Free period ends 2/1/2010

Future:
  • Improve tools
  • Improve operational model
  • Improve application programming model
Saas:
  • Mention of provisioning API's outside of the portal
  • Template databases
  • Metadata tracking
  • Additional billing scenarios
Single unit for deployment and upgrades
Data Synch

Upgrade and downgrade options between SKU's
Read only databases

Scale out support:
  • Today
    • workload partitioning
    • but cluster managmeent is difficult
    • Scale out addressed today:
      • high availability
      • Zero admin
      • No downtime
      • Elastic resources
      • Pay as you grow
      • No friction provisioning
      • Never run out of hardware
  • Future
    • More scale out
      • Dynamic database splits
      • Ability to merge databases
      • Improved schema management across groups of databases
      • Additional database size options
      • multiple database connection management
      • Support for fan-out query
  • Recent requests from customers
    • Profillers, DMVs
    • Spatial data types
    • Full text support
    • Change tracking
    • CLR
    • BI
    • Encryption
    • WIF

Tags:

Azure | Cloud

Notes on Azure Present and Future

by mheydt 18. November 2009 01:27 >
Evolution of Azure
  • 2008: 
    • App hosting, two roles, queue based communication, partial trust asp.net
    • storage: blobs, tables, queues
    • Desktop sdk: cloud simulation
    • Service mgmt portal: vip swap upgrades, automatic os servicing
Coming soon:
Paid usage Feb 2010
New api exposed and updates IP/Port values
Direct inter-role communications
RoleEnvironment.Changed
RoleEnvironment.Changing
Secure certificate store in the cloud
Logging and diagnostics
Random writes to blobs
Disk drives (February)
Geo-replication
Secondary indexes on tables
Service Management API
In-place rolling upgrade

Tags:

Azure | Cloud

More on rewriting url's for Azure

by mheydt 30. April 2009 21:13 >
In my previous post I showed how to rewrite urls which where applied to .asmx web service proxies.  In the last couple of days I needed to convert WCF client proxies for Silveright to do the same thing.  Turns out this is very similar (if not identical), but I found a new syntax for doing it which makes things a little bit easier.  Here is an example:



The ManagementServiceClient class is the proxy class generated for the WCF service.  The change is to use the App.Current.Host.Source variable and append the path to the .svc file.  Works like a champ and looks real nice.

Tags:

Azure | C# | Cloud | Silverlight

Best Practices for Utilizing Cloud Architectures

by mheydt 25. January 2009 23:02 >
  1. Package files for upload instead of uploading many small files
  2. Utilize multi-threaded access to blobs
  3. Utilize exponential back-off and retry when calling services
  4. Store reference information within messages
  5. Use Process-oriented messaging
  6. Use Document oriented messaging
  7. Use queues to distribute load
  8. Utilize controller systems to start and monitor clusters
  9. Use table based storage to store metadata, not the actual data
  10. Launch multiple instances at once
  11. Allocate compute nodes out of pools to lower startup times
  12. Automate as much as is possible
  13. Keep systems loosely coupled
  14. Think parallel
  15. Think on-demand
  16. Design for reboot and relaunch

Tags:

Cloud

Characteristics of Cloud Computing

by mheydt 25. January 2009 22:34 >
  • Self-healing / Reliability: In case of failure, cloud systems are capable of taking advantage of failover to standy systems (both CPU and storage), thereby enabling high levels of availability and reliability to applications that take advantage of these capabilities.
  • SLA-driven: The cloud is dynamically managed dynamically by service-level agreements that define policies such as delivery parameters, costs and availability.
  • Multi-tenancy: The system is built in a way that allows several customers to share infrastructure, without the customers being aware of it and without compromising the privacy and security of each customer’s data.  This also allows for low costs due to sharing of the resources.
  • Service-oriented: Cloud based systems are the epitome of loosely coupled service oriented systems, allowing the composing applications out of discrete services that are independent of each other, and also provide the inherent ability to failover to backups through redundancy.
  • Virtualized: Applications are decoupled from the underlying hardware, and dynamically allocated to available systems, freeing you from needing to worry about where an application runs, what hardware it runs on, and for maintaining the hardware over time.
  • Linearly / Incremental Scalability: Cloud environments provide the ability to allow users and systems to add additional compute resources on-demand in response to application loads.
  • Agility: Cloud system provide the basis to put quickly put together systems that can solve new problems through use of in-cloud services.
  • Virtually Unlimited and Replicated Data: Need more storage?  Just store more in the cloud.  It's the clouds providers responsibility to provide enough storage, and you only have to pay for what you use, not for what you might use.  And you never will get blocked by running out of physical storage.

Tags:

Cloud

Components of a Cloud Architecture

by mheydt 24. January 2009 23:15 >
In the diagram that I previously posted I laid out a cloud architecture composed of 5 major components:
  • Cloud Ingress
This section provides the means by which systems outside of the cloud (and sometimes systems within the cloud) can locate and communicate with systems in the cloud.  This is typically done through one or more of the four means shown (although any network communication is technically possible):
  • Queue based communications
The application outside the cloud posts a message into a message queue, where systems within the cloud look for messages, process them, and potentially send messages back to the client through another queue.
  • HTTP Communications
HTTP communications into the cloud can be broken down in to two primary forms of communications.  First is the access of cloud applications through standard web applications presented through HTML.  Another is through the use of tunneling other protocols over HTTP (although I consider REST and SOAP over HTTP its own category).
  • REST / SOAP
Basically these come down to accessing web services within the cloud.  The historical means of doing this is over SOAP, but REST has become a popular alternative over the last few years do to its relative simplicity compared to SOAP.
  • Service Bus
The "Services Bus", otherwise referred to as the "Internet Service Bus", or ".NET Services", is a Microsoft technology that allows systems to register their "endpoints" in the cloud, whereby the cloud publicly exposes those endpoints, thereby allowing systems outside the cloud (or within) to locate other systems within (or outside the cloud) and establish connections with each other.  This also provides another valuable service in that it will define tunnels through firewalls on each end.
  • Horsepower
This section of the cloud is basically dominated by raw CPU power, although that power can be subcategorized into functional sections.  One of the primary tenants of the CPU in cloud computing is that it is "on-demand", being dynamically allocated only when needed.  The on-demand nature has a direct ramification for cost savings in that you are typically only billed for the CPU time used, but there are also indirect benefits in ability to failover easily.  The on-demand nature of CPU utilization is a direct result of the work over the years in computer and software virtualization, except that it is now extended to be able to on-demand allocate systems from libraries of machine images instead of just within your own enterprise.
  • Cloud Storage Services
All cloud services provide native services for storing data to your applications.  This breaks into a few different types of storage, ranging from "blobs" (basically files, but with limitations compared to normal file systems), table data (structured data, but not as complex as RDBMS systems), structured storage systems (more akin to RDBMS system), and dynamic storage volumes which are comparable to disk drives on regular systems but able to be created on the fly and attached to any virtual system.
  1. Cloud Provided Services
All cloud providers will offer additional services to your systems in the cloud (and possibly to you from outside the cloud).  This can be things along the lines of advanced data services, payment processing, search, web and generalize data cachin, workflow, and more complex application services such as CRM/ERP/MRP.
  1. Intra-Cloud Communications
[Explain why this is important] Cloud service providers will provide you with one or means of allowing your cloud based systems to communicate with each other.  Communications between cloud based systems is typically accomplished via message queuing facilities, and both Amazon and Microsoft provide these services.  Microsoft also provides its .NET Services facility (also know as an Internet Service Bus) that can also allow your systems to find and communicate with each other.

Tags:

Cloud

Service Providers and their Services

by mheydt 24. January 2009 22:49 >
The two primary cloud service providers available at this time are Amazon and Microsoft.

Amazon Web Services (AWS)
  • Elastic Compute Cloud (EC2)
Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.
  • SimpleDB
Amazon SimpleDB is a web service providing the core database functions of data indexing and querying. This service works in close conjunction with Amazon Simple Storage Service (Amazon S3) and Amazon Elastic Compute Cloud (Amazon EC2), collectively providing the ability to store, process and query data sets in the cloud, making web-scale computing easier and more cost-effective for developers.
  • Simple Storage Service (S3)
Amazon S3 is storage for the Internet. It is designed to make web-scale computing easier for developers.
  • CloudFront
Amazon CloudFront is a web service for content delivery. It integrates with other Amazon Web Services to give developers and businesses an easy way to distribute content to end users with low latency, high data transfer speeds, and no commitments.
  • Simple Queue Service (SQS)
Amazon Simple Queue Service (Amazon SQS) offers a reliable, highly scalable, hosted queue for storing messages as they travel between computers. By using Amazon SQS, developers can simply move data between distributed components of their applications that perform different tasks, without losing messages or requiring each component to be always available. Amazon SQS makes it easy to build an automated workflow, working in close conjunction with the Amazon Elastic Compute Cloud (Amazon EC2) and the other AWS infrastructure web services.
Microsoft Azure Services Platform
  • Azure
The Azure™ Services Platform is designed to help developers quickly and easily create, deploy, manage, and distribute web services and applications on the Internet. Windows® Azure is a cloud services operating system that serves as the development, service hosting and service management environment for the Azure Services Platform. Windows Azure provides developers with on-demand compute and storage to host, scale, and manage web applications on the internet through Microsoft data centers.
  • .NET Services
Microsoft .NET Services are a set of Microsoft-hosted, highly scalable, developer-oriented services that provide key building blocks required by many cloud-based and cloud-aware applications. Much like the .NET Framework provides higher-level class libraries that make developers more productive, .NET Services enables a developer to focus on their application logic rather than building and deploying their own cloud-based infrastructure services.

  • Access Control
The Microsoft .NET Access Control Service provides an easy way to control web applications and services while integrating with standards-based identity providers, including enterprise directories and web identity systems such as Windows Live ID. Authorization decisions can be pulled out of the application and into a set of declarative rules that can transform incoming security claims into claims that applications understand.
  • Service Bus
The Microsoft .NET Service Bus makes it easy to connect applications together over the Internet. Services that register on the Bus can easily be discovered and accessed, across any network topology. The Service Bus provides the familiar Enterprise Service Bus application pattern, while helping to solve some of the hard issues that arise when implementing this pattern across network, security, and organizational boundaries, at Internet-scale.
  • Workflow
The Microsoft .NET Workflow Service is a high-scale host for running workflows in the cloud. It provides a set of activities optimized for sending, receiving, and manipulating HTTP and Service Bus messages; a set of hosted tools to deploy, manage and track the execution of workflow instances; and a set of management API’s. Workflows can be constructed using the familiar Visual Studio 2008 Workflow Designer.
  • SQL Services
Microsoft® SQL Services delivers on the Microsoft Data Platform vision of extending the SQL Server capabilities to the cloud as web-based services, enabling you to store structured, semi-structured, and unstructured data. SQL Services will deliver a rich set of integrated services that you can use to perform relational queries, search, reporting, analytics, integration and synchronize data with mobile users, remote offices and business partners. Currently, SQL Services offers relational database service called Microsoft® SQL Data Services. Other services will be available in future.
  • Live Services
Live Services is a set of building blocks within the Azure Services Platform for handling user data and application resources. Live Services provides developers with an easy on-ramp to build rich social applications and experiences, across a range of digital devices that can connect with one of the largest audiences on the Web.
  • Sharepoint Services
In the future, developers will have access to functionality from Microsoft SharePoint® Services in the Azure Services Platform. With the flexibility to use familiar developer tools like Microsoft Visual Studio, developers will be able to rapidly build applications that use SharePoint capabilities as building blocks for their own applications. Developers can expect a breadth of SharePoint capabilities across the spectrum of on-premises and online applications using the Azure Services Platform.
  • Dynamic CRM
In the future, developers will have access to Microsoft Dynamics CRM functionality in the Azure Services Platform. With the flexibility to use familiar developer tools like Visual Studio, developers will be able to rapidly build applications that use Microsoft Dynamics CRM capabilities as building blocks for their own applications. Developers can expect a breadth of Microsoft Dynamics CRM capabilities across the spectrum of on-premises and online applications using the Azure Services Platform.

Tags:

AWS | Azure | Cloud

Examples uses for cloud architectures

by mheydt 24. January 2009 03:38 >
  • Processing pipelines
    • Video transcoding
    • Converting large numbers of documents into other formats
    • Image processing used to convert images from one format to another
    • Searching and indexing of large amounts of data
    • Mining of large amounts of data
  • Batch processing systems
    • Various back office applications that needs heavy, but often infrequent batch processing
    • Reporting systems that analyze operational data and deliver reports, often infrequently
    • Software build systems that do nightly builds
    • Software testing systems that execute tests against many different configurations.
  • Websites
    • Sites that can change the number of servers based upon demand (less at night, more during the day)
    • Instant sites needed on demand for specific events
    • Seasonal sites (like taxes or various holidays)
  • Parallel / concurrent processing
    • Financial data correlation, where different systems can work on their own subset of the problem in order to speed processing time
    • Parallel video conversion into multiple bitrates
  • On-demand / scheduled processing
    • Bi-weekly payroll processing
    • Video transcoding during media events

Tags:

Cloud

Benefits of cloud computing

by mheydt 24. January 2009 03:36 >
I'm going to start this post as a place to build a running list of "benefits" of cloud computing.

Benefits of cloud computing can be broken down into several categories.  One of those are the "Business Benefits":
1. Almost zero upfront infrastructure investment
2. Just-in-time Infrastructure
3. More efficient resource utilization
4. Usage-based costing
5. Potential for shrinking the processing time
6. Virtually unlimited computation and storage

[http://www.webhostingunleashed.com/features/cloud-computing-benefits/]
1. Scalability
2. Easy implementation
3. Skilled practictioners
4. Frees up internal resources
5. Quality of service

[http://www.webhostingunleashed.com/features/essential-guide-cloud-computing/]
1. Access to data from any location with Internet access
2. No need for maintaining a physical storage device
3. No upfront capital expenditures for hardware
4. No ongoing costs for maintaining and upgrading storage facilities
5. Pay as you go
6. Can add or remove capacity based on real rather than projected storage needs
7. Alleviates burden on IT teams

Security Benefits
[http://cloudsecurity.org/2008/07/21/assessing-the-security-benefits-of-cloud-computing/]
1. Centralized Data
2. Incident Response / Forensics
3. Password assurance testing
4. Logging
5. Improve the state of security software
6. Secure Builds
7. Security testing

Another way to look at cloud computing is by comparing how you would use it compared to other services, such as with your own internal IT or with typical managed services (the way we thought of doing it in the late 90's early 2000's), and how it compares to the others in terms of both complexity and $$:

 Internal ITManaged servicesThe cloud
Capital investmentSignificantModerateNegligible
On-going costsModerateSignificantBased on Usage
Provisioning timeSignificantModerateNone
ScalabilityLimitedModerateFlexible
Staff expertise requirementsSignificantLimitedModerate
ReliabilityVariesHighModerate to High



Tags:

Cloud

A Reference Cloud Architecture

by mheydt 24. January 2009 00:16 >
I put this diagram together tonight to use in upcoming posts that I will be making about cloud architecture...


Tags:

Cloud

about the author

I'm a .NET, XAML, and iOS polyglot that loves playing with new things and making cool and innovative stuff.  I am also a Mac junkie.

I am Principal Technologist for SunGard Global Services in NYC, in their Advanced Technologies practice, and I work extensively with SunGard's energy and financial customers.

Note the the posting on this blog are my own and do not represent the position, strategies or opinions of SGS.

twitter

I can't stop thinking big!
Sunday 1:08AM via WindowsLive
Just watched Moneyball. That's my pick for best movie this year.
Saturday 3:51PM via WindowsLive
@vincebelpiede: Report: Skype For Windows Phone Beta Imminent http://t.co/KYNjgg1L#mhtnd
Wednesday 8:39AM via Twitter for Mac
@mashable: Kinect Fusion Will Turn Gaming (and More) Into a 3D Fun House - http://t.co/Ihrq2fY2#mhtnd
Wednesday 8:39AM via Twitter for Mac
New Kinect SDK: http://t.co/57MvA5L5 #mhtnd
Wednesday 8:39AM via Twitter for Mac
Follow me on Twitter

recent comments

None

month list