Architecture of the .NET Services

by mheydt 28. October 2008 21:41 >
.NET Services are the extension of .NET techs to the cloud
Specifically Service bus and Workflow

The Service Bus
  • Allows apps to talk together no matter where located, specifically across the Internet.
  • Secure (WS-Sec), highly scalable
Internet Service Bus Pattern
  • Service Registry
  • Connectivity
  • Pub/Sub
  • Bindings
  • Ensure connectivity
  • Access control
  • Cloud Services (storage, compute, billing, ...)
  • Messaging Fabric
  • Naming
  • Orchestration
  • On-Premises applications
Service Registry
  • Allows attaching services into a distribution means to be found
  • [http|sb]://servicebus.windows.net/services/account/svc/...
  • Maps URIs to services
Connectivity
  • Two key: Relay, Direct Connect
  • Relay ensures connect, available to all services
  • Direct uses relay to connect and then shortvuts for efficiency
  • Available via HTPP/REST/ATOM
  • Available in .NET via WCF

Federated Identity and Access Control

Pub/Sub
  • Builds on the relat and direct connect
  • Initial release is "connected multicast"
  • Over time extended to other models such as anycast, reliable, ...
Bindings - Many of them
  • BasicHttpBinding
  • WS2007HttpBinding,
  • ... (about 10 more)
Connections are buffered with a queue to keep the # of connections low on the network edge.

Access Control
  • Managed through rules
  • Composes with SOAP-over-HTTP
  • SOAP 1.1, 1.2, WS-Security, transport-only message protections
  • Supports any SOAP 1.2/2.0 BP compliant client
Unauthenticated Senders
  • Unauthenticated send capability
  • Client does not need to acquire tokens
  • Plain basic profile SOAP requests
  • opt-in set by listening services
Workflow
  • Challenges: long-running, modularity and nesting, easy to describe but hard to run
  • Approach: 3.0/3.5/4.0 addresses key requests
  • Dublin is one way: IIS/WAS
  • Another is the werkflow servic
Workflow service: workflow.ex.azure.microsoft.com
New activities for azure
APIs that allow deploy, manage and run workflows on the cloud

Basic usage
  • Design using any tool you want
  • Deploy: upload and validate
  • Manage workflow types: add, delete, update and view instances
  • Can dynamically create and deploy workflows
  • Can address messages to a running instance (this is a new activity)

Tags:

.Net | Azure | p2pSB

Finding an initial peer in p2pSB

by mheydt 13. November 2007 16:54 >
One question that always comes up is how does a peer find other peers in the network? The first time the system is run, it will be true that the local node will not know of any other peers. With the assumption that a broadcast on the network is not possible, what must be done is a directed communication to one other peer at a predetermined address, which the client must be configured to know about. Many refer to this particular type of peer as a "rendevous" server, who's responsibility it is to be located at a static IP address (or a dynamic one but reachable through DNS) and can handle initial requests for a peer to connect to the peer network. I also believe that other types of peer networks refer to these rendevous servers as "super nodes".

To accomplish this in p2pSB, a node can be configured with a static set of peer nodes in the configuration file and those peers can be identified as having a rendevous service available. Upon initialization, a p2pSB node will attempt to connect to all other nodes known to have rendevous services. Upon receiving a connect message from another node, the rendevous service will return to the sender an acknowledgment message which the originator can use to know that it is now connected to the network and that it can start further peer discovery through those nodes (and it's through this reply that a node can identify its external ip address and pass that to other nodes so that they can communicate directly).

If you want to think of this from another perspective, a rendevous server in p2pSB can be a stripped down peer node with no other services that operates very similarly to a tracker in a bittorrent network, keeping track of nodes as they come online and go offline, and providing a place for nodes to find others in the network and to begin exchanging information. A difference is that you can designate your node as providing this service, or put other nodes out in the network to handle this for you as well as to spread the load amongst peers connecting to the network.

An additional benefit of this model is that in p2pSB rendevous nodes can be configured to bridge information to other rendevous nodes, forming a virtual backbone to help with scalability of finding nodes in the network. For example, peer discovery services in p2pSB has a special message sent to a rendevous node to ask it for the location of other rendevous nodes, which the client can then use to search for other peers.

Given this architecture, assume your are trying to find a particular node in the network that provides a particular service in the network. You know of this node and its services because you previously became a buddy with it by remembering it's node id so you can locate it again at a later time. You can't store the IP address as it may (and most likely will) change, so you just remember the ID of the buddy node. When connecting to the network, peer discovery services can identify all of the rendevous nodes and then broadcast a peer location request for that particular node by id. If any of the rendevous servers know that peer is online, they will reply with the external connection point for that node so that communications can begin.

If the discovery through rendevous servers fails, the client can then optionally broadcast to other known peers (but non-rendevous service peers) to see if they know of that specific peer. Note that this is more limited in nature as it is the rendevous services / nodes job to track any and all rendevous announcements and report them to others upon request, whereas a normal peer only tracks nodes that it is currently (or recently) in communications with, either by directly establishing communications with them or vice-versa.

Tags:

p2pSB

iCity / p2pSB and multiple tracker support per data package

by mheydt 25. September 2007 15:54 >
I want the operation of trackers in iCity to be more robust than in normal bittorrent clients.  In a normal bittorrent network, a client opens a torrent file which has the address of a tracker for that particular torrent, and the client connects to that tracker.  The tracker is typically a dedicated server system that coordinates communications amongst peers and may not actually share the file itself, but simply manage a client finding other clients in the network.

This makes it such that the tracker system must be online all of the time, and at a particular location (address) in the network.  This is disadventagoues for several reasons, including that 1) if the tracker goes offline, all communications cease (single point of failure), and that a client cannot always itself host tracking services.  The latter point is important as without the capability of tracking items you share, you must upload the torrent file to the tracker system for hosting which involves additional steps and gets you into the tracker availability situation previoiusly mentioned.

What I propose as an implementation of tracking in iCity is the following:

1) Any node can spin up a tracker at any time and track their own shares of packages.

2) Any node can be configured with the identity of other tracker nodes in the iCity community. They can contact those trackers and download a torrent file to download the desired file using that tracker.

3) If not desirous of self hosting a tracker, the node can ask other nodes (in particular other dedicated trackers) to host tracking for a package.  Note that this can be multiple systems to have redundant trackers to provide for greater availability.

4) If not being able to connect to any trackers for a particular package, the node can query other nodes in the community for tracking services for the package and connect to any of them.

In all, this does not change the model of tracking or the protocols involved in bittorrent tracking.  It does however build an overlay p2p network to faciliate distribution and access of tracking services for content in the community.

Tags:

iCity | p2pSB

Publishing in iCity

by mheydt 24. September 2007 15:51 >
I was giving thought to publishing content in iCity (formally now called nInformationCity).  Specifically, I'm trying to work up a particular use case for this.  The ones I've come by at this point are:

Use a WPF app to select files on your system for publishing.  You can then associate them with a particular community, select a package name (something to identify the files by), and select publish.  The system would then mark them as owned by you and assign it a version id (1 for a new package), and mark it as published.  Additionally, you can assign a taxonomy to the package to facilitate searching and location, as well as various other metadata fields such as a description.

What this allows is then for other users of iCity to query your system for packages.  Upon receipt of a query for packages, the system would return to the peer a summary of packages, and a further request could retrieve detail of any or all of the packages.  From that, a peer can request a transfer of a package which would be accomplished by using nDistribute.  This would then spin up a bittorrent tracker on your system (or on a dedicated tracker system) other clients can connect to in order to
transfer the content.

Even more interestingly, other peers can request a subscription to a package.  Upon changes to the package by yourself, iCity would inform peers in that community of new packages being deployed.  Or, upon subscribing to the community, all others in the community could be automatically pushed update messages to any packages published to the community.

The concept also becomes important for versioning and notification of new content.  If the "owner" of some data changes that content, others that already have the data must be able to be able to know if they do not have the most recent version, and when publishing data versions can be incremented, others notified, and content distributed.  Also, it can be used to hosting multiple versions of related data in the network if that is so desired.

I also think that it is probably important to be able to identify in the system a new package id for each version, as well as an id / version tree as if multiple people make changes to a single package, it would be good to be able to identify the history of changes for reconciliation.

Tags:

p2pSB

Command flow in p2pSB

by mheydt 2. July 2006 01:20 >
Workflow for delivering commands from the user to a peer in p2pSB.

Client side
  1. User issues command (ie. ConnectToNetwork)
  2. Service host looks up the appropriate service.  In this case, ConnectToNetworkService.
  3. Execute is triggered on the ConnectToServiceObject.
  4. AvailablePeers are retrieved, and ConnectToNetworkMessage delivered to each by the following process.  Note that Redezvous Servers are considered peers in this list.
  5. The list of known transport types for each peer is identified from the PeerConnectionInfo class associated with the peer.
  6. ConnectToNetworkMessage is sent to each peer via that transport.  Note that this is an asyncronous action and considered fire and forget.
  7. The message is places in the message store for correleation of connect messages.
Server Side
  1. ConnectToNetworkMessage is received by a listener.
  2. Decrement TTL, and if 0 dispose of message.
  3. Message is routed to the message node.  Message node "touches" the message.
  4. Message node routes it to the ConnectToNetworkService.
  5. Connect to network service acknowledges the message with a response back through the incoming message node.
  6. An entry in the peer list is made for this node.
  7. The message is forwarded to all peers on this servers peer list that are not already on the visited peers list.

Technorati Tags:

Tags:

p2pSB

about the author

I'm a .NET, XAML, and iOS polyglot that loves playing with new things and making cool and innovative stuff.  I am also a Mac junkie.

I am Principal Technologist for SunGard Global Services in NYC, in their Advanced Technologies practice, and I work extensively with SunGard's energy and financial customers.

Note the the posting on this blog are my own and do not represent the position, strategies or opinions of SGS.

twitter

I can't stop thinking big!
Sunday 1:08AM via WindowsLive
Just watched Moneyball. That's my pick for best movie this year.
Saturday 3:51PM via WindowsLive
@vincebelpiede: Report: Skype For Windows Phone Beta Imminent http://t.co/KYNjgg1L#mhtnd
Wednesday 8:39AM via Twitter for Mac
@mashable: Kinect Fusion Will Turn Gaming (and More) Into a 3D Fun House - http://t.co/Ihrq2fY2#mhtnd
Wednesday 8:39AM via Twitter for Mac
New Kinect SDK: http://t.co/57MvA5L5 #mhtnd
Wednesday 8:39AM via Twitter for Mac
Follow me on Twitter

recent comments

None

month list