Patrick's Blog

SQL Azure Federations and Enzo Shard Demo

patrick — Thu, 27 Oct 2011 03:32:33 +0000

Attached are my powerpoint and C# demo of the Enzo SQL Shard library for the October KY Dot Net user group meeting. This presentation will cover Azure Federation concepts, sharding strategies, and a short demo of the Enzo SQL Shard Library utilizing a SQL Azure database.

Session information:

“Designing enterprise data solutions for today’s cloud computing environments can leave the average architect puzzled and confused. Listen in as we discuss how database sharding and the upcoming release of Sql Azure Federations will provide virtually unlimited scalability of data for your enterprise applications. Data federation solutions will be analyzed as we examine some of the current data virtualization vendors and preview some of the new features of Sql Azure Federations. Additionally, a federation-aware codeplex library will be demonstrated which supports fan-out queries and parallel round robin loading. Don’t miss!”

Bio: Patrick Riley is a software developer for the Rawlings Group in LaGrange, KY. He has worked on master data management solutions in the .NET framework and SQL Server for several years. He enjoys golf, grilling food on his Webber grill, and watching the technology race to the cloud

Feel free to send any feedback you might have to patrick@patrickmriley.net.

Enjoy!

PPTX:
Sql Azure Data Federations

Sample Enzo Federations Source Code:
EnzoShardCode_v1.0

Bitwise XOR on Byte Array

patrick — Wed, 31 Aug 2011 05:01:11 +0000

Spent the past few hours scrambling for a generic XOR solution on two byte arrays. I needed a solution that would add the match value to a specific result value if it did not already exist (assuming the most significant byte starts at the beginning of the array). You’d think this would be somewhere in the C# libraries by now. After an hour of searching, I decided to write my own. Ended up with the solution below.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

static byte[] BitwiseXOR(byte[] result, byte[] matchValue)
{
if (result.Length == 0)
{
return matchValue;
}

byte[] newResult = new byte[matchValue.Length > result.Length ? matchValue.Length : result.Length];

for (int i = 1; i < newResult.Length+1; i++)
{
//Use XOR on the LSBs until we run out
if(i > result.Length)
{
newResult[newResult.Length - i] = matchValue[matchValue.Length - i];
}
else if (i > matchValue.Length)
{
newResult[newResult.Length - i] = result[result.Length - i];
}
else
{
newResult[newResult.Length -i] =
(byte)(matchValue[matchValue.Length - i] ^ result[result.Length - i]);
}
}
return newResult;
}

Windows Azure Marketplace

patrick — Thu, 25 Aug 2011 21:10:46 +0000

In an attempt to update my blog, I am posting this from my January discussion at the KY .NET user group meeting on consuming cloud services. I will be speaking this October on Sql Azure Federations, and will be focusing my efforts in the coming year on cloud based applications. More to come…

Windows Azure Marketplace: Consuming the Cloud DataMarket
Leveraging the Cloud’s data services can provide real-time feeds that keep your users bustling. Learn how to quickly absorb the DataMarket’s datasets using OData APIs and expose them to client applications on any platform. A sample from Microsoft’s Windows Azure Support team utilizing a Windows Azure service to consume crime data from Data.gov will be discussed. OData, one of the cloud’s increasingly popular and flexible web protocols for querying and updating data, will be examined and reviewed.

Link to slides:

PPTX: Windows_Azure_Marketplace_Consuming_the_Cloud

Fuzzy matching: a programmer’s view of the Damerau-Levenshtein algorithm

patrick — Tue, 23 Aug 2011 03:30:33 +0000

The sheer mechanics of fuzzy matching algorithms are fascinating. Optimized variations are behind search engines and optical character recognition (OCR) softwares we see on the market today. An interesting algorithm I’ve recently used for a patient matching solution utilizes the Damerau-Levenshtein distance algorithm to locate fuzzy matches.

The Levenshetin approach considers how many substitutions, deletions, and insertions are needed to make two strings equal (also known as the edit distance between the strings). The algorithm was later improved by Damerau to include transposition (step highlghted on line 55). This solution is an example of bottom-up dynamic programming, because a matrix is flood filled to find the minimum distance. The complexity is cubic, but is improved to K x L complexity when using a maximum distance of interest (where K is the maximum distance, and L is the shortest string). The algorithm below uses a maximum distance (threshold paramter) and other simple optimizations to vastly improve the performance of the Damerau-Levenshtein. It has been tested to run over 10K string compares/second with two given strings under ten characters in length and a maximum threshold of three edits.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70

public static int DamerauLevenshteinDistanceImproved(string string1,
string string2, int threshold)
{
// Return trivial case - where they are equal
if (string1.Equals(string2))
return 0;

// Return trivial case - where one is empty
if (String.IsNullOrEmpty(string1) || String.IsNullOrEmpty(string2))
return (string1 ?? "").Length + (string2 ?? "").Length;

// Ensure string2 (inner cycle) is longer
if (string1.Length > string2.Length)
{
var tmp = string1;
string1 = string2;
string2 = tmp;
}

// Return trivial case - where string1 is contained within string2
if (string2.Contains(string1))
return string2.Length - string1.Length;

var length1 = string1.Length;
var length2 = string2.Length;

var d = new int[length1 + 1, length2 + 1];

for (var i = 0; i <= d.GetUpperBound(0); i++)
d[i, 0] = i;

for (var i = 0; i <= d.GetUpperBound(1); i++)
d[0, i] = i;

for (var i = 1; i <= d.GetUpperBound(0); i++)
{
var im1 = i - 1;
var im2 = i - 2;
var minDistance = threshold;

for (var j = 1; j <= d.GetUpperBound(1); j++)
{
var jm1 = j - 1;
var jm2 = j - 2;
var cost = string1[im1] == string2[jm1] ? 0 : 1;

var del = d[im1, j] + 1;
var ins = d[i, jm1] + 1;
var sub = d[im1, jm1] + cost;

//Math.Min is slower than native code here
//d[i, j] = Math.Min(del, Math.Min(ins, sub));
d[i, j] = del <= ins && del <= sub ? del : ins <= sub ? ins : sub;

if (i > 1 && j > 1 && string1[im1] == string2[jm2]
&& string1[im2] == string2[jm1])
d[i, j] = Math.Min(d[i, j], d[im2, jm2] + cost);

if (d[i, j] < minDistance)
minDistance = d[i, j];
}

if (minDistance > threshold)
return int.MaxValue;
}

return d[d.GetUpperBound(0), d.GetUpperBound(1)] > threshold
? int.MaxValue
: d[d.GetUpperBound(0), d.GetUpperBound(1)];
}

How failure can breed success…

patrick — Sat, 05 Mar 2011 18:30:07 +0000

Having wrapped up two major projects in my first four months with a new company, one quote seems to come to mind: “success is overcoming your failures, one at a time”. For a software developer, this point might be frequently overlooked. Small hurdles quickly get overshadowed with the next challenge. Reaching one milestone quickly opens discussion about the next.

But what do we hold onto on our journey? How do we learn from constant change?

One small victory opens a pandora of new opportunity, and a bigger gap for failure at the same time. With great risk, failure is inevitable. Those of us who embrace failure, will also embrace success.

Thanks for reading.

To the Cloud!

patrick — Wed, 29 Dec 2010 01:58:22 +0000

This week I am taking some time to discuss integrating your WP7 app with Windows Azure and the Microsoft “cloud”. This can be hugely beneficial if you happen to write the next “farmville”. The cloud offers the possibility to scale up and out, and in a matter of minutes.

In this post, I will show a quick demo and the tools you’ll need for constructing your all-embracing nimbus. Let’s get the basics first. You’ll need IIS 7, the WP7 developer tools, and the Windows Azure SDK. Once you get those out of the way, head over to the MSDN All-In-One Code team blog and download the demo. Ahhh, now for the fun stuff.

But first a bit of background. A core component to the Windows Azure platform has been neatly constructed in something we like to call “roles”, or runtimes for execution. Roles can be quickly targeted to any platform “in the cloud” and allow you to specify however many instances of your service or web application you would like. Roles are also easily configured in your role configuration file, or within the Azure platform app fabric portal. Web applications (defined within a web role) can be pushed across several endpoints with the push of a button! Wow, right? Running as many background processes (worker roles) as you need is also supported. I know…I was excited too.

This demo showcases the Windows Azure Datamarket, another service being provided by Microsoft’s cloud which allows you to consume cloud data using a REST service and the OData protocol. Read the whitepaper for more fascinating tidbits. The Azure DataMarket requires your Microsoft Live Id and a data service subscription. Luckily, this demo uses a free Web Crime service provided by Data.gov. There are several others in the store which provide real time data feeds.

Enough chit chat, let’s see this in action. First you’ll want to pull down your Account subscription key and add it to the CrimeService.svc.cs file, i.e.:

private static string azureDataMarketAccountKey = “ey1Oab3KdvlEkIhk=”;

public List GetCrimes()

{

List results = new List();

datagovCrimesContainer svc = new datagovCrimesContainer(new Uri(“https://api.datamarket.azure.com/Data.ashx/data.gov/Crimes/”));

svc.Credentials = new NetworkCredential(“yourLiveId”, azureDataMarketAccountKey);

// Obtain all crime data in New York.

var query = from c in svc.CityCrime where c.State == “New York” select c;

…

Next, fire up the demo in VS 2010 along with the Windows Azure Compute Emulator. This will enable you to watch web roles boot up. Here you can see I have specified two instances in my service configuration (csfg) file. The UI emulator will show your service endpoints along with your role runtime statements.

Once you have this piece in play, you’re almost ready to go. You can fire up your WP7 client application (preferably in another instance of Visual Studio) and see your cloud services at work. You may want to change the ServiceAddress property in the MainPage code behind depending on where the CrimeService web role was deployed, i.e.:

private static string ServiceAddress = “http://127.0.0.1:81/CrimeService.svc/Crimes”;

This demo shows how easy it is to expand your WP7 application into a Azure Datamarket Cloud Consuming Machine! Hope you enjoyed.

MVVM & WP7 Presentation

patrick — Wed, 27 Oct 2010 14:51:19 +0000

I am excited about the chance to share some of my recent work with Dr.Ragade’s Software Engineering class today! We will be covering basics of Model View-View Model pattern, MVVM Light toolkit, and demo’ing some WPF & Windows Phone 7 Applications.

Here are some docs from the presentation:
PPTX: MVVM-WP7_2010

MVVM applications:
WP7 Twitter Search source code
WPF Message Viewer source code
BingMaps WP7 source code

My MVVM Light RTM (release to manufacturing) template:
MvvmLightPhone7Rtm template (place in your Silverlight templates folder)

Some helpful links for getting started with MVVM and Windows Phone 7:

Developer Tools: http://create.msdn.com/
MVVM Light: http://www.galasoft.ch/mvvm/getstarted/
Aaron Stanndard’s blog: http://www.aaronstannard.com/
Bing Maps Silverlight Control: http://msdn.microsoft.com/en-us/library/ff941093(v=VS.92).aspx
Unity: http://unity.codeplex.com/
Rest Client RestSharp: http://restsharp.org/
Dreamspark: https://www.dreamspark.com/Default.aspx
Josh Smith’s MVVM Article: http://msdn.microsoft.com/en-us/magazine/dd419663.aspx
Compiled Experience (several demos): http://compiledexperience.com/windows-phone-7

Hope you enjoy! Let me know if you have any questions.

Hello world!

patrick — Sat, 23 Oct 2010 19:18:12 +0000

Introducing Patrick’s blog:

The latest news in a Louisville Kentucky software engineer’s life. Things I’m likely to post about:
Windows Phone 7
.NET development
Healthcare
Football, Golf, Basketball & other sports
Traveling
Food

Enjoy!