Expert Skills update

The Expert Skills book has now been officially published and is being listed by Amazon. It may be a few days before Amazon has the book in stock, but it is available for pre-order in the UK.

Expert Skills update

The Expert Skills course is currently in the final stages of proofreading.

After proofreading, the book will need to be indexed before a test print run can go ahead. Assuming there are no problems with the test print, it will then be ready to go on sale.

Everything is on schedule for a June release.


Markov chains in ASP.NET

My previous article covers the concepts behind Markov chains. In this article, we’ll look at how Markov chains can be generated using C# code. You can  download the complete source code of this example on the Downloads page. To start … Continue reading

Markov Chains Explained

A couple of years ago, I worked on a natural language processing application along with fellow C# developer Michael Cronin. While working on this, I did extensive research into Markov Chains. Although there are many articles on the internet, I found that none of them did a good job of explaining the concepts behind Markov Chains.

Markov Chains are used to produce random text that still largely conforms to the rules of the English language. This random text is sometimes used by authors and musicians as a source of inspiration, although it is also used by spammers to trick spam filters.

Markov chains are generated by analzying a piece of text to determine the probability of each word. Here’s an example, calculating the probability data for the word Jack in the nursery rhyme “Jack and Jill”:

Jack and Jill went up the hill to fetch a pail of water
Jack fell down and broke his crown
And Jill came tumbling after.
Up got Jack, and home did trot
As fast as he could caper
He went to bed and bound his head
With vinegar and brown paper.

There are three instances of the word Jack in the text. There are two cases where Jack is followed by and, and one instance where it is followed by fell.

A Markov chain sees this as a 66% chance (2/3) of the word Jack being followed by the word and, and a 33% chance (1/3) of Jack being followed by the word fell.

To produce a Markov chain, you need to calculate probabilities for every word in the text. This produces the following probability data:

and Jill (33%) broke (16%) home (16%) bound (16%) brown (16%)
Jack and (33%) fell (66%)
as fast (50%) he (50%)
he could (50%) went (50%)
his crown (50%) head (50%)
Jill went (50%) came (50%)
to fetch (50%) bed (50%)
up the (50%) got (50%)
went up (50%) to (50%)

All of the other words in the rhyme are followed by only one word (100% probability).

Let’s now create some example chains from this data, using the word Jack as a starting point. We’ll stop at 7 words for the sake of simplicity.

Jack fell down and home did trot  (80%)
Jack fell down and brown paper (76%)
Jack fell down and Jill came tumbling (75%)
Jack fell down and broke his crown (72%)
Jack fell down and broke his head (72%)
Jack fell down and bound his crown (72%)
Jack fell down and bound his head (72%)

All of these chains could potentially be generated from the text, although some are more likely than others.

Here’s a breakdown of the first chain:

Jack has a 66% chance of being followed by fell
fell has a 100% chance of being followed by down
has a 100% chance of being followed by and
has a 16% chance of being followed by home
has a 100% chance of being followed by did
has a 100% chance of being followed by trot

The average of these percentages is around 80%.
The actual probability of this result is around 10%.

Online marketing by the numbers

Online marketing has become essential to do business in the internet age, but many businesses are paying over the odds for marketing. It’s notoriously difficult to calculate return on marketing spend, but I have created a mathematical formula that works well to benchmark spending.

Formula for return on marketing investment:

( ( [Number of impressions] / 100 / 100 ) * [Price of product] )
– [Cost of campaign]

The formula is quite similar when paying per click:
( ( [Number of impressions] / 100 ) * [Price of product] )
[Cost of campaign]

The formula depends on two assumptions:
Firstly,  that online ads will have an average click-through rate of 1%. This means that for every 100 people who see the ad, one will click through to the website.
Secondly, that 1% of visitors to the website will buy the product.

Studies have shown that click-through rates vary from around 0.5% to around 3%, averaging at around 1%.  Conversion rate studies show a greater variance (from less than 1% to 8%), but I have found 1% to be a reliable benchmark.

In this example, I am paying £1,000 for 100,000 impressions and selling my products for £30 each:

100,000 / 100 / 100 = 10 expected sales
10 * 30 = £300 sales value
£300 – £1000 = -£700

As you can see, this deal would result in a loss of £700 if I were to accept it. For this deal to be viable it would have to cost less than £300 (£0.003 per impression).

Formula for maximum price per Impression

[Max Price per impression] = [Price of product] / 100 / 100
[Max Price per click] = [Price of product] / 100

It’s often more useful to look at the amount paid per impression or per click.  Using this formula you can calculate a break-even price based on the price of your product.

For a product that costs £30:
[Max Price per impression] = £0.003
[Max Price per click] = £0.30

For a product that costs £100:
[Max Price per impression] = £0.01
[Max Price per click] = £1

If your product sells for £100 and you are spending more than £1 per click or £0.01 per impression, you are probably making a loss on your marketing spend!

ASP.NET and security

In the course of writing the Expert Skills course, I’ve recently written lessons about SQL injection and cross-site scripting attacks.  What struck me when writing these lessons is just how secure ASP.NET is.

It’s extremely difficult to make an ASP.NET web application vulnerable to cross-site scripting attacks. A developer would have to manually disable ASP.NET’s security features before an attack would even be remotely possible. Even then, most of ASP.NET’s controls will automatically filter out any attempt at cross-site scripting.

SQL injection attacks present a slightly greater risk, as it’s still possible to create a vulnerable application if you send SQL code directly to the database. Standard LINQ, however, provides complete immunity to SQL injection.

It really is impressive how ASP.NET’s protects your applications from the most dangerous forms of attack, even when using its most basic features.

Expert Skills update

Another update has just been made to the Expert Skills course outline.

The editing and indexing process will soon begin, changing the book from a first draft into a polished and complete course.

Expert Skills update

The Expert Skills course is progressing well and the course outline has been updated again.

There’s now an entire session dedicated to crpytography and threading, as there was just too much to cover for them to be grouped with the other advanced .NET Framework features.

The Cube – An Interactive Fiction game

Back in 2010, I co-authored an Interactive Fiction game for the Casual Gameplay Design Competition #7. That game was called The Cube.

Although The Cube only placed 12th out of the 30 entries, I’m still very proud of it. It was written using a special purpose programming language called Inform.

You can still play the game online, and you can download the source code and other game files from the downloads page.

How is C# threading like peeling potatoes?

I’ve been working on the Threading lessons for the C#, ASP.NET and Visual Studio Expert Skills book, and an interesting analogy struck me.

I have a sack of potatoes that need to be peeled as quickly as possible. I also have two chefs, Jo and Bill, who are in kitchens on opposite sides of a building.

In the first scenario, I deliver the potatoes to Jo and Bill one at a time, running backwards and forwards from the sack. It takes so long for me to carry each potato to the chefs that it would actually be faster to peel them myself.

This is the same when working with C# threads. Using two threads to accomplish a lot of small tasks is actually slower than doing all of the tasks using a single thread because of the overhead that’s added by managing the threads.

This code uses the “single potato” approach to check prime numbers up to MaximumValue:

while (CurrentValue < MaximumValue)
    if (!FirstThread.IsAlive)
        FirstThread = CreateThread();
    if (!SecondThread.IsAlive)
        SecondThread = CreateThread();

This code gives each prime number to the two threads individually. The prime numbers are the ‘potatoes’ in this instance. Because there’s so much overhead starting and stopping the threads, this code is actually much slower than it would be if it didn’t use any threading code at all!

So what’s the solution to the potato problem?  Well of course, it doesn’t make any sense to carry the potatoes one at a time. It would be much more logical to give half of the potatoes to Bill and the other half to Jo. That way they can do their jobs and there’s no time wasted running backwards and forward from the potato sack.

In C# terms, that means our solution is to divide up the prime numbers between our two threads and have each thread process them all in one go instead of starting a new thread for each number.

Here’s how the code looks with the prime numbers divided up between the two threads.

Thread Thread1 =
new Thread(new ParameterizedThreadStart(CalculatePrimes));
Thread1.Start(new int[]{2,StartingPointForThread2-1});
Thread Thread2 =
new Thread(new ParameterizedThreadStart(CalculatePrimes));
Thread2.Start(new int[] { StartingPointForThread2, MaximumValue });

while (Thread1.IsAlive || Thread2.IsAlive)

In this code, the two threads only start once and don’t need to restart for each prime number. This implementation is much faster than working with a single thread (or peeling all of the potatoes yourself!).

There’s a lot more about threading in the Expert Skills book, including using the ThreadPool class to manage threads more easily. ‘Learn ASP.NET 4.0, C# and Visual Studio Expert Skills with The Smart Method’ will be available for purchase around June 2012.