Christopher Bird » Software Development

JavaScript Multiple Callbacks

Chris — Mon, 21 Jun 2010 15:32:12 +0000

On a previous project we were writing a web UI using a lot of javascript. The UI was constructed in an MVP style with many Views, Presenters and Models all active at the same time. One of the problems we had was with having multiple Presenters subscribed to listen for a single callback from a Model.

Initially we had quite an ugly solution which involved chaining functions in a similar fashion as to DOM events were handled in the pre jQuery days. To do this we used the following function…

Example usage…

someObject.someEvent = addEventHandler(someObject.someEvent, this, doSomething);
function doSomething() {
}

After some refactoring we created an Event class (semi inspired by Node.Js EventEmitter). This seemed like a nice way to allow us to simplify the syntax, make it easier to read and attempt to use a familiar syntax to listeners else where in JavaScript.

Example usage…

someObject.someEvent.addListener(doSomething);
function doSomething() {
}

NodeJS Testing

Chris — Fri, 30 Apr 2010 16:05:14 +0000

First thing that struck me when i started working with NodeJS was a lack of anything to run my tests with. I wanted to be able to write tests, run them, tell me which one failed and why. I thought this may be usefull for others so here is “Nest” my node testing module that works with the existing assert module.

Nest @ GitHub

Example

var nest = require("./nest");
var assert = require("assert");

var testCases = new nest.Nest();
testCases.add("TestSomething", {
test_true_is_true: function() {
assert.equal(true, false);
}
});
testCases.run();

NodeJS and Scope

Chris — Sun, 14 Mar 2010 18:51:03 +0000

Scope is a little different in NodeJS but only in a artificial sense. let’s dive into a quick example of what i mean.

Circle.js

var PI = 3.14;
exports.getPi = functon() {
return PI;
}

If you were to include this file into a browser you would expect to be able to access the variable PI from another file (ignoring the syntax error which would be created by assigning to the undefined object “exports”). This is not the case in NodeJS, when you call require to include this script the contents of the file is inserted into a function as follows…

var wrapper = "(function (exports, require, module, __filename, __dirname) { "
+ content
+ "\n});";

By calling this wrapper function and passing in an object to the exports parameter, your exported variables/functions are added to the exports object which is then returned as a result of a “require” call.

Because your exported functions are defined inside another function (the one generated when the script is imported) any variables (like PI) are still accessible by your exported functions due to JavaScripts normal scoping. Take a look at JavaScript closures for more information.

Conclusion

Each script can access what you “require” into it and what you define in it.
Requires are cached so every call to require for the same module should get a reference to the same thing
Finally this is my current understanding, ill update as i learn, if you can enlighten me any further leave me a message

New validation rules

Chris — Sat, 16 Jan 2010 17:23:16 +0000

Few conversations with colleagues later and a few small changes to the validation framework.

Fixed typo in adding custom rules test.
Added “length less than”, “length greater than” and “is a number” rules
Updated example to include some of the new validation rules

What next?

Date validation seems to be one that keeps cropping up. DateJs provides a great library for dealing with all sorts of date formats. Im considering whether this should be an optional set of rules as the sizes of DateJs would negate the bennifit of the small validation core if included with it. I’m also considering an optional set of patterns for common international problems (e.g.Email address and Uri), if they are small enough perhaps they should be in the core as well.

Javascript validation

Chris — Sun, 10 Jan 2010 20:02:49 +0000

To quote Eddie Izzard “Look, you’re British, so scale it down a bit, all right?”, well perhaps i should say “Look, you’re a validation framework, so scale it down a bit, all right?”. Recently i’ve been working on ALOT of javascript and experiences (reassuringly) from the past have come back to haunt me yet again. What am i talking about? Well it would seem that every framework starts out with the good intention trying to avoid repetition, but over time, someone adds a little here, and a little there and before you know it you have a behemoth of a framework which does things well out side the scope of what it should do.

How does this relate to javascript validation? Well after using quite a popular JQuery validation framework i noticed the size of the library minified as 29KB. This doesn’t sound a great deal but when you include a few frameworks like this, plus JQuery the page weights seem to get pretty big. Yes caching, gziping and content delivery networks will help but the first hit on these pages over mobile networks or on mobile phones and they can take quite a while to load. What i would like to see more of is smaller more directed frameworks/utilities that i can customise into what i want and keep page sizes to a minimum.

To this end i flippantly stated “It can’t be that hard” and embarked on writing a validation framework that will allow me to do most of what i need without the overhead. This is only a first stab that works for what i need, any comments on other validation types that i should support would be good.

Currently it supports… required (and ability to define a default value to ignore), patterns, less than, greater than, equal to, equal to other field and the ability to add custom rules. On top of this you can all specify rules that enable and disabled validation of a field. Currently the minified size stands at about 2k even tripling the number of rules would only take that up to about 4k.

I got the size down by only putting into the framework what really needed to be there. So what won’t it do? well it wont manage form click events for you, thats for you to decided how you want the validation to fire. It won’t do anything clever with the error messages, if you want something fancy write what you want.

Take a look at the code on git hub http://github.com/chrisabird/validate

Retlang, some examples and explanations

Chris — Sun, 13 Dec 2009 22:45:16 +0000

Last week I spent some time trying out the message based concurrency library Retlang in and effort speed up a portion of an application I’ve been working on. When picking up Retlang I found very few examples/explanations that were relevant to the current release (0.4.3.0).

In the beginning

Retlang is constructed of various channels that can have messages published to them. A channel also has fibers subscribed to them with a delegate to a function that accept a message as a parameter. So create a channel, subscribe a fiber to it and publish your messages to the channel and the fiber will process them one after another leaving your program free to do other things, Simple yeah? not one thread, mutex, monitor or lock in sight, don’t believe me how about an example?

using(var fiber = new ThreadFiber())
{
fiber.Start();
var channel = new Channel<string>();
channel.Subscribe(fiber,
value => {
new WebServiceClient.SendString(value);
}
);
channel.Publish("hello");
channel.Publish("world");
}

Talk to me!

Ok so you might have noticed these published messages are asynchronous, you get nothing back. There are two solutions use the RequestReplyChannel instead of Channel which will process your message and wait for a response with each call. That’s gonna slow things down though, so instead we’ll publish the result on to another channel and pick them up after. Here goes…

using(var requestFiber = new ThreadFiber())
using(var responseFiber = new ThreadFiber())
{
requestFiber.Start();
responseFiber.Start();
var requestChannel = new Channel<KeyValuePair<int, string>>();
var responseChannel = new Channel<KeyValuePair<int, bool>>();
requestChannel.Subscribe(
requestFiber,
value => {
responseChannel.Publish(
new KeyValuePair<int, bool>(
value.Key, value.Value.Contains("world")
)
);
}
);
var results = new List<KeyValuePair<int, bool>>();
responseChannel.Subscribe(
responseFiber,
value => {
results.Add(value);
}
);
requestChannel.Publish(new KeyValuePair<int, string>(1, "hello london"));
requestChannel.Publish(new KeyValuePair<int, string>(2, "hello world"));
}

You said concurrent

Ok so the previous example used one fiber to find strings that contained the word “world” and another to collate them after. It’s hardly concurrent, still feels a bit sequential. In the example we can’t really add any concurrency to the collation of those responses, well not without locking that results list, plus that’s a very simple/fast operation anyway. What we can do is speed up the finding of the word “world” in strings because there is no shared state, that’s simple to do. The channel will allow you to subscribe as many fibers to it as you wish, only problem is every fiber will be called for each message. This doesn’t really do what we want, we’ll end up with many results for each message, what we really want is the first free fiber to process the next published message. Step in the QueueChannel which does just that, so same example but with more “fiber”!

using(var requestFiber1 = new ThreadFiber())
using(var requestFiber2 = new ThreadFiber())
using(var responseFiber = new ThreadFiber())
{
requestFiber1.Start();
requestFiber2.Start();
responseFiber.Start();
var requestChannel = new QueueChannel<KeyValuePair<int, string>>();
var responseChannel = new Channel<KeyValuePair<int, bool>>();
requestChannel.Subscribe(
requestFiber1,
value => {
responseChannel.Publish(
new KeyValuePair<int, bool>(
value.Key, value.Value.Contains("world")
)
);
}
);
requestChannel.Subscribe(
requestFiber2,
value => {
responseChannel.Publish(
new KeyValuePair<int, bool>(
value.Key, value.Value.Contains("world")
)
);
}
);
var results = new List<KeyValuePair<int, bool>>();
responseChannel.Subscribe(
responseFiber,
value => {
results.Add(value);
}
);
requestChannel.Publish(new KeyValuePair<int, string>(1, "hello london"));
requestChannel.Publish(new KeyValuePair<int, string>(2, "hello world"));
}

Those threads are a bit heavy

At the moment i’m using the ThreadFiber which maps down to a Thread. So for every ThreadFiber we’re creating new threads which is expensive so instead we might want to use the PoolFiber which uses the .Net ThreadPool to manage creation and reuse. It’s a very simple change to use the PoolFiber, just replace the ThreadFibers. The only difference between ThreadFiber and PoolFiber is that it will batch up a few messages to be processed in a single thread.

Pause for breath

This seems like a good place to stop for now. There are a few more things to cover in more detail like batch execution and some other things like customising execution but those can wait till another day.

Updated

I’ve updated the last to examples to solve a type typo.

Lessons learnt from acceptance testing… again

Chris — Wed, 09 Dec 2009 22:26:48 +0000

After writing my last post i’ve been thinking more about the question of “How should i organise my tests?”. The answer i gave last time seemed to work for fixing existing tests but didn’t seem to fix the cause of the problem that all the tests were still too much like unit tests. Oh yeah, at this point i’d like to point out i reserve the right to contradict myself.

While working on a separate RoR project I’ve used Cucumber to drive acceptance tests. If you’ve not tried it already, it’s great give it a go! Also on a side note i also discovered the guys who wrote selenium-client put a great deal more thought into the API than the .NET counterpart. Lots of extensions for waits and JQuery/Prototype integration are included by default. Anyway back to the problem at had, Cucumber forces you to write plain text stories (or features) and one or more scenarios for that story. Scenarios are written with each line start with “Given”, “When” and “Then” (and optionally And). Each line of the scenario corresponds to an executable “Step” that in my case performed a selenium action (or it could also quite easily perform an action against an API).

Feature: Manage blog posts
In order to manage my blog posts
As a writer of poor blog posts
I want to be able to create blog posts

Scenario: Create a blog post
Given i have clicked add blog post
And i have written "My first post" with content "My first post"
When i click publish
Then i should see the post "My first entry" in the list of posts

Anyway this method of writing test with the emphasis on the behaviour of the user (not surprising for a BDD tool) meant i ended up with less focus on one bit of functionality on a single page and more on the overall journey of the users actions through the application. Each scenario taking a different path or journey through that story.

Overall cucumber forced me to stay true to testing behaviour, avoiding the pitfalls of the test written in a plain unit testing framework. So, try cucumber, even if you don’t use it in your project i hope it makes you think differently about how you write the same tests in your native language.

Lessons learnt from acceptance testing (with Selenium)

Chris — Fri, 20 Nov 2009 22:34:32 +0000

While recently working with automated acceptance tests I noticed two major problems. The tests had started to intermittently fail and were taking longer and longer to run. With a loss in confidence of the tests they started to fall into disrepair.

This post tries to cover some of the root causes of our problems and try to suggest practices to help avoid these problem in the future. While I talk about Selenium most of these practices in theory could be applied to any automation package…

What should I test with Selenium?

Actually this should really be where are my tests best placed? A lot of acceptance tests seemed to delve too deeply into the functionality of a story which partially leads to the problem of too many/long running acceptance test. Try to keep acceptance tests at a high level. Anything of the story that you can test in unit tests, test it there – don’t repeat yourself in the acceptance tests.

Try not to use Selenium as a tool to unit test Javascript, this again dramatically increases the number of tests you have and can become very brittle when waiting for expected DOM changes. Use an appropriate Javascript unit testing tool like JsTestDriver.

How should I organise my tests?

One of the key problems to organising tests as if they are single unit tests is they tend to navigate to a page, change something, assert something happened then the close page. Now write 25 tests for one page and for every tests you have to wait for the page to load, which as you many expect seriously increases the time taken to run the tests. This is especially a problem testing functionality based entirely in Javascript which is relatively quick to test.

Where possible group all tests for a page and browse to the page once. This does leave the problem of tests effecting each other but if you remember to follow a pattern of, change something, assert something and then reset the page you shouldn’t have any problems.

Something else that I came across was when testing functionality on a page that was in a sequence after others, you always ended up completing the first page and then moving to the second before starting your tests. Now think about organising your tests where for each test you are completing the first page to get to the second before performing your test, they start to take a very very long time to run. This leads to organising the tests into journeys through the system (or groups of related stories that can be run in sequence) allowing us to tests each criteria as we navigate through the system.

How should I organise the code that interacts with my web application?

Use a page model. Construct classes that represent pages or popup windows in your application that have an interface that reflects the user interface. For example a form page that takes a name may have a page model with properties for title, first name and surname that when set or got perform the relevant selenium actions on the browser to get or set the value. This has a couple of benefits, you can reuse these page models across tests and they can be easily replaced if a better automated testing tool comes along. Another point to note is to make sure your page model interface follows a single semantic, e.g. use properties to get/set values and methods to perform actions like clicking a button. Tests tend to become very confused and actions tend to get repeated if there is no common way of defining the interface.

How do I know when something has happened on the page?

It’s very important to know when something has happened on the page (especially on Javascript heavy pages), if you don’t, expectations become very brittle when they check to early or to late. Hopefully your first reaction is not to start putting Thread.Sleep()’s all throughout your tests, what happens when something takes longer than you expected? The best way to solve this problem in Selenium is to use the built-in WaitForCondition(condition, timeout) method. You can give a condition in Javascript which has access to the client side Selenium object. For example you can do things like this…

selenium.WaitForCondition(
"selenium.isElementPresent('result-div')",
"1000");

The important difference between this and Thread.Sleep() is that you can give longer timeout periods. A Thread.Sleep() will always make your tests wait for that period of time where as WaitForCondition will move on once the condition is met. This allow you to account for differences in the speed tests run without making your tests take longer and longer to run when increasing Thread.Sleep times.

Summary

Test where it is most appropriate.
Group tests into journeys where possible.
Hide selenium browser interactions behind page models.
Don’t Sleep just WaitForCondition’s to happen.

Equals Builder for c#

Chris — Sat, 31 Oct 2009 14:16:58 +0000

I recently did a direct port of the java EqualsBuilder into c#. After some experimenting i’ve created a new version for c# that takes advantage of some of language features that java doesn’t have.

Usage

EqualityCheck.Of(
stringA.And(stringB),
intA.And(intB),
decimalA.And(decimalB),
objectA.AndPropertiesOf(objectB))

Code is available for download from here. Download

Simple alternative to enums in C#

Chris — Sat, 31 Oct 2009 13:06:13 +0000

I’ve seen quite a few solutions for extending enum’s in c#, some of them quite complicated. One of the more common is adding additional attributes to store more information. The solution i prefer for this is not to use enum’s at all, e.g.

public class Title {
public static readonly Title Mr = new Title(Gender.Male);
public static readonly Title Master = new Title(Gender.Male);
public static readonly Title Miss = new Title(Gender.Female);
public static readonly Title Mrs = new Title(Gender.Female);

public Gender Gender { get; private set; }

private Title(Gender gender) {
Gender = gender;
}
}

One another bonus is it avoids temptation of using switches to perform different functionality on each item of an enum. Instead it forces you to separate functionality using patterns like chain of responsibility.