Pain and Glory from the Trenches of the IT World

Several days back I wrote about how unit testing is not a substitute for static typing. A comment posted to that article by James asked for more clarification regarding what I was talking about. James wrote, “I can’t recall the last time I had Ruby code break because I tried to act on an object of the “wrong” type.” Well, I will give a simple example of how such problems arise, and how different languages deal with them. The languages in question will be Ruby, Python, OCaml and Haskell.

Our example program will be simple. It will consist of two functions. One will be called ‘test’, it will take two integers, and it will return the arithmetic sum of them. The other function will be called ‘main’, and it will invoke the ‘test’ function in two different ways, depending on the size of the array or list containing the command line arguments passed to the program. If there are over three command line arguments, the result of the ‘test’ function applied to the values 1 and the function ‘test’ is returned. Otherwise, the value of ‘test’ applied to values 1 and 2 is returned. Thus the program should have a return value of 3 up until the array or list storing the command line parameters has more than 3 elements.

Since James mentioned Ruby, let’s start with the Ruby version of the code:

def test(a, b)
  a + b
end

def main()
  if ARGV.length > 3
    test(1, test)
  else
    test(1, 2)
  end
end

Process.exit(main())

And now we’ll run it, with warnings enabled and set at the most verbose setting:

$ ruby -w -W2 t.rb; echo $?
3
$ ruby -w -W2 t.rb 0; echo $?
3
$ ruby -w -W2 t.rb 0 1; echo $?
3
$ ruby -w -W2 t.rb 0 1 2; echo $?
3
$ ruby -w -W2 t.rb 0 1 2 3; echo $?
t.rb:7:in `test': wrong number of arguments (0 for 2) (ArgumentError)
        from t.rb:7:in `main'
        from t.rb:13
1
$

As is expected from a dynamically typed language like Ruby, the error wasn’t detected until runtime. Not only that, but the program ran quite successfully before then, without giving any indication that a hidden problem might arise were too many command line arguments given. Even were unit tests to be used, it is quite possible that duplicating such a scenario would be missed, and a perplexed user would be faced with an error such as the one above.

Python doesn’t fare much better. Here is the code we’ll use for it:

"""docstring"""
import sys

def test(first_arg, second_arg):
    """docstring"""
    return first_arg + second_arg

def main():
    """docstring"""
    if len(sys.argv) > 3:
        return test(1, test)
    else:
        return test(1, 2)

sys.exit(main())

Just to be safe, that code was run through the pylint utility, which rates the above code as “10.00/10″. So an unsuspecting programmer may believe they have written high-quality Python code, when they surely have not, as we will soon see when we go to run the code:

$ python -W error t.py; echo $?
3
$ python -W error t.py 0; echo $?
3
$ python -W error t.py 0 1; echo $?
3
$ python -W error t.py 0 1 2; echo $?
Traceback (most recent call last):
  File "t.py", line 15, in ?
    sys.exit(main())
  File "t.py", line 11, in main
    return test(1, test)
  File "t.py", line 6, in test
    return first_arg + second_arg
TypeError: unsupported operand type(s) for +: 'int' and 'function'
1
$

Python behaves similarly to Ruby. The error isn’t caught until runtime, and there’s a good chance that unit testing would not have caught it, as well.

Let us try OCaml, a statically typed language. Compiling:

let test a b =
  a + b;;

let main _ =
  if (Array.length Sys.argv) > 3 then
    test 1 test
  else
    test 1 2;;

exit (main ());;

gives the following error:

$ ocamlopt -w A t.ml
File "t.ml", line 6, characters 11-15:
This expression has type int -> int -> int but is here used with type int
$

The OCaml interpreter also catches the error, even without the code being executed:

$ ocaml -w A t.ml
File "t.ml", line 6, characters 11-15:
This expression has type int -> int -> int but is here used with type int
$

Unlike when using Python or Ruby, the OCaml compiler and interpreter catch the error before the code begins to execute. And note that this is done without any source-level type annotations. This compile-time failure forces the programmer to deal with the error, rather than the user. Thus we end up with a more reliable program. Not only that, but the error was caught without the developer having to write even a single unit test. Now instead of writing unit tests to check if his or her code types correct, the developer can write unit tests to check the actual functionality of his or her software.

We can write a similar program using Haskell, another statically typed language, and it will also catch the error during compilation:

import System(getArgs)
import System.Exit

test :: Int -> Int -> Int
test a b = a + b

main :: IO ()
main = do
  args <- getArgs
  if (length args) > 3
    then exitWith (ExitFailure (test 1 test))
    else exitWith (ExitFailure (test 1 2))

When we go to compile the above program using GHC, we get:

$ ghc -Wall t.hs 

t.hs:11:39:
    Couldn't match expected type `Int'
           against inferred type `Int -> Int -> Int'
    In the second argument of `test', namely `test'
    In the first argument of `ExitFailure', namely `(test 1 test)'
    In the first argument of `exitWith', namely
        `(ExitFailure (test 1 test))'
$

The same error is given by ghci, the interactive REPL of GHC. As with OCaml, the problem is caught at compile time, rather than runtime. Thus the developer must actively deal with it for his or her program to just compile, let alone execute. This helps increase the program’s reliability.

As we have clearly seen above, dynamically typed languages like Ruby and Python can allow for some flawed code to be written with ease. But more dangerously, it is possible for the code to run just fine, until a certain context arises upon which a runtime error occurs. Even when running the interpreter with warnings enabled, or after using a code-checking tool like pylint, such problems go completely undetected. This deception is dangerous. Some developers think that unit testing for such situations is appropriate, but there’s a very good chance that such typing errors won’t be detected by the unit tests, either.

It can be said that the code above is unrealistic. That’s true. But the scenario it simplifies is very real. Based on my own experience, I have seen far too many problems arise with dynamically typed languages, where rarely-executed code contains a type-related error, and the execution of this code causes a runtime error (usually at a most inopportune time).

Thankfully, statically typed languages provide a very natural way of avoiding such problems at runtime, instead having them be caught at compile-time. When using languages like Haskell and OCaml, that support type inference, it’s not even necessary for the developer to specify the types in the source code. So such developers get the convenience of languages like Python and Ruby, but without the inherent runtime danger of those languages, and without the inconvenience of having to write unit tests to handle checking that a proper compiler can do automatically, and more rigorously than a human could.

(UPDATE: Fixed mangled Haskell code.)

Email This To A Friend or Social Bookmark It!

This entry was posted on Sunday, February 17th, 2008 at 5:32 pm and is filed under Development, Haskell, Python, Ruby, Software and. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.