Embedded vs. Non-embedded Tests

Embedded vs. Non-embedded Tests

Post by Tran » Thu, 06 Sep 2007 03:43:00


As far as I know, Facets is the only large project that uses embedded
unit tests. The tests are housed in block comments, eg.

=begin test
...
=end

I have a command line tool that can run the embedded test directly.
For deployment, I have another tool that extracts all the embedded
tests and copies them to stand-alone test/ files so others can run
them. It works fairly well. And the advantage of course is that the
unit tests are right there with the libs they test.

However, considering that just about everyone else is putting units
test in their on test files, I'm wondering if I actually missing out
on superior advantages to that approach. If not why haven't others
pursued better tools for supporting embedded tests?

Thanks,
T.
 
 
 

Embedded vs. Non-embedded Tests

Post by Rick DeNat » Thu, 06 Sep 2007 04:07:01


I think that you're swimming against the tide.

Personally, it just feels right to me to separate the test code from
the code under test, most test equipment belongs in the garage instead
of the trunk of the car.

And when it gets beyond just unit testing individual classes, it's not
apparent to me which file an embedded test should call home.

From a practical tool perspective, there are lots of things which
expect a relationship between code files and test files. Two which
spring immediately to mind are autotest and rcov.

--
Rick DeNatale

My blog on Ruby
http://www.yqcomputer.com/

 
 
 

Embedded vs. Non-embedded Tests

Post by Rick DeNat » Thu, 06 Sep 2007 05:30:22


For the most part, that's instrumentation, not testing.

In the case where sensor information is used to affect the operation
of the vehicle, such as
engine managment, abs, or traction control, it's no more testing
equipment than carburettors, brakes, and master cylinders. Yes, data
gets logged, but that's not testing, any more than a server log file
is.


Yes, but most the testing equipment is not in the car, the engineers
in the pits have laptops, and the heavy stuff is in places like Woking
or Maranello, connected through the 'internets.'

--
Rick DeNatale

My blog on Ruby
http://www.yqcomputer.com/
 
 
 

Embedded vs. Non-embedded Tests

Post by Tran » Thu, 06 Sep 2007 05:57:09


Interesting analogy. We do put some gages right on the dash. But
that's more like embedding the tests directly into code --eg. rescue
and try clauses. So maybe you're analogy is a good one. I can see some
reasons for doing so, it certainly make it easier to send some one a
test, that they can try against their own implementation, for
instance. But then I start to wonder about API documentation too.
Should that also be separated out into dedicated files? Why tests but
not docs?


That's true. Embedded tests only apply for unit test. Integration
tests still need to be is separate files anyway.


That's part of what I was wondering. I've never tried these tools. I
guess I've always suspected they'd be too magical to introduce into an
already established project, and thus lead to more trouble than
they're worth --though I'm sure they're quite useful if one starts
using them from the get-go. Am I wrong about that? But even so,
couldn't such tools be adapted to handle embedded tests too?

T.
 
 
 

Embedded vs. Non-embedded Tests

Post by Rick DeNat » Thu, 06 Sep 2007 06:17:29


Well , it probably seems more obvious to most to put the doc with the
code (assuming that there is any doc <G>). Note that we've just now
seemed to get tools like dcov.

It's long been pretty standard process to have tools which extract doc
from code. On the other hand there are some arguments for keeping
even these two separate. In "The Psychology of Computer Programming"
Gerry Weinberg talked about the trap of "debugging" the comments
instead of the code, and advocated tools which would not just extract
code commentary but REMOVE it.

On the other hand without having the doc with the code and tools like
rdoc we'd probably have even less documentation than we do now, just a
guess.

--
Rick DeNatale

My blog on Ruby
http://www.yqcomputer.com/
 
 
 

Embedded vs. Non-embedded Tests

Post by Terry Poul » Thu, 06 Sep 2007 07:44:56


One thing I am a fan of is reading over my changes at the end of a session and
updating to-do lists, especially when it reminds me clean up my comments and
rewrite quick-fixes to proper code the next day ;-)


It's generally my opinion that the comments tell you what it does and the code
how (with comments for the less obvious). If the comments fall out of sync
with the code, the comments should be updated once the code is less likely to
change to much. Personally I think documentation about using the program
should be separate from the regular source code. Documentation about working
on the program and it's sources should be kept near by. Just how much, is a
matter of developers tastes.


I advocate people using common sense, that is to us the gray matter
approximately three feet above their romp roast.


TerryP.



Email and shopping with the feelgood factor!
55% of income to good causes. http://www.yqcomputer.com/
 
 
 

Embedded vs. Non-embedded Tests

Post by Stefan Rus » Thu, 06 Sep 2007 18:13:36


Take a look at Eiffels Design By Contract. I'm a huge fan of it. Allows
you to move most testing code into the contract, the unit tests are then
merely running your code with contracts enabled.
For an idea on how that works, take a look at the movies with "Design by
Contract" in the title here: http://www.yqcomputer.com/
Would be a very nice feature if ruby had that, even though I'd introduce
additional syntax and I've no idea how to integrate it nicely.

Regards
Stefan
--
Posted via http://www.yqcomputer.com/
 
 
 

Embedded vs. Non-embedded Tests

Post by Alex Youn » Thu, 06 Sep 2007 18:24:18


Allow me to shamelessly copy and paste from an old post of mine:

$ cat testable.rb
module Test; module Unit; class TestCase; end; end; end

module Testable
def test_case(&block)
Class.new(Test::Unit::TestCase).module_eval &block
end
end

$ cat builtin_tests.rb
require 'testable'

class MyWorkingClass
extend Testable

def foo(a,b)
return a+b
end

test_case {
def test_foo_null
assert_equal 2, MyWorkingClass.new.foo(1,1)
end
}
end

if __FILE__ == $0
puts MyWorkingClass.new.foo(1,1)
end

$ ruby builtin_tests.rb
2
$ testrb builtin_tests.rb
Loaded suite builtin_tests.rb
Started
 
 
 

Embedded vs. Non-embedded Tests

Post by Stefan Rus » Thu, 06 Sep 2007 19:07:15


No, the whole approach is different. I'd say the movies I linked to can
explain it far better than I can.
With DBC, if contracts are enabled (in Eiffel you can disable contracts
to get better performance), your method is tested everytime you call the
method. It tests pre-conditions and/or post-conditions (providing you
the old state for your post-conditions). Also after every call to a
method the class invariant is tested to validate your object. That's the
basic idea.

One way that could probably be implemented (though, only as of ruby 1.9
due to &block params) might look this:

class DayOfTime
extend DBC
invariant {
@hour.between?(0,23)
@min.between?(0,59)
@sec.between?(0,59)
}

define(:min=) {
pre_condition { |value|
value.kind_of?(Numeric)
value.between?(0,59)
}
body { |value|
@value = value
}
post_condition { |old, value|
@hour == old.hour
@min == value
@sec == old.second
}
}
end

Regards
Stefan
--
Posted via http://www.yqcomputer.com/
 
 
 

Embedded vs. Non-embedded Tests

Post by Tran » Thu, 06 Sep 2007 19:26:36

 
 
 

Embedded vs. Non-embedded Tests

Post by micatho » Thu, 06 Sep 2007 19:40:17

> Would be a very nice feature if ruby had that, even though I'd introduce

There are a few libraries around that try to do this (google for "ruby
dbc" or similar). Most of these efforts are not entirely convincing.
Unfortunately, the ruby community seems somewhat dogmatic in arguing
against dbc. Maybe because it's not enough of a duck.
 
 
 

Embedded vs. Non-embedded Tests

Post by Stefan Rus » Thu, 06 Sep 2007 20:03:45


I know, but I found none of them convincing enough. Might change with
ruby2, as there are more possibilities then.
IMHO DBC is orthogonal to duck typing. It changes just what you test
for. Instead having a contract that states "argument must be a duck" you
just will have "argument must quack like a duck". I fail to see how that
goes against duck typing.

Regards
Stefan
--
Posted via http://www.yqcomputer.com/
 
 
 

Embedded vs. Non-embedded Tests

Post by micatho » Thu, 06 Sep 2007 20:22:16

> I fail to see how that goes against duck typing.

Me too.