Are the test programs complete?

[burge91]

Just wondering if I can rely on them such that any program I write that passes the test is a valid implementation. Particularly with the ALU.

[ivant]

Tests which check for every possible values of the parameters (and state for stateful systems) are impractical or even impossible for all but the most simple systems. Instead, tests try cover enough common cases and all the corner cases.

The ALU has 6 1-bit and 2 16-bit inputs. All possible inputs would be 2^(6+16+16) = 2^38 = 274877906944 possible inputs. The ALU test define just 36 checks.

[burge91]

So is it possible to write a HDL program that passes the tests but that is not a correct implementation of the chip?

[cadet1620]

Most of the tests are fairly simple and are only checking for gross errors.

For example the ALU test checks that each of the documented functions generates correct results for two different pairs of values. It assumes that the underlying parts are correct. It does not test that your implementation handles the control bits exactly as specified in all combinations.

Some of the tests, particularly the Xxx16 parts where the HDL has many copies of the same line but with different bus indices, could do a better job of testing that every one of the indices is correct.

So is it possible to write a HDL program that passes the tests but that is not a correct implementation of the chip?

Easy to do this intentionally, less likely to do it accidentally if all the underlying parts are good, except that the zr status is not tested for every out bit handled correctly.

    Not16(in[0..7]=x[0..7], out=notx);

--Mark

[burge91]

seems my problem with the ALU was that in the description of f ("if f then out x+y else out x&y) i confused that description with the behaviour of a multiplexor and made the first input of the multiplexor i used as the first output of that description, when it actually needs to be the other way around (for the second test, testing f(x,y) = 1). When i got those multiplexor inputs fixed (swapped around) the test ran through until computing f(x,y) = y-1, so it passed 10 tests just from one variable change, but it has not passed them all, which is the next problem i need to solve, which will be fun :) thanks for your inputs

[cadet1620]

If you are not doing so, you should test with ALU_nostat.tst first.  That test only tests the computation part of the ALU. Once that passes then it is easier to debug problems with zr.

--Mark

[burge91]

Thanks I will try that as I didn't really know what that program was for.

It would be nice if someone made some more exhaustive test files for the programs as I sense I am getting false positives in some of my programs.

[cadet1620]

If you want to, mail them to me in a zip file and I'll take a look at them and return comments.

--Mark