Adjusting command line execution for all the tests

[longj_silver]

Greetings to everybody!
I am considering the idea to propose the course (and the tests) to a large number of students, therefore I was preparing some batch scripts (Windows7) in order to automatically verify the students' work. Actually I rely on command line executions of the simulators and I stumbled on testing the Memory chip by means of HardwareSimulator. When I execute it in command line mode, it seems to indefinitely loop: probably that is due to the test mechanism requiring for this chip to press a couple of keys.
I was wondering if there is any workaround for such a problem. I took the on-line course some time ago and I remember that I was able to submit all my chips for grading. Since I believe that they were automatically verified, I suppose there could be a way to adjust the command line mode even for the Memory chip.
Many thanks for all the help you are going to provide me!

[cadet1620]

[Updated files. Test now reads back Screen to ensure that the horizontal lines are present.]

longj_silver wrote

I rely on command line executions of the simulators and I stumbled on testing the Memory chip by means of HardwareSimulator. When I execute it in command line mode, it seems to indefinitely loop: probably that is due to the test mechanism requiring for this chip to press a couple of keys.

Memory.tst is definitely hanging because the keys are never being pressed.

I don't know how the Coursera grader works; I'm not involved with that.

The quickest solution would be to remove the code in the test script that is waiting for a key and modifying the compare file to match the new output, but this is non-ideal because the Keyboard interface in Memory.hdl then goes untested.

The better solution is to give the test scripts some way to press keys. I took a look at the source code and saw that all the infrastructure for the built-in Keyboard part to recieve values from the test script's "set" command is already there; it just raises an "unimplemented" exception.

So I've spent the last hour or so hacking code... (Warning, I'm not a Java programmer; somebody who is should vet my changes!)

So, here's a Keyboard part that tests can drive, and a test script that will work from the command line.

Keyboard.java		Source code with annotated changes.
Keyboard.class		Replaces file in nand2tetris/tools/builtInChips
MemoryNoKB.tst		Test script for Memory.hdl with automatic key presses.
MemoryNoKB.cmp		Compare file for MemoryNoKB.tst.

Keep a copy of the original Keyboard.class in case mine is buggy.

Test script examples:

    set Keyboard[] 75;  // press 'K'
    set Keyboard[] 0;   // release key

--Mark

[longj_silver]

Dear Mark,
your help went beyond my best expectations!
After a quick attempt it seems that everything works fine and my problem appears to be solved.
I am not a Java programmer either, but I will take a look at your code. If I understood well you incorporated the "key-press" into the test itself (just like a human user who is interactively pressing on the keyboard), didn't you?
Maybe the only open issue is the screen test, where the user is called to verify the presence of 2 horizontal lines in the middle of the screen, but I think this could be considered a minor drawback.
Thank you very much for your kind and quick support.

[cadet1620]

longj_silver wrote

 If I understood well you incorporated the "key-press" into the test itself (just like a human user who is interactively pressing on the keyboard), didn't you?
Maybe the only open issue is the screen test, where the user is called to verify the presence of 2 horizontal lines in the middle of the screen

Exactly. Just before the while loops waiting for keyboard input I added a set command to press the appropriate key.

Good catch. I'll add a readback check for the lines to the test later this afternoon and post an update.

// Keyboard test

set address 24576,
echo "Click the Keyboard icon and hold down the 'K' key (uppercase) until you see the next message (it should appear shortly after that) ...",
// It's important to keep holding the key down since if the system is busy,
// the memory will zero itself before being outputted.

set Keyboard[] 75;        <================== 'K' pressed here

while out <> 75 {
    eval,
}

clear-echo,
output;				// checks KB input value
set Keyboard[] 0;         <================== 'K' released here

// Screen test

--Mark

[cadet1620]

cadet1620 wrote

Good catch. I'll add a readback check for the lines to the test later this afternoon and post an update.

The files in the original post have been updated.

Note that there is now a MemoryNoKB.cmp file.

--Mark

[longj_silver]

Dear Mark,
I have been reading your files and I compared them with the original ones. Actually, at first I tried to get a flavour of the test script language in order to fix some points in my mind (until now I did not linger on this topic).
It seems to me that you verify the appearance of the lines on the screen by introducing a couple of test cases in MemoryNoKB.tst: in practice you read (load =0) from the very same addresses previously set for writing in the RAM (the addresses should be X4FCF and X504F, which memory-map towards the screen). As concerning the in-value 9999 I believe it could be simply a dummy entry without any usefulness. Accordingly, you modified the .out file so that the expected "out" results should be equal to -1 (i.e. the black pixels to draw the lines). Probably the .java and .class files remained unchanged.
Is this correct?
I beg your pardon for taking advantage of your patience, but I would like to fully understand your novel test implementations, rater than simply (and blindly) applying them. In this way, I hope I will be able to realize something useful in case of future necessity.
Many thanks!

[cadet1620]

longj_silver wrote

Dear Mark,
I have been reading your files and I compared them with the original ones. Actually, at first I tried to get a flavour of the test script language in order to fix some points in my mind (until now I did not linger on this topic).
It seems to me that you verify the appearance of the lines on the screen by introducing a couple of test cases in MemoryNoKB.tst: in practice you read (load =0) from the very same addresses previously set for writing in the RAM (the addresses should be X4FCF and X504F, which memory-map towards the screen). As concerning the in-value 9999 I believe it could be simply a dummy entry without any usefulness. Accordingly, you modified the .out file so that the expected "out" results should be equal to -1 (i.e. the black pixels to draw the lines). Probably the .java and .class files remained unchanged.
Is this correct?

All of the above is correct.  The reason for changing the 'in' to something other than -1 is to verify that load=0 is being honored; that a write does not occur.

Yes, Keyboard.java and .class didn't change.

I beg your pardon for taking advantage of your patience, but I would like to fully understand your novel test implementations, rater than simply (and blindly) applying them. In this way, I hope I will be able to realize something useful in case of future necessity.
Many thanks!

No worries.  Teaching what I know is the way I pay back all the old programmers and engineers that helped me out when I was just starting.

--Mark