20140603

Developing LavaMite

The other day I finally published LavaMite, which is now happily posting to Twitter. That blog post was long, like almost 3000 words. I have at least as much more I can say about various things I encountered while developing it, so that's what this post is for. This one is all about nerdy coding stuff, you have been forewarned!

Firstly, though, I should also say that I published the source code and a downloadable package, plus instructions. Woah!

Start and stop without missing a beat

Oddly enough, one feature that took me a while code up was the ability to quit the program at any time and restart it from the point it left off. This was useful for restarting the program with any bug fixes or tuning updates. It required persisting the start time and other parameters of the last-executing round and reloading it on program start.

//
// if we're missing any of the information to calculate the correct
// offset into the last round, we put ourselves after the end of the last
// round
//
if (!haveAllLastActionTimeInfo)
{
    lastActionTime = rs.wholeRoundInterval.end + dur!"seconds"(1);
}

//
// this is really important. shift the round that we loaded from file
// to be offset from the current time. We calculate how far we made it
// into the last round, then adjust the startTime of the round to
// position the current time at that point.
//
r = new Round(
        rs.number,
        g_clock.currTime - (lastActionTime - rs.startTime),
        rs.startTime,
        rs.priorCooldownTime,
        rs.warmUpTime,
        rs.timeToStabilization,
        rs.cooldownTime);

setCurrentRound(r);

The program then has to skip whatever phases of the round (warmup, stabilization, or cooldown) that were already passed before the quit and pick up in the right phase.

//
// wait for a while to warm up the lamp partway, but not so much
// that it goes into "steady state" (with bubbles continually
// rising and falling). the whole point of this program is to
// capture the formations before it's stabilized
//
if (g_currentRound.warmUpInterval.contains(g_clock.currTime))
{
    //
    // when continuing a round, determine how far into the round
    // it is now. calculate the time remaining in this phase of
    // the round and only wait for that long.
    //
    // for new rounds, this will be the whole time for the phase,
    // since the time is currently just after the start of the
    // phase.
    //
    // add 5 sec to the time we wait, which will comfortably put
    // us into the next phase, once this phase is completed
    //
    Duration remainingWarmUpTime = g_currentRound.warmUpInterval.end - g_clock.currTime + dur!"seconds"(5);
    log(format("Leaving on for %s", stripFracSeconds(remainingWarmUpTime)));
    if (sleepWithExitCheck(remainingWarmUpTime))
    {
        break;
    }

    //
    // with the lamp warmed up as much as we want, take the money
    // shot and post it for the world to see
    //
    takeAndPostPhoto(camera);
}

By itself, it's not really that big a deal, except doing time math makes my head hurt. D has really nice classes like core.time.Duration, std.datetime.Interval, and std.datetime.SysTime that make this math readable and simple to do.

So why did it take me so long to implement? Making it testable took a lot more effort.

Becoming a Time Lord

Say I want to test that the program picks up properly where it left off. There are three phases of a round (warmup, stabilization, and cooldown), so at a minimum I need three test cases, each one quitting during one of the phases. I expect the program to resume and continue from the correct point in that phase.

Now consider that each phase might last over an hour. I can't wait 3 hours to get into cooldown phase to quit. As a programmer and human being, I definitely don't have the patience for that. I need a way to skip over all that dead time and get to the interesting stuff.

I went through at least three implementations of faking out the time until arriving at the version that worked. I can't even describe those older versions very well, because they were kind of convoluted and confusing. I don't think it's worth it. Let me say the salient features of the final version, which I think is not bad.

A clockmaker. No, a time-maker

The most important feature of the final, testable version of the dry run in the program involves a synthetic clock. For most things normally, I would Clock.currTime, but with the synthetic clock, I don't always want to fetch the real current time. I might want to fetch the fake time that the program thinks it is.

What sort of things would cause the real time and the fake time to differ? In a word, sleeping. Whenever the program sleeps, especially if it's a long sleep, I want very little real time to advance but the whole time requested by the program to pass.

Let's see what that would look like.

bool sleepWithExitCheck(Duration d)
{
    assert(!d.isNegative());

    Duration fakeSleepDuration = d;

    if (!g_forReals)
    {
        d = dur!"seconds"(5);
    }

    bool didReceive = receiveTimeout(d, (bool dummy) { });

    if (didReceive)
    {
        fakeSleepDuration /= 2;
    }

    g_clock.advance(fakeSleepDuration);

    return didReceive;
}

All sleeps, be they long or short, are truncated to 5 seconds, but the fake clock, g_clock is advanced by the original amount requested. g_clock is set to the true time at program startup, and calls to g_clock.currTime return the synthetic time in dry mode.

But these are not mere sleeps, rather waiting for the user-has-typed-Quit thread to tell us if the user wants to exit the program. The assumption here is that if the user did type quit, they most likely did it sometime during the requested interval, so we just guess that they asked halfway through. This fulfills an important requirement that the sleep duration was exited sometime before its end.

It's not great to have everything sleeping for 5 seconds, but it seems to work well enough. Ideally, short sleeps would sleep for less than long sleeps, but long sleeps would still not sleep for a boring amount.

Need to COMmunicate in serial

The PC part of the program talks over the COM port to the microcontroller and tells it when to turn on and off. I already had an idea off the top of my head of how to talk to something that's connected to a COM port in Windows. Whatever its failings, CreateFile does let you simply open devices like COM ports. Then you can use ReadFile and WriteFile to communicate over them.

Time to dip into the wonderful world of D interoperability with C. Actually, this is a core design goal of D, so it was really easy.

First, however, I needed to find wrappers for basic Windows types. These exist in the core.sys.windows.windows module, but I don't know where anything tells you that. I think I kind of stumbled upon it by accident. Once I imported this, I was free to define my own wrappers with impunity.

Though the module I just mentioned has a lot of things from kernel32.dll, it doesn't have everything, namely the COM port functions. I knew that I'd need to set the baud rate, parity bits, and so on. The Windows API to do that is SetCommState, with a few others. Here is an example wrapper.

struct DCB {
  DWORD DCBlength;
  DWORD BaudRate;
  /*
  DWORD fBinary  :1;
  DWORD fParity  :1;
  DWORD fOutxCtsFlow  :1;
  DWORD fOutxDsrFlow  :1;
  DWORD fDtrControl  :2;
  DWORD fDsrSensitivity  :1;
  DWORD fTXContinueOnXoff  :1;
  DWORD fOutX  :1;
  DWORD fInX  :1;
  DWORD fErrorChar  :1;
  DWORD fNull  :1;
  DWORD fRtsControl  :2;
  DWORD fAbortOnError  :1;
  DWORD fDummy2  :17;
  */
  DWORD fStuff;
  WORD  wReserved;
  WORD  XonLim;
  WORD  XoffLim;
  BYTE  ByteSize;
  BYTE  Parity;
  BYTE  StopBits;
  char  XonChar;
  char  XoffChar;
  char  ErrorChar;
  char  EofChar;
  char  EvtChar;
  WORD  wReserved1;
};

extern(C)
BOOL SetCommState(
  HANDLE hFile,
  DCB* lpDCB
);

Notice all the bit-field members of DCB that are commented out and replaced by a single DWORD, because D doesn't support bit fields in C structs. Anyone who wants to touch those bits will have to access them with some bitwise arithmetic--or I could have written member function wrappers for them. I didn't need to, myself, so I didn't bother. Tee hee.

In the context of Arduino I was able to find some sample code to make sure I was calling these functions right, too, which was handy.

private void setCOMPortParameters()
{
    DCB dcb;
    if (!GetCommState(m_hCOMPort, &dcb))
    {
        throw new Exception(format("failed GetCommState: %d", GetLastError()));
    }

    //log(format("Previous baud: %d, byteSize: %d, parity: %d, stopbits: %d", dcb.m_BaudRate, dcb.ByteSize, dcb.Parity, dcb.StopBits));

    dcb.BaudRate = m_baudRate;    // set the baud rate
    dcb.ByteSize = 8;             // data size, xmit, and rcv
    dcb.Parity = NOPARITY;        // no parity bit
    dcb.StopBits = ONESTOPBIT;    // one stop bit

    if (!SetCommState(m_hCOMPort, &dcb))
    {
        throw new Exception(format("failed SetCommState: %d", GetLastError()));
    }
}

Click click click click

Due to the way the relay is set up, it is "live" by default. The microcontroller's first job is to turn it off at startup. This produces an audible click, because the relay is mechanical inside.

So when does the microcontroller turn on? It turns out, a lot. When the USB cable is plugged in, it of course turns on, because it just received power. It also resets, apparently by design, whenever serial communication is initiated. You can work around it by soldering a capacitor, but I didn't bother reading any further because they said capacitor.

Actually, every time you open the serial port for communications, it resets several times, so when my program starts, you hear several weird clicks. It's unnerving but otherwise harmless, I think.

To compound this problem, my initial protocol for talking to the microcontroller used the strings "0" and "1" instead of the nice strings "switch_1" and "switch_0". I am almost certain that during the initial COM handshake or whatever the system does, it throws a 1 or 0 out there (the character value of '0', I mean). Once I changed it to the longer string it wasn't as bad.

My program only opens the COM port once (i.e. calls CreateFile on it) at startup, so the multiple resets aren't a major problem during the main loop of the program.

Don't talk so fast

I did my initial testing on my old laptop (which I'm using to write this blog right now), even portions on the bus. It cannot be understated how unwieldy it is to have a microcontroller and cables all over the place on the bus. If the other riders didn't think I was a weirdo before...

Anyway, it all tested nicely on here, but when I plugged it into my PC I had all kinds of trouble reliably controlling the power. Sometimes it wouldn't turn on. Other times it wouldn't turn off. It seemed like it just wasn't hearing the commands I was sending, even though the Arduino serial monitor tool worked fine, and I checked my COM port configuration settings thoroughly and reinstalled the FTDI drivers, to no avail.

I don't remember how I arrived at it, but I fixed this problem by putting a delay between opening the COM port for writing and actually sending the first command. I think my PC, which is beastly fast, was trying to talk to the Arduino well before it was actually ready, but my slow laptop just took longer to get to that point.

Looking ahead

I have two ideas left for this program, though only one I will probably get around to. I'd like to look at more interesting patterns for warmup. Right now it is a fixed time that the lamp just stays on. What if it pulsed on and off with a 1 min / 1 min duty cycle or something? I have so much of the infrastructure ready to go that it'd be a shame not to explore this a bit more.

My thinking is that I will determine the total warm-up time at the start of the round like I do today, and then turn the lamp on and off a random number of times with random durations of "on", to total that amount of time spent powered.

The only hiccup is my feature of resuming in the middle of the round, but I think I can solve this by persisting the number of active periods remaining in the round, and the current state of the lamp.

My other idea is to use some kind of image processing to choose the picture to tweet, instead of a random time like I do today. I already have the camera taking pictures every 1 minute. When it gets to the cooldown phase, it has a whole set of pictures from the round, and some image processing could look at them all and pick a "good one" based on some heuristics. This could get arbitrarily complex, and it's in a whole new field of computer science, so I'm going to put it off for now.

Posted by knutaf at 11:51 PM 1 comments
Labels: ,
Subscribe to: Posts (Atom)