XmlArray and Bags

In the previous post, I was talking about the aliases feature, which lets you use short commands to expand to longer ones. One useful feature here is that aliases that you enter are saved along with your preferences, so essentially they're remembered. It turns out that I probably spent about half of the time from this feature on getting loading and saving correctly. It's not that loading and saving things like aliases (basically pairs of strings with no outward dependencies) is difficult. It's more like I am vulnerable to feature creep.

The object that is directly responsible for loading the aliases is called CharacterPreferences. The ConnectionManager, which has the primary responsibility of entering and exiting the character from the world, holds the prefs object and tells it to start loading. So the preferences are stored alongside the character data in the file, but are managed separately since they're the player's settings, not the character's.

Just for example, the data in the file looks like this.

<preferences>
  <prompt>jointface&gt;</prompt>
  <aliases>
    <alias str='greet' cmd='t &1 hello sir, how are you doing?' order='0'/>
    <alias str='gre' cmd='greet' order='1'/>
  </aliases>
</preferences>

You can imagine other types of preferences being stored here eventually, such as color preferences and settings related to verbosity in messages.

The loading system on the MUD is nice at times and maybe too complicated at other times. I wanted alias loading to hook into the loading system in a fairly hands-off way, since these particular objects are pretty simple. Let me review loading briefly to frame the content. This is review from a more complete post from ages ago.

Things on the MUD can be XmlReadable and optionally also PartiallyLoadable. If something is XmlReadable, it means it implements fromXmlTree, whose contract is to take an XML element as input and produce an instance of the object as output. If the object is also PartiallyLoadable, then at this point it may not be fully loaded; certain members might be stubbed out due to unresolved dependencies.

In this partially loaded state, the loading system enumerates a set of "properties" in the member that are added in the ctor. These properties say the XPath needed to get to this property, given the element representing the parent, information about how to construct an instance of this object (in most cases, just its class name, upon which the ctor will be invoked), and instructions on what to do if the sub-element isn't found at all (usually just create a blank new instance of the object).

This partially loaded state is absolutely essential to the process of resolving circular dependencies. While it doesn't support true cycles, it will handle cycles that terminate at some depth by allowing repeated passes over all of the properties in all of the objects that aren't fully loaded.

The ideal case is to have a property on the CharacterPreferences object that points to the "aliases" element within the preferences element, and delegate the loading of multiple aliases to some other object, since there are plenty of situations where you need to load a bunch of objects with the same name and type. Additionally, the specifics of the type of the objects that should be loaded multiple times should be abstracted from the code that does the repeated loading.

Introducing XmlArray

This is where XmlArray comes in. Exactly what it sounds like, this represents an array of things in XML, knows how to load them into a real Ruby array, and doesn't know too much about the objects themselves. Using this, the preferences object can fully delegate the loading of aliases to an XmlArray by creating one that knows to enumerate all the "alias" elements and create an AliasCommand object for each one.

Need some class properties

One interesting limitation I came upon as I started to design this is that the property system is fairly heavily based on the class of the object that a property is reponsible for. For example, check out the property responsible for loading the prompt text from above.

addProp(PersistedProperty.new('prompt', ref(:promptText), String))

I declare that the name of the element needed is "prompt", that it is of type String, and in order to read or write its current value, I access the methods promptText() and promptText=(). String isn't a great example here, but it gives you a feel for the tie to the class.

Likewise, for the XmlArray, when the loading system creates it automatically in response to encountering the property, the class it creates needs to know some information. The information it needs to know is the class of the object it will create for each element and the element name in the array to look for.

The way I solve this is a little bit similar to C++ templates or C# generics. I put in functionality to make XmlArray not really a class itself, but a class generator, where the classes it generates have this extra information embedded.

    def self.newClassWithType(element_name, type)
        return newClassWithCreator(element_name) { |el|
            XmlReader.constructFromElement(el, type, true)
        }
    end # function newClassWithType

    def self.newClassWithCreator(element_name, &creator)
        tmp = Class.new(self)
        tmp.procCreator = creator
        tmp.element_name = element_name
        return tmp
    end # function newClassWithCreator

Notice that these functions aren't member functions; they're class methods (since they're declared on the class - see the self). The magic is the use of Class.new(self), which creates a new subclass of the current class, upon which I then set some members that apply for all objects created from that class. The other nice thing is that these "subclasses" are exactly that - new instances of them are still true for thing.kind_of?(XmlArray).

When I was trying to figure out how to get this class nonsense to work, I thought the way to do it was using metaclasses. I stll don't understand metaclasses well, but I get the feeling they aren't the right tool for the job.

And then here's how someone might consume the XmlArray. This creates a property that is an array that will load up all of the elements named alias under the element aliases, and each element will be loaded as type AliasCommand.

@xarrClass = XmlArray.newClassWithType('alias', AliasCommand)
addProp(PersistedProperty.new('aliases', ref(:aliasesForLoading), @xarrClass, PersistedProperty::UseNewTypeFallback))

Array properties

Not only is the XmlArray typically consumed as a property, it implements its own loading functionality using properties. It has to, in fact, in order to get automatic resolution of dependencies. It uses a cool trick to do this.

Recall from earlier in the post that the loading code first calls fromXmlTree to let the object instantiate itself from the element, then enumerates all the properties and loads them one by one. XmlArray takes advantage of this by using the fromXmlTree call to make an early pass over all the elements and add a property in place for each one. Once the placeholders are there, the loading system will automatically take care of loading them, since they're in the property list.

Put it all in a Bag

Midway through designing and coding this up, I noticed that another component of the MUD was doing almost the same thing: bags. Bags are containers represented in the MUD world that hold real MUD objects inside (like dealies, mobs, etc.). Functionally, they're basically sets. Bags have to load from XML (e.g. the nipics that spawn in a room) and save to XML (the contents of your inventory). The requirements are roughly the same, so it would be great to abstract away the loading and saving code into an XmlArray.

A lot of the work I went through was getting these to work with Bags. The idea and design is pretty straightforward, but like all things loading, there are gotchas.

@xarrClass = XmlArray.newClassWithCreator('item') { |el|
    XmlReader.constructFromElement(el, (@restrictClass || Object), false)
}

addProp(PersistedProperty.new('.', ref(:contentForLoading), @xarrClass, PersistedProperty::UseNewTypeFallback))

The somewhat clever part here is that the array loads from ".", i.e. the current node in XML, rather than some sub-element. I ended up having to fix some silly little bugs like not being able to load the XmlArray if the bag containing it isn't specified in the XML (duh! if the bag isn't there, of course any sub-elements aren't there too).

And since I didn't actually use the array as the backing store for the elements in the bag (since bags use sets as their underlying storage, not arrays), I had to make sure to sync content between the bag proper and the "loading/saving contents" at the right times. In contrast, the CharacterPreferences does use the loading array as the actual storage, so it is always up to date.

The other reason this work took so long is that I kept taking these detours to fix other loading bugs or add little enhancements here and there. For instance, I fixed the code that gives every object that is loaded one and only one call back to loadingComplete, which they can optionally implement. And it turns out to be a convenient way to clean up or finalize structures after loading is all done.

For next time I plan to talk about the testing I did to make sure things kept working.