Chapter 2 structure.py — the Universal Serialization Base Class

The root-level structure.py defines the Structure base class — every class in keytab.py, ccache.py, smb3structs.py and friends inherits from it, and it underpins all protocol packet structures used throughout this manual. The long comment at the top of the file describes a data-format description language: think of it as an extension of the standard struct format, whose type notations look a bit like regular expressions; the pack / unpack methods then serialize and deserialize variables according to that format (corrections welcome if I misread anything). It is used to describe the request packet structures of SMB, RPC, Kerberos and other intra-domain protocols.

    """ sublcasses can define commonHdr and/or structure.
        each of them is an tuple of either two: (fieldName, format) or three: (fieldName, ':', class) fields.
        [it can't be a dictionary, because order is important]

        where format specifies how the data in the field will be converted to/from bytes (string)
        class is the class to use when unpacking ':' fields.

        format specifiers:  # identical to the struct module (x c b B h H l L i I q Q s p f d = @ ! < >)
          some additional format specifiers:
            :       just copy the bytes from the field into the output string (input may be string, other structure, or anything responding to __str__()) (for unpacking, all what's left is returned)
            z       same as :, but adds a NUL byte at the end (asciiz)  [asciiz string]
            u       same as z, but adds two NUL bytes at the end  [unicode string]
            w       DCE-RPC/NDR string (it's a macro for [  '<L=(len(field)+1)/2','"\\x00\\x00\\x00\\x00','<L=(len(field)+1)/2',':' ]
            ?-field length of field named 'field', formatted as specified with ?
            ?1*?2   array of elements. Each formatted as '?2', the number of elements is stored as specified by '?1'
            'xxxx / "xxxx   literal xxxx (field's value doesn't change the output)
            # printf-style (%08x, %s, ...) and the rarer _ / ?=packcode / ?&fieldname
            # specifiers are omitted here — see the structure.py source for the full description
    """

Take SMB as an example:

# eg./impacket/smb3structs.py

class SMB2Negotiate(Structure):
    structure = (
        ('StructureSize','<H=36'),
        ('DialectCount','<H=0'),
        ('SecurityMode','<H=0'),
        ('Reserved','<H=0'),
        ('Capabilities','<L=0'),
        ('ClientGuid','16s=""'),
        ('ClientStartTime','8s=""'),  # or (NegotiateContextOffset/NegotiateContextCount/Reserved2) in SMB 3.1.1
        ('Dialects','*<H'),
        # SMB 3.1.1
        ('Padding',':=""'),
        ('NegotiateContextList',':=""'),
    )

The same pattern appears in ccache.py — it parses the binary credential cache file used by Kerberos:

class Header(Structure):
    structure = (
        ('tag','!H=0'),
        ('taglen','!H=0'),
        ('_tagdata','_-tagdata','self["taglen"]'),
        ('tagdata',':'),
    )