What's in a Datastore?

This memo looks at the traditional definition of a datastore based on published RFC of YANG and NETCONF and interpolates some of the gaps.

NETCONF is a protocol for NETwork CONFiguration, developed by the IETF in response to the IAB Network Management Workshop , where operators requested a character-based (not binary) protocol that made a clear distinction between configuration data, data that describes operational state and statistics. Configuration data is defined by NETCONF as the data that is provided by an user to take a box from its initial, straight from the factory, state to a functioning one. It excludes data that the box will learn of itself, such as by DHCP or routing protocols, or from hot-plugging hardware. Thus NETCONF could support a scenario where the configuration data is read from a device, the hardware of the device is replaced, the configuration data is written to the replacement hardware which then resumes operation without the need for further intervention. Configuration data is data that may need to be written. Other data is read-only. NETCONF does not draw a distinction between statistics and operational state - that is all read-only data. The consequence of this is that what may seem to a user as a single data structure, a routing table or an interface table, is split by netconf into those parts that may need to be written - configuration data - and the rest - state data. While the IAB workshop formulated a distinction between configuration data, operational state data and statistical data, this three-fold split has not been maintained in the design of NETCONF. Netconf operations take place on a datastore, either the entire datastore (copy-config, delete-config, commit) or on part of a datastore (edit-config, get-config). So while netconf defines a datastore as

o datastore: A conceptual place to store and access information. A datastore might be implemented, for example, using files, a database, flash memory locations, or combinations thereof. and more strictly a configuration datastore as

o configuration datastore: The datastore holding the complete set of configuration data that is required to get a device from its initial default state into a desired operational state. a datastore is more usefully seen as the target or source of a NETCONF operation. Thus this concept of a datastore is fundamental to NETCONF. NETCONF does not specify how the datastore is organised. Implicit is the assumption that there will be a DDL but that DDL is not specified. Indeed, the specification of NETCONF was well advanced before consideration was given to the selection of a DDL. There are however implicit assumptions in NETCONF about the nature of the DDL; it is assumed to define a tree structure with one or more roots with data at the leaves and at the nodes (this is implied by the ability of NETCONF to filter data). Datastores are named; they have to be in order for NETCONF to be able to target them. NETCONF defines three, running, startup and candidate. Users can define any number of additional datastores. The focus of NETCONF is on configuration but it provides facilities to read state data. NETCONF does not define how the state data is organised. It must be in a datastore else NETCONF could not access it but such a datastore is not given a name. The implicit assumption is that it is in some sense part of the running datastore, at least as far as operations are concerned. NETCONF get is the only operation that defined to act on state data.

The DDL chosen to specify a NETCONF datastore is YANG. The initial specification thereof does not define the term datastore nor does it import the definition from NETCONF but it does use it in several places, sometimes as 'datastore' at other times as 'NETCONF datastore' or 'configuration datastore'. YANG has several facilities for constraint checking (e.g. the 'must', 'mandatory' or 'when' statements) which may require data nodes to be present or absent or to take a given value. These constraint checks are defined to take place during validation within a datastore or part thereof. Where constraints involve an XPath expression, YANG specifies the accessible tree for this operation and the context node. YANG differentiates between configuration and state data with the 'config' substatement and when a constraint in enforced involving the state data, YANG refers to

'all state data on the device, and the <running/> datastore' . i.e. YANG implies that state data is not in a datastore, or at least, not in the running datastore. YANG does not use the term 'operational state'. So state data exists, can be read (but not written) and can be referenced via the YANG schema but little or nothing more is said about it.

NETCONF requires the existance of the running configuration datastore on a device but no others; this datastore must appear and must appear only once. This datastore has an optional feature :writable-running, or at least, the NETCONF protocol definition has such an optional capability. The specification of NETCONF is incomplete when it comes to the life cycle of data in datastores. If running is the only datastore and :writable-running is not present, then presumably configuration data can be read but not written. If running is the only datastore and :writable-running is present, then presumably any changes made are written to non-volatile storage and persist across a restart of the device. Posts to the IETF NETCONF mailing list report that this is how this combination has been implemented. NETCONF defines a candidate datastore as a 'full configuration data set that serves as a work place for creating and manipulating configuration data'. NETCONF provides a specific operation, commit, that copies the candidate datastore to running. It also provides an option to discard changes that have not been committed. YANG recognises the 'work in progress' nature of the candidate datastore by not requiring the datastore to conform to validation constraints after an edit. Rather it delays such constraint checking until the datastore is committed to running. Alternatively, NETCONF provides a validate operation which performs such constraint checking on demand. NETCONF defines a startup configuration datastore as

The configuration datastore holding the configuration loaded by the device when it boots. Like candidate, this is an optional feature. When present, NETCONF now specifies that a

<copy-config> from running to startup is also necessary to save the changes to startup ie changes made to running will not persist unless the copy-config is performed and will otherwise be discarded on the next boot. startup can be used as the target of a NETCONF copy, get-config or validate. (The last seems slightly perverse since YANG specifies that an attempted edit of startup should be followed by an implicit validate). startup can be deleted by NETCONF but this does not delete the configuration - rather it returns startup to its factory defaults. NETCONF also allows that

'Even if it advertises the :writable-running capability, a device MAY choose not to support the <running/> configuration datastore as the <target> parameter of a <copy-config> operation.' The only way to estatblish whether or not this occurs would appear to be trial and error. User defined datastores are an optional capability. The datastores are specified by a URL and this can appear as the target of a NETCONF copy-config, delete or validate, or the source of a copy-config.

This informational memo introduces no security considerations.