Graph

Apstra uses the Graph model to represent a single source of truth regarding infrastructure, policies, constraints etc. This Graph model is subject to constant change and we can query it for various reasons. It is represented as a graph. All information about the network is modeled as nodes and relationships between them.

Every object in a graph has a unique ID. Nodes have a type (a string) and a set of additional properties based on a particular type. For example, all switches in our system are represented by nodes of type system and can have a property role which determines which role in the network it is assigned (spine/leaf/server). Physical and logical switch ports are represented by an interface node, which also has a property called if_type.

Relationships between different nodes are represented as graph edges which we call relationships. Relationships are directed, meaning each relationship has a source node and a target node. Relationships also have a type which determines which additional properties particular relationship can have. E.g. system nodes have relationships of type hosted_interfaces towards interface nodes.

A set of possible node and relationship types is determined by a graph schema. The schema defines which properties nodes and relationships of particular type can have along with types of those properties (string/integer/boolean/etc) and constraints. We use and maintain an open source schema library, Lollipop, that allows flexible customization of value types.

Going back to the graph representing a single source of truth, one of the most challenging aspects was how to reason about it in the presence of change, coming from both the operator and the managed system. In order to support this we developed what we call Live Query mechanism which has three essential components:

• Query Specification
• Change Notification
• Notification Processing

Having modeled our domain model as a graph, you can run searches on the graph specified by graph queries to find particular patterns (subgraphs) in a graph. The language to express the query is conceptually based on Gremlin, an open source graph traversal language. We also have parsers for queries expressed in another language - Cypher, which is a query language used by popular graph database neo4j.

You start with a node() and then keep chaining method calls, alternating between matching relationships and nodes:

node('system', name='system').out().node('interface', name='interface').out().node('link', name='link')

The query above translated in english reads something like: starting from a node of type system, traverse any outgoing relationship that reaches node of type interface, and from that node traverse all outgoing relationship that lead to node of type `link.

At any point you can add extra constraints:

node('system', role='spine', name='system').out().node('interface', if_type='ip', name='interface')

Notice role=`spine` argument, it will select only system nodes that have role property set to spine.

Same with if_type property for interface nodes.

node('system', role=is_in(['spine', 'leaf']), name='system')
.out()
.node('interface', if_type=ne('ip'), name='interface')

That query will select all system nodes that have role either spine or leaf and interface nodes that have if_type anything but ip (ne means not equal).

You can also add cross-object conditions which can be arbitrary Python functions:

node('system', name='system')
.out().node('interface', name='if1')
.out().node('link')
.in_().node('interface', name='if2')
.in_().node('system', name='remote_system')
.where(lambda if1, if2: if1.if_type != if2.if_type)

Name objects to refer to them and use those names as argument names for your constraint function (of course you can override that but it makes a convenient default behavior). So, in example above it will take two interface nodes named if1 and if2, pass them into given where function and filter out those paths, for which function returns False. Don't worry about where you place your constraint: it will be applied during search as soon as all objects referenced by constraint are available.

Now, you have a single path, you can use it to do searches. However, sometimes you might want to have a query more complex than a single path. To support that, query DSL allows you to define multiple paths in the same query, separated by comma(s):

match(
    node('a').out().node('b', name='b').out().node('c'),
    node(name='b').out().node('d'),
)

This match() function creates a grouping of paths. All objects that share the same name in different paths will actually be referring to the same object. Also, match() allows adding more constraints on objects with where(). You can do a distinct search on particular objects and it will ensure that each combination of values is seen only once in results:

match(
    node('a', name='a').out().node('b').out().node('c', name='c')
).distinct(['a', 'c'])

This matches a chain of a -> b -> c nodes. If two nodes a and c are connected through more than one node of type b, the result will still contain only one (a, c) pair.

There is another convenient pattern to use when writing queries: you separate your structure from your criteria:

match(
    node('a', name='a').out().node('b').out().node('c', name='c'),
    node('a', foo='bar'),
    node('c', bar=123),
)

Query engine will optimize that query into:

match(
    node('a', name='a', foo='bar')
    .out().node('b')
    .out().node('c', name='c', bar=123)
)

No cartesian product, no unnecessary steps.

Ok, now you have a graph query defined. What does a notification result look like? Each result will be a dictionary mapping a name that you have defined for a query object to object found. E.g. for following query

node('a', name='a').out().node('b').out().node('c', name='c')

results will look like {'a': <node type='a'>, 'c': <node type='c'>}. Notice, only named objects are present (there is no <node type='b'> in results, although that node is present in query because it does not have a name).

You register a query to be monitored and a callback to execute if something will change. Later, if someone will modify the graph being monitored, it will detect that new graph updates caused new query results to appear, or old results to disappear or update. The response executes the callback that is associated with the query. The callback receives the whole path from the query as a response, and a specific action (added/updated/removed) to execute.

When the result is passed to the processing (callback) function, from there you can specify reasoning logic. This could really be anything, from generating logs, errors, to rendering configurations, or running semantic validations. You could also modify the graph itself, using graph APIs and some other piece of logic may react to changes you made. This way, you can enforce the graph as a single source of truth while it also serves as a logical communication channel between pieces of your application logic. The Graph API consists of three parts:

Graph management - methods to add/update/remove stuff in a graph. add_node(), set_node(), del_node(), get_node()add_relationship(), set_relationship(), del_relationship(), get_relationship(), commit() Query get_nodes()get_relationships() Observable interface add_observer(),remove_observer()

Graph management APIs are self-explanatory. add_node() creates new node, set_node() updates properties of existing node, and del_node() deletes a node.

commit() is used to signal that all updates to the graph are complete and they can be propagated to all listeners.

Relationships have similar API.

The observable interface allows you to add/remove observers - objects that implement notification a callback interface. Notification callback consists of three methods:

• on_node() - called when any node/relationship is added, removed or updated
• on_relationship() - called when any node/relationship is added, removed or updated
• on_graph() - called when the graph is committed

The Query API is the heart of our graph API and is what powers all searching. Both get_nodes() and get_relationships() allow you to search for corresponding objects in a graph. Arguments to those functions are constraints on searched objects.

E.g. get_nodes() returns you all nodes in a graph, get_nodes(type='system') returns you all system nodes, get_nodes(type='system', role='spine') allows you to constrain returned nodes to those having particular property values. Values for each argument could be either a plain value or a special property matcher object. If the value is a plain value, the corresponding result object should have its property equal to the given plain value. Property matchers allow you to express a more complex criterias, e.g. not equal, less than, one of given values and so on:

graph.get_nodes(
    type='system',
    role=is_in(['spine', 'leaf']),
     system_id=not_none(),
)

In your graph schema you can define custom indexes for particular node/relationship types and the methods get_nodes() and get_relationships() pick the best index for each particular combination of constraints passed to minimize search time.

Results of get_nodes()/get_relationships() are special iterator objects. You can iterate over them and they will yield all found graph objects. You can also use APIs that those iterators provide to navigate those result sets. E.g. get_nodes() returns you a NodeIterator object which has methods out() and in_(). You can use those to get an iterator over all outgoing or incoming relationship from each node in the original result set. Then, you can use those to get nodes on the other end of those relationships and continue from them. You can also pass property constraints to those methods the same way you can do for get_nodes() and get_relationships().

graph.get_nodes('system', role='spine') \
    .out('interface').node('interface', if_type='loopback')

The code in the example above finds all nodes with type system and role spine and then finds all their loopback interfaces.

Thequery below is an example of an internal rule that Apstra can use to derive telemetry expectations -- for example, link and interface status. The @rule will insert a callback to process_spine_leaf_link, in which case we write to telemetry expectations.

@rule(match(
    node('system', name='spine_device', role='spine')
    .out('hosted_interfaces')
    .node('interface', name='spine_if')
    .out('link')
    .node('link', name='link')
    .in_('link')
    .node('interface', name='leaf_if')
    .in_('hosted_interfaces')
    .node('system', name='leaf_device', role='leaf')
))
def process_spine_leaf_link(self, path, action):
    """
    Process link between spine and leaf

    """
    spine = path['spine_device']
    leaf = path['leaf_device']
    if action in ['added', 'updated']:
        # do something with added/updated link
        pass
    else:
        # do something about removed link
        pass

To avoid creating complex where() clauses when building a graph query, use convenience functions, available from the Apstra GUI.

1. From the blueprint navigate to the Staged view or Active view, then click the GraphQL API Explorer button (top-right >_). The graph explorer opens in a new tab.

2. Type a graph query on the left. See function descriptions below.

3. From the Action drop-down list, select qe.

4. Click the Execute Query button (looks like a play button) to see results.

• Functions
• PathQueryBuilder Nodes
• Property matchers

match(
  node('system', name='leaf', role='leaf'),
  node('system', name='spine', role='spine'),
)

def find_router_facing_systems_and_intfs(graph):
    return q.iterate(graph, q.match(
        q.node('link', role='to_external_router')
        .in_('link')
        .node('interface', name='interface')
        .in_('hosted_interfaces')
        .node('system', name='system')
    ))

node('system', name='device', role='leaf')

node('system', name='system') \
 .out('hosted_interfaces')

node('interface', name='interface') \
 .in_('hosted_interfaces')

node('system', name='system') \
 .where(lambda system: system.role in ('leaf', 'spine'))

match(node('system', name='system', role='leaf') \
  .out('hosted_interfaces') \
  .node('interface', name='interface', ipv4_addr=not_none()) \
  .out('link') \
  .node('link', name='link') \
  .in_('link') \
  .node('interface', name='remote_interface', ipv4_addr=not_none())) \
  .ensure_different('interface', 'remote_interface')

match(node('system', name='system', role='leaf') \
  .out('hosted_interfaces') \
  .node('interface', name='interface', ipv4_addr=not_none()) \
  .out('link') \
  .node('link', name='link') \
  .in_('link') \
  .node('interface', name='remote_interface', ipv4_addr=not_none())) \
  .where(lambda interface, remote_interface: interface != remote_interface)

node('system', name='system', role=eq('leaf'))

node('system', name='system', role='leaf')

node('system', name='system').where(lambda system: system.role == 'leaf')

{
  "count": 4,
  "items": [
    {
      "system": {
        "tags": null,
        "hostname": "l2-virtual-mlag-2-leaf1",
        "label": "l2_virtual_mlag_2_leaf1",
        "system_id": "000C29EE8EBE",
        "system_type": "switch",
        "deploy_mode": "deploy",
        "position": null,
        "role": "leaf",
        "type": "system",
        "id": "391598de-c2c7-4cd7-acdd-7611cb097b5e"
      }
    },
    {
      "system": {
        "tags": null,
        "hostname": "l2-virtual-mlag-2-leaf2",
        "label": "l2_virtual_mlag_2_leaf2",
        "system_id": "000C29D62A69",
        "system_type": "switch",
        "deploy_mode": "deploy",
        "position": null,
        "role": "leaf",
        "type": "system",
        "id": "7f286634-fbd1-43b3-9aed-159f1e0e6abb"
      }
    },
    {
      "system": {
        "tags": null,
        "hostname": "l2-virtual-mlag-1-leaf2",
        "label": "l2_virtual_mlag_1_leaf2",
        "system_id": "000C29CFDEAF",
        "system_type": "switch",
        "deploy_mode": "deploy",
        "position": null,
        "role": "leaf",
        "type": "system",
        "id": "b9ad6921-6ce3-4d05-a5c7-c31d96785045"
      }
    },
    {
      "system": {
        "tags": null,
        "hostname": "l2-virtual-mlag-1-leaf1",
        "label": "l2_virtual_mlag_1_leaf1",
        "system_id": "000C297823FD",
        "system_type": "switch",
        "deploy_mode": "deploy",
        "position": null,
        "role": "leaf",
        "type": "system",
        "id": "71bbd11c-ed0f-4a38-842f-341781c01c24"
      }
    }
  ]
}

node('system', name='system', role=ne('spine'))

node('system', name='system').where(lambda system: system != 'spine')

node('vn_instance', name='vlan', vlan_id=gt(200))

node('vn_instance', name='vlan', vlan_id=ge(200))

node('vn_instance', name='vlan', vlan_id=lt(200))

node('vn_instance', name='vlan').where(lambda vlan: vlan.vlan_id <= 200)

node('vn_instance', name='vlan', vlan_id=le(200))

node('vn_instance', name='vlan').where(lambda vlan: vlan.vlan_id < 200)

node('system', name='system', role=is_in(['leaf', 'spine']))

node('system', name='system').where(lambda system: system.role in ['leaf', 'spine'])

node('system', name='system', role=not_in(['leaf', 'spine']))

node('system', name='system').where(lambda system: system.role not in ['leaf', 'spine'])

node('interface', name='interface', ipv4_addr=is_none()

node('interface', name='interface').where(lambda interface: interface.ipv4_addr is None)

node('interface', name='interface', ipv4_addr=not_none()

node('interface', name='interface').where(lambda interface: interface.ipv4_addr is not None)

The Apstra graph datastore is an in-memory graph database. The log file size is checked periodically, and when a blueprint change is committed. If the graph datastore reaches 100MB or more, a new graph datastore checkpoint file is generated. The database itself does not remove any graph datastore persistence logs or checkpoint files. Apstra provides clean-up tools for the main graph datastore.

Valid graph datastore persistence file groups contain four files: log, log-valid, checkpoint, and checkpoint-valid. Valid files are the effective indicators for log and checkpoint files. The name of each persistence file has three parts: basename, id, and extension.

# regex for sysdb persistence files.
# e.g.
#      _Main-0000000059ba612e-00017938-checkpoint-valid
#       \--/ \-----------------------/ \--------------/
#     basename           id                extension

• basename - derived from the main graph datastore partition name.
• id - a unix timestamp obtained from gettimeofday. Seconds and microseconds in the timestamp are separated by a "-". A persistence file group can be identified by id. The timestamp can also help to determine the generated time sequence of persistence file groups.
• extension - log, log-valid, checkpoint, or checkpoint-valid.