We are excited to announce the release of P4Runtime v1.2.0. We have come a long way since the creation of the P4 API Working Group in 2017 and the P4Runtime 1.0.0 release in 2019! Congrats to everyone in the P4 API Working Group for making this possible and continuing to improve the quality of the P4Runtime API and specification document.
The end of 2019 was big for P4Runtime, with ONF announcing that the Stratum project was released as open-source. This makes Stratum the first open switch OS to offer P4Runtime as a northbound interface, with multiple hardware platforms already supported. If you do not have access to hardware, fret not as Stratum also supports the bmv2 simulator, which anyone can run on a Linux system.
The first half of 2020 was also very exciting, with 2 minor releases for P4Runtime, v1.1 (Feb 2020) and v1.2 (July 2020).
Notable Changes in P4Runtime v1.1 and v1.2
For the full list of changes, refer to the revision history.
Major Overhaul of Master-Arbitration (v1.1)
The P4Runtime interface allows multiple controllers to be connected to the P4Runtime server running on the device at the same time: one master controller - the only one with write access to the device - and potentially several stand-by controllers. Each controller has a numerical election ID, usually assigned through a leader election algorithm (e.g. Paxos, as implemented in Chubby), which it advertises to the server, with the master being the controller with the highest election ID. In case the master goes offline, one of the stand-by controllers is available to take its place. In P4Runtime v1.0, this hand-over was automatic and handled entirely by the server: upon detecting that the master had disconnected, the server would look for the controller with the next highest election ID, and appoint it in its place. The P4 API Working Group received feedback from Google and ONF engineers implementing SDN controllers that this could lead to issues in production deployments. In particular, in case of a network partition, it is possible for a controller in a minority partition to gain write access to the device. To address this issue in P4Runtime v1.1, we specified that the master election ID at each P4Runtime server must be strictly monotonically increasing. While this is a significant change for P4Runtime server implementations, the Working Group felt that it did not warrant increasing the major version number for the P4Runtime API, as the format of the Protobuf messages did not change.
New error Field to Report Stream Errors (v1.1)
This is another change that was requested by P4Runtime users. It enables the server to asynchronously report an error back when the controller sends an invalid message on the bi-directional stream, without shutting-down the stream altogether.
This is what the StreamMessageResponse message now looks like in P4Runtime v1.1:
message StreamMessageResponse {
oneof update {
MasterArbitrationUpdate arbitration = 1;
PacketIn packet = 2;
DigestList digest = 3;
IdleTimeoutNotification idle_timeout_notification = 4;
.google.protobuf.Any other = 5;
// Used by the server to asynchronously report errors which occur when
// processing StreamMessageRequest messages.
StreamError error = 6;
}
}
A typical use-case is for the server to use this new error field when a packet-out sent by the controller is invalid or not supported by the device. For example, if a packet-out exceeds the MTU (Maximum Transmission Unit) for the egress link, the server may generate the following StreamMessageResponse:
error {
canonical_code: 3 # INVALID_ARGUMENT
message: "Packet exceeds the MTU for port."
space: "targetX-psa-vendor1"
code: 123 # MTU_EXCEEDED
packet_out {
# we do not set the packet_out field as it does not provide any
# extra information to the client
}
}
New Capabilities RPC (v1.1)
Now that P4Runtime has minor releases, it may be necessary for a client to have a mechanism through which to detect the exact version P4Runtime version implemented by the server, in order to determine which feature set is available. This is now possible thanks to a new Capabilities RPC, inspired by the one found in gNMI:
service P4Runtime {
// ...
rpc Capabilities(CapabilitiesRequest) returns (CapabilitiesResponse) {
}
}
message CapabilitiesRequest {
}
message CapabilitiesResponse {
// The full semantic version string (e.g. "1.1.0-rc.1") corresponding to the
// version of the P4Runtime API currently implemented by the server.
string p4runtime_api_version = 1;
}
At the moment, the response only includes a semantic version string. In the future, we may consider providing additional information: for fixed-function devices, this may include a well-known set of P4 programs that are supported by the device, similarly to how the gNMI Capabilities RPC returns a list of YANG models implemented by the server.
Support for Wildcard Reads for Multicast Groups and Clone Sessions (v1.1)
One shortcoming of P4Runtime v1.0 is the inability for the controller to retrieve the entire list of multicast group entries or clone session entries programmed into the device. This makes it difficult to implement reconciliation correctly at the controller, since there is no way to read these entries without assuming that their numerical identifiers are already known. This was addressed in P4Runtime (without any change to the message format), and “wildcard” reads are now supported for multicast groups and clone sessions, just like they are for every other object (tables, counters, etc.).
The optional Match Kind (v1.2)
The optional match kind was proposed by Stefan Heule and is simply a ternary match where the mask is restricted to be all ones (exact match) or all zeros (wildcard match). So why do we need it? Based on our experience writing P4 programs and control planes for these programs, we believe that it offers the following advantages:
- It provides an additional tool to the P4 programmer who can now express value restrictions for the mask used in match entries. This specific use case is also fairly common in P4 pipelines when supporting two mutually-exclusive proptocols, such as IPv4 and IPv6. A match table’s key may select both IPv4 and IPv6 fields; using
optional, one can easily exclude IPv4 fields from the match for IPv6 packets, and vice versa. - Some P4 targets (hardware or software) can take advantage of the extra information provided by
optionalto achieve more efficient resource utilization or better performance.
While the optional match kind is not part of the core P4 language yet, it is part of the v1model architecture and P4Runtime supports programming optional matches starting with v1.2. By making optional part of P4Runtime rather than relying on existing ternary support, we ensure alignment between the P4 program and the control plane application. By constraining the syntax of the P4Runtime Protobuf messages based on the match kind, we help reduce the possibility of runtime errors.
Improvements to Annotations Support (v1.2)
Several tools have been built by P4Runtime users and implementers, which leverage P4 annotations to provide additional information to P4Runtime clients and servers. One example is p4-constraints, a project hosted on p4lang which extends the P4 language with support for constraint annotations enforced at runtime (e.g. by a P4Runtime server implementation). These tools rely on the P4Info message including a verbatim copy of the annotations (we often refer to them as “raw annotations” in the context of P4Info) present in the P4 program. Oftentimes, annotations end up having a similar structure (e.g. a list of key-value pairs), which means that these tools end up doing very similar and repetitive parsing work. For P4Runtime v1.2, we decided to help simplify writing such tools with two main changes:
- support for structured annotations
- ability to include source location information (P4 filename and line number) for all annotations
Defining structured annotations was a joint effort by the P4 Language Design Working Group and the P4 API Working Group led by Chris Sommers. Structured annotations come with their own syntax and are used to annotate objects with key-value lists or expression lists. The P4 compiler is expected to tokenize the body of structured annotations and this information will be included in the P4Info message using a Protobuf representation which can be easily consumed by other tools.
This is an example of a structured annotation (on a P4 table declaration):
@MyStructuredAnnotation[label="text", my_bool=true, int_val=2*3]
table t {
// ...
}
The P4Info message generated by the P4 compiler will include the following information for this table:
structured_annotations {
name: "MyStructuredAnnotation"
kv_pair_list {
kv_pairs {
key: "label"
value {
string_value: "text"
}
}
kv_pairs {
key: "my_bool"
value {
bool_value: true
}
}
kv_pairs {
key: "int_val"
value {
int64_value: 6
}
}
}
source_location {
file: "example.p4"
line: 145
column: 1
}
}
By also including source location information (for both regular annotations and structured ones), we enable tools which perform additional parsing on the annotation bodies, as is the case for p4-constraints, to generate more helpful error messages in case of invalid annotation usage.
Golang bindings (v1.2)
P4Runtime v1.2 is the first release for which we are publishing the Go bindings generated by the Protobuf compiler to the p4lang/p4runtime Github repo, so that they can more easily be consummed by third-party Go applications. For an example, take a look here.
Looking Forward
Looking forward to the end of 2020 and beyond, the Working Group is planning to start working on P4Runtime v2. We already have some ideas on how to address some know limitations of P4Runtime v1 and make P4Runtime even better, without being restrained by the shackles of backwards-compatibility. Nothing is set in stone yet, and we would greatly appreciate additional feedback from users and vendors. If you’d like to get involved, please consider attending our bi-weekly meetings. And don’t hesitate to open an issue on Github if you have any suggestion!
