FIDL ABI and Source Compatibility Guide

Date: 2019-03

Author: thatguy@google.com

Contributors: pascallouis@google.com, yifeit@google.com, cramertj@google.com

Intended Audience

This doc is written for engineers who want to evolve FIDL APIs. It describes what can be done safely without disrupting fellow teammates or downstream clients.

What is ABI compatibility?

ABI (application binary interface) compatibility is concerned with the encoding and decoding of data over binary interfaces. FIDL messages (method calls on FIDL protocols) end up serialized as bytes over a zircon channel. Both channel endpoints (the client and server) must agree on the size, ordering and meaning of the bytes. A mismatch in expectations leads to binary incompatibility.

Considerations when changing FIDL source

A change to a FIDL type that is source compatible (also known as API compatible) means it is possible for someone to write code using the generated code for a type that compiles both before and after the change is made. Making a source-incompatible change requires changing all client source code at the same time (difficult if clients exist outside the repository) to avoid breaking builds.

Disclaimer: The FIDL compatibility story today contains a number of edge cases. Language bindings may expose interfaces whose usage may or may not be resilient to changes in the underlying FIDL protocol. There are ongoing efforts to standardize these interfaces, but in the meantime, this document exists as a best-effort guide towards what types of code may be broken by what sorts of FIDL changes. If you discover an omission or mistake in this document, please suggest the appropriate change, and refrain from enacting retribution on this document's authors.

The Guide

structs

General guidance: once a struct is defined and in use, it cannot be changed.

Renaming the struct

struct A {         struct A_new {
  int32 a;           int32 a;
  string b;          string b;
};                 };

green checkmark ABI Compatibility: YES

Transition Considerations:

  1. Introduce an alias
  2. Make a copy of the struct with the new name
  3. Migrate all clients
  4. Remove alias
  5. Delete the old struct

Reordering members

struct A {         struct A {
  int32 a;           string b;
  string b;          int32 a;
};                 };

red x ABI Compatibility: NO

Transition Considerations:

  • Positional initializers will break e.g.,:
    • C++:
      • auto a = A{10, "foo"};
    • Go:
      • A {10, "foo"};
  • Prefer:
    • C++:
      • auto a = A{
          .a = 10,
          .b = "foo"
        };
    • Go:
      • a := A{
          A:10.
          B: "foo",
        }

Renaming members

struct A {         struct A {
  int32 a;           int32 a_new;
  string b;          string b;
};                 };

green checkmark ABI Compatibility: YES

red x Transition Considerations: NO

  • Named reference / initialization will break:
    • C++: f(a.a_new) and auto a = A{.a = 10};

Adding members

struct A {         struct A {
  int32 a;           int32 a;
  string b;          string b;
  int32 c;
};                 };

red x ABI Compatibility: NO

Transition Considerations: * Depends on the language bindings. * Go positional initializers & Rust and Dart struct literals will break.

Removing members

struct A {         struct A {
  int32 a;           int32 a;
  string b;
};                 };

red x ABI Compatibility: NO

green checkmark Transition Consideration: * So long as b is not referenced any more (including in positional initializers).

tables

Renaming the table

table T {          table T_new {
  1: int32 a;        1: int32 a;
  2: string b;       2: string b;
};                 };

green checkmark ABI Compatibility: YES

Reordering members

table T {          table T {
  1: int32 a;        2: string b;
  2: string b;       1: int32 a;
};                 };

green checkmark ABI Compatibility: YES * Just don't change the ordinal values.

green checkmark Transition Considerations: YES

Renaming members

table T {          table T {
  1: int32 a;        1: int32 a_new;
  2: string b;       2: string b;
};                 };

green checkmark ABI Compatibility: YES

red x Transition Considerations: NO

Adding members

table T {          table T {
  1: int32 a;        1: int32 a;
  2: string b;       2: string b;
                     3: int32 c;
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES

Removing members

table T {          table T {
  1: int32 a;        1: int32 a;
  2: string b;
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES * So long as b is not referenced any more.

Adding [NoHandles]

                   [NoHandles]
table T {          table T {
  1: int32 a;        1: int32 a;
  2: string b;       2: string b;
};                 };

ABI Compatibility: TODO

Transition Considerations: TODO

unions

xunions

Reordering members

xunion A {         xunion A {
  int32 a;           string b;
  string b;          int32 a;
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES

Renaming members

xunion A {         xunion A {
  int32 a;           int32 a_new;
  string b;          string b;
};                 };

green checkmark ABI Compatibility: YES * Use the [Selector] to retain compatibility: * xunion A {
  [Selector = "a"]
  int32 a_new;
  string b;
};

red x Transition Considerations: NO

Adding members

xunion A {         xunion A {
  int32 a;           int32 a;
  string b;          string b;
                     int32 c;
};                 };

green checkmark ABI Compatibility: YES

Transition Considerations: * Depends on language bindings. * Exhaustive matching (i.e., C++ switch{} on union tag) will break.

Removing members

xunion A {         xunion A {
  int32 a;           int32 a;
  string b;
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES * So long as b is not referenced any more.

vectors

Changing the size

vector<T>:N        vector<T>:M

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations:

  • If the maximum size of the vector is growing (i.e. M > N) then all consumers MUST be updated first.
  • If the maximum size of the vector is shrinking (i.e. M < N) then all producers MUST be updated first.

Changing the element type

vector<T>:N        vector<U>:N

In many cases, this is neither ABI compatible, nor transitionable. Specific cases can be discussed, but do not rely on this for evolvability of your protocols.

yellow warning ABI Compatibility: DEPENDS

yellow warning Transition Considerations: DEPENDS

strings

Similar to vectors.

enums

Reordering members

enum E {           enum E {
  A = 1;             B = 2;
  B = 2;             A = 1;
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES

Renaming members

enum E {           enum E {
  A = 1;             A_NEW = 1;
  B = 2;             B = 2;
};                 };

green checkmark ABI Compatibility: YES

red x Transition Considerations: NO

Adding members

enum E {           enum E {
  A = 1;             A = 1;
  B = 2;             B = 2;
                     C = 3;
};                 };

green checkmark ABI Compatibility: YES

Transition Considerations: * C++ switch{} without default will break * Rust match without "_" will break

Removing members

enum E {           enum E {
  A = 1;             A = 1;
  B = 2;
};                 };

green checkmark ABI Compatibility: YES

Transition Considerations: * Code which uses E::B will break

protocol libraries & names

Renaming [Discoverable]

[Discoverable]     [Discoverable]
protocol P {       protocol P_new {
  M1() -> ();        M1() -> ();
  M2() -> ();        M2() -> ();
};                 };

red x ABI Compatibility: NO

  • Renaming breaks service discoverability; names are used for service paths in namespaces/Directories.

red x Transition Considerations: NO

Renaming non-[Discoverable]

protocol P {       protocol P_new {
  M1() -> ();        M1() -> ();
  M2() -> ();        M2() -> ();
};                 };

red x ABI Compatibility: NO

  • Protocol names are part of method ordinal hashes.

red x Transition Considerations: NO

Renaming the library

library A:         library A.new:

protocol P {       protocol P {
  M1() -> ();        M1() -> ();
  M2() -> ();        M2() -> ();
};                 };

red x ABI Compatibility: NO

  • Library names are part of method ordinal hashes for all protocols within them.

red x Transition Considerations: NO

protocol methods

Reordering members

protocol P {       protocol P {
  M1() -> ();        M2() -> ();
  M2() -> ();        M1() -> ();
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES

Renaming members

protocol P {       protocol P {
  M1() -> ();        M1_new() -> ();
  M2() -> ();        M2() -> ();
};                 };

green checkmark ABI Compatibility: YES

  • Use the [Selector] to retain compatibility:
    • protocol P {
        [Selector= "M1"]
        M1_new() -> ();
        M2() -> ();
      };

red x Transition Considerations: NO

Adding members

protocol P {       protocol P {
  M1() -> ();        M1() -> ();
  M2() -> ();        M2() -> ();
                     M3() -> ();
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES

  • Add [Transitional] to the new member:
    • protocol P {
        M1() -> ();
        M2() -> ();
        [Transitional="msg"]
        M3() -> ();
      }
  • See FTP-021

Removing members

protocol P {       protocol P {
  M1() -> ();        M1() -> ();
  M2() -> ();
};                 };

green checkmark ABI Compatibility: YES

green checkmark Transition Considerations: YES

  1. Add [Transitional] to M2()
  2. Remove references
  3. Delete from .fidl

protocol method arguments & return values

Follow the same rules as structs.