| SkSL Binary Format |
| ================== |
| |
| Background |
| ---------- |
| |
| The SkSL binary format was originally created for internal use, to store our built-in include files |
| in a format which required less processing time than the original text versions. |
| |
| This improved compiler startup time and memory footprint, but as the format was purpose-built for |
| encoding our include files, it could not encode a normal SkSL program. It only supported the subset |
| of SkSL required by the include files, had no means of referring to external symbols, and had not |
| been tested outside of being used to encode our small handful of include files. |
| |
| To both improve testability and to support the needs of clients requiring a binary SkSL format, this |
| feature has been extended to support arbitrary SkSL files. Full test coverage is achieved by |
| encoding our entire corpus of SkSL tests to binary, decoding them, and ensuring that the decoded |
| output program matches the unmodified original program. |
| |
| Design |
| ------ |
| |
| ### Overview |
| |
| A binary SkSL file has the following header: |
| |
| | Type | Field Name | |
| |----------------------|--------------| |
| | `uint16` | version | |
| | `uint16` | stringLength | |
| | `char[stringLength]` | stringData | |
| |
| The version number is incremented whenever the file format changes. This document describes version |
| 8. |
| |
| `stringLength` is the total length of all of the string data in the file, including the length bytes |
| of the strings, but not counting the `stringLength` field itself. Each string consists of a `uint8` |
| length followed by the string’s characters, which are not null terminated. |
| |
| The header is immediately followed by a sequence of one or more commands encoding the contents. A |
| typical SkSL binary will contain exactly one `kProgram_Command`. |
| |
| ### Types |
| |
| The following types may appear as data members of commands. All numbers are stored in little endian |
| byte ordering. |
| |
| | Name | Description | |
| |------------------------|-------------------------------------------------------------------------| |
| | `int8` | a signed 8 bit integer | |
| | `int16` | a signed 16 bit integer | |
| | `int32` | a signed 32 bit integer | |
| | `uint8` | an unsigned 8 bit integer | |
| | `uint16` | an unsigned 16 bit integer | |
| | `uint32` | an unsigned 32 bit integer | |
| | `float` | a 32 bit IEEE floating point number | |
| | `bool` | a single byte with the value either 0 (`false`) or 1 (`true`) | |
| | `String` | a `uint8` length, followed by a `uint16` offset [^1] | |
| | `ProgramKind` | a `uint8` mapping to a value in the `SkSL::ProgramKind` enum | |
| | `VariableStorage` | a `uint8` mapping to a value in the `SkSL::VariableStorage` enum | |
| | `Operator` | a `uint8` mapping to a value in the `SkSL::Token::Kind` enum | |
| | `FieldAccessOwnerKind` | a `uint8` mapping to a value in the `SkSL::FieldAccessOwnerKind` enum | |
| | `VariableRefKind` | a `uint8` mapping to a value in the `SkSL::VariableRefKind` enum | |
| | `SwizzleComponent` | a `uint8` mapping to a value in the `SkSL::SwizzleComponent::Type` enum | |
| | `Symbol` | an instance of any command in the Symbol category | |
| | `SymbolId` | a `uint16` ID specified by a Symbol category command [^2] | |
| | `Expression` | an instance of any command in the Expressions category | |
| | `ProgramElement` | an instance of any command in the ProgramElements category | |
| | `Statement` | an instance of any command in the Statements category | |
| | `Type` | an instance of one of the “Type” Symbol commands [^3] | |
| | `Layout` | an instance of the one of the Layout commands [^4] | |
| | `Modifiers` | an instance of one of the Modifiers commands [^5] | |
| | `k*_Command` | an instance of the specified command | |
| |
| [^1]: The offset of a `String` is relative to the beginning of the `stringData` field of the header. |
| |
| [^2]: There is one exception to `SymbolId`s referring to Symbol commands: if a `SymbolId` is |
| `kBuiltin_Symbol`, the `SymbolId` is followed by a String representing the name of the symbol. |
| |
| [^3]: The Type symbol commands are `kArrayType_Command`, `kStructType_Command`, or a |
| `kSymbolRef_Command` referring to an existing type. |
| |
| [^4]: The "Layout" commands are `kBuiltinLayout_Command`, `kDefaultLayout_Command`, and |
| `kLayout_Command`. |
| |
| [^5]: The "Modifiers" commands are `kDefaultModifiers_Command`, `kModifiers8Bit_Command`, and |
| `kModifiers_Command)`. |
| |
| Commands |
| -------- |
| |
| Each command consists of a single `uint8` identifying the command, followed by its data. |
| |
| ### General Commands |
| |
| #### kBuiltinLayout_Command |
| |
| | Type | Field Name | |
| |---------|------------| |
| | `int16` | builtin | |
| |
| An `SkSL::Layout` with the specified builtin value, but all other fields left with their default |
| values. |
| |
| --- |
| |
| #### kDefaultLayout_Command |
| |
| An `SkSL::Layout` with every field set to its default value. |
| |
| --- |
| |
| #### kDefaultModifiers_Command |
| |
| An `SkSL::Modifiers` with every field set to its default value. |
| |
| --- |
| |
| #### kElements_Command |
| |
| | Type | Field Name | |
| |-----------------------------|------------| |
| | `ProgramElement[]` | elements | |
| | `kElementsComplete_Command` | | |
| |
| The end of the elements array is indicated by the presence of `kElementsComplete_Command`. |
| |
| --- |
| |
| #### kElementsComplete_Command |
| |
| Signifies the end of a list of elements. |
| |
| --- |
| |
| #### kLayout_Command |
| |
| | Type | Field Name | |
| |---------|----------------------| |
| | `int32` | flags | |
| | `int8` | location | |
| | `int16` | offset | |
| | `int16` | binding | |
| | `int8` | index | |
| | `int8` | set | |
| | `int16` | builtin | |
| | `int8` | inputAttachmentIndex | |
| |
| Represents an `SkSL::Layout` with all fields specified. |
| |
| --- |
| |
| #### kModifiers8Bit_Command |
| |
| | Type | Field Name | |
| |----------|------------| |
| | `Layout` | layout | |
| | `uint8` | flags | |
| |
| A shortened version of `Modifiers` which only represents the least significant 8 bits of `flags`, |
| leaving the other bits as zero. |
| |
| --- |
| |
| #### kModifiers_Command |
| |
| | Type | Field Name | |
| |----------|------------| |
| | `Layout` | layout | |
| | `uint32` | flags | |
| |
| --- |
| |
| #### kProgram_Command |
| |
| | Type | Field Name | |
| |------------------------|------------------| |
| | `ProgramKind` | programKind | |
| | `kSymbolTable_Command` | symbolTable | |
| | `kElements_Command` | elements | |
| | `bool` | useFlipRTUniform | |
| |
| `useFlipRTUniform` refers to a field in `Program::Inputs`. |
| |
| --- |
| |
| #### kSymbolTable_Command |
| |
| | Type | Field Name | |
| |----------------------------|------------------| |
| | `uint16` | ownedSymbolCount | |
| | `Symbol[ownedSymbolCount]` | ownedSymbols | |
| | `uint16` | symbolCount | |
| | `SymbolID[symbolCount]` | symbols | |
| |
| The IDs in the symbols array do not relate to indices in `ownedSymbols`, and instead refer to the |
| IDs specified at the beginning of commands in the Symbol category. |
| |
| --- |
| |
| ### ProgramElement Commands |
| |
| #### kFunctionDefinition_Command |
| |
| | Type | Field Name | |
| |-------------|-------------| |
| | `SymbolId` | declaration | |
| | `Statement` | body | |
| |
| Provides the body corresponding to a `FunctionDeclaration`. |
| |
| --- |
| |
| #### kFunctionPrototype_Command |
| |
| | Type | Field Name | |
| |----------|-------------| |
| | `uint16` | declaration | |
| |
| Represents a function declaration appearing in the program as a function prototype. |
| |
| --- |
| |
| #### kInterfaceBlock_Command |
| |
| | Type | Field Name | |
| |------------|--------------| |
| | `SymbolId` | var | |
| | `String` | typeName | |
| | `String` | instanceName | |
| | `uint8` | arraySize | |
| |
| An interface block. `instanceName` may be zero-length to indicate the lack of an instance name. |
| `arraySize` will be zero if the instance name was not present or was not followed by an array size. |
| |
| --- |
| |
| #### kStructDefinition_Command |
| |
| | Type | Field Name | |
| |--------|------------| |
| | `Type` | structType | |
| |
| Represents a struct definition appearing at top-level scope. |
| |
| --- |
| |
| #### kSharedFunction_Command |
| |
| | Type | Field Name | |
| |----------------------------|----------------| |
| | `uint8` | parameterCount | |
| | `Variable[parameterCount]` | parameters | |
| | `FunctionDeclaration` | decl | |
| | `FunctionDefinition` | defn | |
| |
| Represents a function from `Program.fSharedElements`. |
| |
| --- |
| |
| #### kGlobalVar_Command |
| |
| | Type | Field Name | |
| |---------------------------|------------| |
| | `kVarDeclaration_Command` | decl | |
| |
| A `VarDeclaration` appearing in global scope. |
| |
| --- |
| |
| ### Symbol Commands |
| |
| #### kArrayType_Command |
| |
| | Type | Field Name | |
| |----------|---------------| |
| | `uint16` | id | |
| | `Type` | componentType | |
| | `uint8` | count | |
| |
| --- |
| |
| #### kField_Command |
| |
| | Type | Field Name | |
| |------------|------------| |
| | `SymbolId` | ownerId | |
| | `uint8` | index | |
| |
| Symbol referring to a particular field of a `StructType`. Unlike other symbols, Field does not have |
| its own ID; references to a Field are compiled in terms of the owner’s ID. |
| |
| --- |
| |
| #### kFunctionDeclaration_Command |
| |
| | Type | Field Name | |
| |----------------------------|----------------| |
| | `uint16` | id | |
| | `Modifiers` | modifiers | |
| | `String` | name | |
| | `uint8` | parameterCount | |
| | `SymbolId[parameterCount]` | parameters | |
| | `Type` | returnType | |
| |
| --- |
| |
| #### kStructType_Command |
| |
| | Type | Field Name | |
| |---------------------|------------| |
| | `uint16` | id | |
| | `String` | name | |
| | `uint8` | fieldCount | |
| | `Field[fieldCount]` | fields | |
| |
| The `Field` type referenced in this command is: |
| |
| struct Field { |
| Modifiers modifiers; |
| String name; |
| Type type; |
| }; |
| |
| --- |
| |
| #### kSymbolRef_Command |
| |
| | Type | Field Name | |
| |------------|------------| |
| | `SymbolId` | id | |
| |
| Refers to a Symbol which has already been written by another command. |
| |
| --- |
| |
| #### kUnresolvedFunction_Command |
| |
| | Type | Field Name | |
| |------------------------------|------------| |
| | `uint16` | id | |
| | `uint8` | count | |
| | `FunctionDeclaration[count]` | functions | |
| |
| --- |
| |
| #### kVariable_Command |
| |
| | Type | Field Name | |
| |----------------------|------------| |
| | `uint16` | id | |
| | `Modifiers` | modifiers | |
| | `String` | name | |
| | `Type` | type | |
| | `VariableStorage` | storage | |
| |
| --- |
| |
| ### Expression Commands |
| |
| #### kBinary_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Expression` | left | |
| | `Operator` | op | |
| | `Expression` | right | |
| | `Type` | type | |
| |
| Represents a binary operator applied to two expressions. |
| |
| --- |
| |
| #### kBoolLiteral_Command |
| |
| | Type | Field Name | |
| |--------|------------| |
| | `bool` | value | |
| |
| Represents a literal `true` or `false` value. |
| |
| --- |
| |
| #### kConstructorArray_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorArray`. |
| |
| --- |
| |
| #### kConstructorArrayCast_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorArrayCast`. |
| |
| --- |
| |
| #### kConstructorCompound_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorCompound`. |
| |
| --- |
| |
| #### kConstructorCompoundCast_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorCompoundCast`. |
| |
| --- |
| |
| #### kConstructorDiagonalMatrix_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorDiagonalMatrix`. |
| |
| --- |
| |
| #### kConstructorMatrixResize_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorMatrixResize`. |
| |
| --- |
| |
| #### kConstructorScalarCast_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorScalarCast`. |
| |
| --- |
| |
| #### kConstructorSplat_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorSplat`. |
| |
| --- |
| |
| #### kConstructorStruct_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents an instance of `SkSL::ConstructorStruct`. |
| |
| --- |
| |
| #### kFieldAccess_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Expression` | base | |
| | `uint8` | index | |
| | `FieldAccessOwnerKind` | ownerKind | |
| |
| A reference to the `index`’th field of the `base` struct. |
| |
| --- |
| |
| #### kFloatLiteral_Command |
| |
| | Type | Field Name | |
| |---------|------------| |
| | `float` | value | |
| |
| Represents a floating point literal. |
| |
| --- |
| |
| #### kFunctionCall_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Type` | type | |
| | `SymbolId` | function | |
| | `uint8` | argCount | |
| | `Expression[argCount]` | arguments | |
| |
| Represents a call to `function(arguments...)`. |
| |
| --- |
| |
| #### kIndex_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Expression` | base | |
| | `Expression` | index | |
| |
| Represents the expression `base[index]`. |
| |
| --- |
| |
| #### kIntLiteral_Command |
| |
| | Type | Field Name | |
| |---------|------------| |
| | `int32` | value | |
| |
| Represents an integer literal. |
| |
| --- |
| |
| #### kPostfix_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Operator` | op | |
| | `Expression` | operand | |
| |
| Represents applying the postfix operator `op` to `operand`. |
| |
| --- |
| |
| #### kPrefix_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Operator` | op | |
| | `Expression` | operand | |
| |
| Represents applying the prefix operator `op` to `operand`. |
| |
| --- |
| |
| #### kSetting_Command |
| |
| | Type | Field Name | |
| |----------|------------| |
| | `String` | name | |
| |
| Represents a field of `sk_Caps`, such as `“builtinDeterminantSupport”`. |
| |
| --- |
| |
| #### kSwizzle_Command |
| |
| | Type | Field Name | |
| |------------------------------------|----------------| |
| | `Expression` | base | |
| | `uint8` | componentCount | |
| | `SwizzleComponent[componentCount]` | components | |
| |
| Represents the swizzle `base.components`. |
| |
| --- |
| |
| #### kTernary_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Expression` | test | |
| | `Expression` | ifTrue | |
| | `Expression` | ifFalse | |
| |
| Represents the ternary expression `test ? ifTrue : ifFalse`. |
| |
| --- |
| |
| #### kVariableReference_Command |
| |
| | Type | Field Name | |
| |-------------------|------------| |
| | `SymbolID` | var | |
| | `VariableRefKind` | refKind | |
| |
| Represents a reference to the variable `var`. |
| |
| --- |
| |
| ### Statement Commands |
| |
| #### kBlock_Command |
| |
| | Type | Field Name | |
| |-----------------------------|----------------| |
| | `kSymbolTable_Command` | symbolTable | |
| | `uint8` | statementCount | |
| | `Statement[statementCount]` | statements | |
| | `bool` | isScope | |
| |
| A block of statements. |
| |
| --- |
| |
| #### kBreak_Command |
| |
| A `break` statement. |
| |
| --- |
| |
| #### kContinue_Command |
| |
| A `continue` statement. |
| |
| --- |
| |
| #### kDiscard_Command |
| |
| A `discard` statement. |
| |
| --- |
| |
| #### kDo_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Statement` | stmt | |
| | `Expression` | test | |
| |
| Represents `do stmt; while(test);`. |
| |
| --- |
| |
| #### kExpressionStatement_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `Expression` | expression | |
| |
| Represents the statement `expression;`. |
| |
| --- |
| |
| #### kFor_Command |
| |
| | Type | Field Name | |
| |---------------|-------------| |
| | `Statement` | initializer | |
| | `Expression` | test | |
| | `Expression` | next | |
| | `Statement` | body | |
| | `SymbolTable` | symbols | |
| |
| Represents `for (initializer; test; next) body;`. |
| `test`, `next`, and `body` may all be `kVoid_Command` to represent their absence. |
| |
| --- |
| |
| #### kIf_Command |
| |
| | Type | Field Name | |
| |--------------|------------| |
| | `bool` | isStatic | |
| | `Expression` | test | |
| | `Statement` | ifTrue | |
| | `Statement` | ifFalse | |
| |
| Represents `if (test) ifTrue; else ifFalse;` (or `@if`, if `isStatic` is true). |
| `ifFalse` may be `kVoid_Command` to represent the absence of an `else` statement. |
| |
| --- |
| |
| #### kInlineMarker_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `SymbolId` | function | |
| |
| Represents an `SkSL::InlineMarker`, which is inserted before an inlined function’s code. |
| |
| --- |
| |
| #### kNop_Command |
| |
| Represents an empty statement (a bare semicolon). |
| |
| --- |
| |
| #### kReturn_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `Expression` | value | |
| |
| Represents the statement `return value;`. |
| `value` may be `kVoid_Command` to represent the absence of a return value. |
| |
| --- |
| |
| #### kSwitch_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `bool` | isStatic | |
| | `kSymbolTable_Command` | symbols | |
| | `Expression` | value | |
| | `uint8` | caseCount | |
| | `Case[caseCount]` | cases | |
| |
| The `Case` type referenced in this command is: |
| |
| struct Case { |
| bool isDefault; |
| (when !isDefault) Expression value; |
| uint8 statementCount; |
| Statement statements[statementCount]; |
| }; |
| |
| Represents a `switch` or `@switch` statement on value. Each case begins with a `bool` identifying |
| whether or not it is the default case; the default case does not contain the `value` field. |
| |
| --- |
| |
| #### kVarDeclaration_Command |
| |
| | Type | Field Name | |
| |------------------------|------------| |
| | `SymbolId` | var | |
| | `Type` | baseType | |
| | `uint8` | arraySize | |
| | `Expression` | value | |
| |
| Represents the variable declaration statement `baseType var[arraySize] = value;`. |
| An `arraySize` of zero omits the array declaration. A value of `kVoid_Command` omits the |
| initializer. |
| |
| --- |
| |
| #### kVoid_Command |
| |
| A marker indicating that a node is not present. |
