Advanced Solidity and Yul

Types in Yul

Yul only has one type. bytes32.

Every data type represented by solidity is just a 32-byte value in yul.

Solidity implicitly converts the 32-byte value to either uint, bool, address, etc.

yul 1 → bool true | yul 1 → uint 1 | yul “0x1” → uint 1 | yul → “0x0100AF…” - >string hello world etc

Yul cannot access memory variables in a function

You cannot add a value more than 32 bytes in Yul.

function getNumber() external pure returns (uint256) {
    uint256 x;
    assembly {
        x := 10
    }
    return x;
}

function getString() external pure returns(bytes32) {
    bytes32 myString;
    assembly {
        myString := "hello world"
    }
    return myString;
}

Basic operations

function isPrime(uint256 x) external pure returns(bool p) {
    p = true;
    assembly{
        let halfX := add(div(x, 2), 1)
        for {let i:= 2} lt(x, halfX) {i := add(x, 1)} {
            if iszero(mod(x, i)) {
                p := 0
                break
            }
        }
    }
}

Yul doesn’t have terenary operators.

Falsy value is 0 in Yul

Don’t use the not function to negate in Yul. It’ll apply not to the 32 byte represenation of the number. Use iszero

eg: not(2) ≠ 0; iszero(2) ≠ 0

Storage Slots

The below code will not work in Yul. It cannot directly access storage variables like this.

uint256 x;
function getXYul() external view returns(uint256 ret) {
    assembly {
        ret := x
    }
}

You can use the .slot function to get the storage slot of the variable and sload to load it to a variable.

uint256 x;

function getXYul() external view returns(uint256 ret) {
    assembly {
        ret := sload(x.slot)
    }
}

You can set the value of a variable using the sstore function

uint256 x;
function setVarYul(uint256 value) external  {
    assembly {
        sstore(x.slot, value)
    }
}

Here both a and b will be in the same storage slot

uint128 a = 1;
uint128 b = 2;

function getASlot() external pure returns(uint256 slot) {
    assembly {
        slot := a.slot
    }
}

function getBSlot() external pure returns(uint256 slot) {
    assembly {
        slot := b.slot
    }
}

function getA() external view returns (bytes32 value) {
    assembly {
        value := sload(a.slot)
    }
}

function getB() external view returns (bytes32 value) {
    assembly {
        value := sload(b.slot)
    }
}

both getA and getB will return 0x0000000000000000000000000000000200000000000000000000000000000001

when you use variables with smaller size than 32 bytes, the compiler tries to pack them in together and save it in the same slot to save storage cost.

uint128 C = 4;
uint96 D = 6;
uint16 E = 8;
uint8 F = 1;

here all the variables will be stored in the same slot.

How do we get the right variable from the slot?

function getOffsetE() external view returns(uint256 slot, uint256 offset) {
    assembly {
        slot := E.slot
        offset := E.offset
    }
}

In this case the slot will be 0 and the offset will be 28. This means E is stored 28 bytes from the right.

Here’s how you can get the value of E using the offset, bit shifting and masking

function readE() external view returns(uint16 e) {
    assembly {
        let value := sload(E.slot)
        // value = 0x0001000800000000000000000000000600000000000000000000000000000004

        let shifted := shr(mul(E.offset, 8), value) // multiply by 8 because 1 byte = 8 bits and shr takes bits not bytes
        // shifted = 0x0000000000000000000000000000000000000000000000000000000000010008

        e := and(0xffff, shifted) // mask the last 4 bytes to 1 and the rest to 0
    }
}

When smaller variables are packed in a single 32-bit storage slot, this is how you write a to a single variable

function writeToE(uint16 newE) external {
  assembly {
    let currentVal := sload(E.slot)
                            // 0x0001000800000000000000000000000600000000000000000000000000000004
    let clearedE := and(currentVal, 0xffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff)

    // mask =       0xffff0000ffffffffffffffffffffffffffffffffffffffffffffffffffffffff
    // currentVal = 0x0001000800000000000000000000000600000000000000000000000000000004
    // clearedE =   0x0001000000000000000000000000000600000000000000000000000000000004

    let shiftedNewE := shl(mul(E.offset, 8), newE)
    let newVal := or(clearedE, shiftedNewE)
    // shiftedNewE = 0x0000000a00000000000000000000000000000000000000000000000000000000
    // clearedE =    0x0001000000000000000000000000000600000000000000000000000000000004
    // newCal =      0x0001000a00000000000000000000000600000000000000000000000000000004
    sstore(E.slot, newVal)

  }
}

Storage for Arrays and Mapping

uint256[3] fixedArray;
uint256[] dynamicArray;
uint8[] smallArray;

mapping(uint256 => uint256) public myMapping;
mapping(uint256 => mapping(uint256 => uint256)) nestedMapping;
mapping(address => uint256[]) addressToList;

constructor() {
  fixedArray = [99, 100, 101];
  dynamicArray = [50, 20, 45];
  smallArray = [1, 2, 3];

  myMapping[10] = 5;
  myMapping[11] = 6;
  nestedMapping[2][4] = 7;

  addressToList[0x5B38Da6a701c568545dCfcB03FcB875f56beddC4] = [
      42,
      1337,
      777
  ];
    
}

here we have a fixed array, dynamic array, a small sized array, mapping, nested mapping and mapping to an array. Let’s look at how the variables are stored in each case.

Fixed Array
- uint256[3] fixedArray
- In a fixed array, the storage slots for each elements are simply array.slot + index
- If you have a fixed array of length 3, it’ll take 3 consecutive storage slots.
- It’s similar to just declaring 3 variables back to back.
```
function fixedArrayView(uint256 index) external view returns (uint256 ret) {
    assembly {
        ret := sload(add(fixedArray.slot, index))
    }
}
```

Dynamic Array

uint256[] dynamicArray

In a dynamic array, the length of the array is stored in dynamicArray.slot.

function dynamicArrayLength() external view returns (uint256 ret) {
      assembly {
          ret := sload(dynamicArray.slot)
      }
    }

Each index of the array is stored in keccack256(abi.encode(dynamicArray.slot)) + index

function readdynamicArrayLocation(uint256 index)
    external
    view
    returns (uint256 ret)
{
    uint256 slot;
    assembly {
        slot := dynamicArray.slot
    }
    bytes32 location = keccak256(abi.encode(slot));

    assembly {
        ret := sload(add(location, index))
    }
}

Small Array
- uint8 smallArraay[]
- In the small array, solidity performs variable packing into 32 byte storage slots.

Mappings

mapping(uint256 => uint256) public myMapping;

A map stores the value in slot of hash of the mapping’s slot and the key keccak256(abi.encode(myMapping.slot, key))

function getMapping(uint256 key) external view returns (uint256 ret) {
    uint256 slot;
    assembly {
        slot := myMapping.slot
    }

    bytes32 location = keccak256(abi.encode(key, uint256(slot)));

    assembly {
        ret := sload(location)
    }
}

Nested mapping

mapping(uint256 => mapping(uint256 => uint256)) nestedMapping;

Nested mappings are just hashes of hashes

function getNestedMapping(uint256 key1, uint256 key2) external view returns (uint256 ret) {
    uint256 slot;
    assembly {
        slot := nestedMapping.slot
    }

    bytes32 location = keccak256(
        abi.encode(
            uint256(key1),
            keccak256(abi.encode(uint256(key2), uint256(slot)))
        )
    );
    assembly {
        ret := sload(location)
    }
}

Mapping to an Array

mapping(address => uint256[]) addressToList;

The length of the array is stored in the hash of the key and the storage slot of the mapping

function lengthOfNestedList() external view returns (uint256 ret) {
    uint256 addressToListSlot;
    assembly {
        addressToListSlot := addressToList.slot
    }

    bytes32 location = keccak256(
        abi.encode(
            address(0x5B38Da6a701c568545dCfcB03FcB875f56beddC4),
            uint256(addressToListSlot)
        )
    );
    assembly {
        ret := sload(location)
    }
  }

To get the location of each element in the array, get the hash of the slot in which the length is stored and add the index of the element to it.

  function getAddressToList(uint256 index) external view returns (uint256 ret)
  {
      uint256 slot;
      assembly {
          slot := addressToList.slot
      }

      bytes32 location = keccak256(
          abi.encode(
              keccak256(
                  abi.encode(
                      address(0x5B38Da6a701c568545dCfcB03FcB875f56beddC4),
                      uint256(slot)
                  )
              )
          )
      );
      assembly {
          ret := sload(add(location, index))
      }
  }

Memory in Solidity

Memory is required for
- return values for external calls
- see the function arguments for external calls
- get values from external calls
- revert with an error string
- log messages
- create other smart contracts
- use the keccak256 hash function

Memory in solidity is the equivalent to heap in other languages

But solidity does not have a garbage collector or a free command

Solidity memory is laid out in 32 byte sequences

[0x00 - 0x20] [0x20 - 0x40] [0x40 - 0x60] [0x60 - 0x80] .....

Memory only has 4 operations mstore, mload , mstore8 and msize

In pure Yul program, memory is easy to use. But it gets tricky with solidity + Yul since solidity expects memory to be used in a certain way.

Memory is cheaper compared to storage. But the further memory you try to access, the more gas it takes. This is to disincentivise contracts from using too much of the node’s memory.

eg mload(0xffffffffffffffffffffffffff) will run out of gas

so you can’t randomly access memory from hash like we did for storage.

mstore(p, v) - stores v in memory slot p

mload(p) - retreives 32 bytes from memory slot p[p, p+0x20]

mstore8(p, v) - like mload but for 1 byte

msize - largest accessed memory index in the transaction

mstore works very similar to sstore . It’ll store the value in the 32 byte slot.

With mstore8, it only store the data in the given byte

How Solidity uses Memory

Solidity allocates slots [0x00, 0x20), [0x20, 0x40) (0 - 63) for “scratch space”. You can write values here and expect it to be ephermal.

Solidity reserves slot [0x40, 0x60) as the free memory pointer. If you want to write something new to memory, this slot tells you where the next free memory is

Solidity keeps [0x60, 0x80) empty

You start writing your memory variables starting from 0x80

In the below code we are first checking the value of the free memory pointer and then storing a struct to memory and checking the free memory pointer again

The first emitted event will show the free memory to be 0x80 and the next event will show it as 0xc0.

The struct holds 64 bytes of data (2 * 32 bytes(uint256)).

0xc0 - 0x80 = 64 . The free memory pointer moved by 64 bytes to account for P in memory.

struct Points {
    uint256 a;
    uint256 b;
}

event MemoryPointer(bytes32);
event MemoryPointerMsize(bytes32, bytes32);

function memoryPointer() external {
    bytes32 x40;
    assembly {
        x40 := mload(0x40)
    }
    
    emit MemoryPointer(x40);

    Points memory p = Points({
        a: uint256(10),
        b: uint256(20)
    });

    assembly {
        x40 := mload(0x40)
    }

    emit MemoryPointer(x40);
}

msize gives the farthest accessed memory in the transaction.

The first event will emit 0x80 and 0x60 because the free memory starts from 0x80 but we haven’t written anything to memory yet. So the msize is 0x60

The second event will emit 0xc0 and 0xc0 becuase the free memory starts from 0xc0 and the biggest memory read is also 0xc0

The third event will emit 0xc0 and 0x120 because we read from a further away byte

function memoryPointerV2() external {
    bytes32 x40;
    bytes32 _msize;
    assembly {
        x40 := mload(0x40)
        _msize := msize()
    }
    
    emit MemoryPointerMsize(x40, _msize);

    Points memory p = Points({
        a: uint256(10),
        b: uint256(20)
    });

    assembly {
        x40 := mload(0x40)
        _msize := msize()
    }

    emit MemoryPointerMsize(x40, _msize);
    
    assembly {
        pop(mload(0xff))
        x40 := mload(0x40)
        _msize := msize()
    }

    emit MemoryPointerMsize(x40, _msize);
}

Arrays of fixed length will behave similar to the struct. The below code will show similar results to the struct code.

function fixedArray() external {
    bytes32 x40;
    assembly {
        x40 := mload(0x40)
    }
    emit MemoryPointer(x40);
    uint256[2] memory arr = [uint256(5), uint256(6)];
    assembly {
        x40 := mload(0x40)
    }
    emit MemoryPointer(x40);
}

abi.encode works similar to an array in terms of memory. But it stores extra 20 bytes in the starting of it’s memory. This is the total size of the arguments.

The first event will emit 0x80 and the second will emit 0xe0 . 0xe0 - 0x80 = 96 bytes . This contains 2 * 32 bytes(arguments) + 1 * 32 bytes(length) .

  function abiEncode() external {
      bytes32 x40;
      assembly {
          x40 := mload(0x40)
      }
      emit MemoryPointer(x40);
      abi.encode(uint256(5), uint256(19));
      assembly {
          x40 := mload(0x40)
      }
      emit MemoryPointer(x40);
  }

abi.encodePacked will pack the arguments if it’s less than 32 bytes to make it more effecient.

In the below example abi.encodePacked will only take 80 bytes of memory while abi.encode will take 96 bytes of memory for the same operation.

function abiEncodePacked() external {
    bytes32 x40;
    assembly {
        x40 := mload(0x40)
    }
    emit MemoryPointer(x40);
    abi.encodePacked(uint256(5), uint128(19));
    assembly {
        x40 := mload(0x40)
    }
    emit MemoryPointer(x40);
}

Solidity uses memory for
- abi.encode and abi.encodePacked
- structs and arrays. but you need to explicitly mention memory keyword
- since objects are laid end-to-end it’s not possible to grow arrays. that is why arrays in memory can’t do push unlike arrays in storage. becuase it might crash into the value next to it.

Yul
- The variable itself is where it begins in memory
- To access a dynamic array in Yul, you need to add 32 bytes or 0x20 to skip the length.
- Call this function passing [5, 7] as the argument. The location will be 0x80 since it’s the first variable. The length of the array will be stored in the location of the array. So mload(location) will return 2 which is the length of the array.
- Each value will be stored 32 bytes from the previous value starting from the location.
- val1 will be stored in location + 0x20
- val2 will be stored in location + 0x40
```
function dynamicArrayInMemory(uint256[] memory myArr) external {
    bytes32 location;
    bytes32 length;
    bytes32 val1;
    bytes32 val2;

    assembly {
        location := myArr
        length := mload(myArr)
        val1 := mload(add(location, 0x20))
        val2 := mload(add(location, 0x40))
    }

    emit Debug(location, length, val1, val2);

}
```