---
title: Serialize Custom Instruction Data for Native Program Development
objectives:
  - Explain the contents of a transaction
  - Explain transaction instructions
  - Explain the basics of Solana's runtime optimizations
  - Explain Borsh
  - Use Borsh to serialize program data for native programs
description: How to deserialize data fetched from Solana accounts.
---

## Summary

- Native (non-Anchor) Solana development requires manual serialization and
  deserialization of data.
- Transactions are made up of an array of instructions, a single transaction can
  have any number of instructions in it, each targeting different programs. When
  a transaction is submitted, the Solana runtime will process its instructions
  in order and atomically, meaning that if any of the instructions fail for any
  reason, the entire transaction will fail to be processed.
- Every _instruction_ is made up of 3 components: the intended program's ID, an
  array of all accounts involved, and a byte buffer of instruction data.
- Every _transaction_ contains an array of all accounts it intends to read or
  write, one or more instructions, a recent blockhash, and one or more
  signatures.
- To pass instruction data from a client, it must be serialized into a byte
  buffer. To facilitate this process of serialization, we will be using
  [Borsh](https://borsh.io/).
- Transactions can fail to be processed by the blockchain for any number of
  reasons, we'll discuss some of the most common ones here.

## Lesson

### Transactions

<Callout type="info">

This course requires completing
[Introduction to Solana](/developers/courses/intro-to-solana) or equivalent
knowledge. It's also aimed at advanced developers that prefer more control over
the ease of use and safe defaults Anchor provides. If you're new to developing
onchain programs you may prefer
[Anchor](/developers/courses/onchain-development)

</Callout>

In [Introduction to Solana](/developers/courses/intro-to-solana) we learned how to
create transactions with instructions for common Solana programs.

This lessons shows how to create instructions for our own native Solana
programs, which we will develop in a few lessons. Specifically, we're going to
learn about serialization and deserialization, which is required for native
(non-Anchor) program development.

#### Transaction Contents

Every transaction contains:

- An array that includes every account it intends to read or write
- One or more instructions
- A recent blockhash
- One or more signatures

`@solana/web3.js` simplifies this process for you so that all you need to focus
on is adding instructions and signatures. The library builds the array of
accounts based on that information and handles the logic for including a recent
blockhash.

### Instructions

Every instruction contains:

- The program ID (public key) of the intended program
- An array listing every account that will be read from or written to during
  execution
- A byte buffer of instruction data

Identifying the program by its public key ensures that the instruction is
carried out by the correct program.

Including an array of every account that will be read from or written to allows
the network to perform several optimizations which allow for high transaction
load and quicker execution.

The byte buffer lets you pass external data to a program.

You can include multiple instructions in a single transaction. The Solana
runtime will process these instructions in order and atomically. In other words,
if every instruction succeeds then the transaction as a whole will be
successful, but if a single instruction fails then the entire transaction will
fail immediately with no side-effects.

The account array is not just an array of the accounts' public keys. Each object
in the array includes the account's public key, whether or not it is a signer on
the transaction, and whether or not it is writable. Including whether or not an
account is writable during the execution of an instruction allows the runtime to
facilitate parallel processing of smart contracts. Because you must define which
accounts are read-only and which you will write to, the runtime can determine
which transactions are non-overlapping or read-only and allow them to execute
concurrently. To learn more about Solana's runtime, check out this
[blog post on Sealevel](/news/sealevel---parallel-processing-thousands-of-smart-contracts).

#### Instruction Data

The ability to add arbitrary data to an instruction ensures that programs can be
dynamic and flexible enough for broad use cases in the same way that the body of
an HTTP request lets you build dynamic and flexible REST APIs.

Just as the structure of the body of an HTTP request is dependent on the
endpoint you intend to call, the structure of the byte buffer used as
instruction data is entirely dependent on the recipient program. If you're
building a full-stack dApp on your own, then you'll need to copy the same
structure that you used when building the program over to the client-side code.
If you're working with another developer who is handling the program
development, you can coordinate to ensure matching buffer layouts.

Let's think about a concrete example. Imagine working on a Web3 game and being
responsible for writing client-side code that interacts with a player inventory
program. The program was designed to allow the client to:

- Add inventory based on a player's game-play results
- Transfer inventory from one player to another
- Equip a player with selected inventory items

This program would have been structured such that each of these is encapsulated
in its own function.

Each program, however, only has one entry point. You would instruct the program
on which of these functions to run through the instruction data.

You would also include in the instruction data any information the function
needs to execute properly, e.g. an inventory item's ID, a player to transfer
inventory to, etc.

Exactly _how_ this data would be structured would depend on how the program was
written, but it's common to have the first field in instruction data be a number
that the program can map to a function, after which additional fields act as
function arguments.

### Serialization

In addition to knowing what information to include in an instruction data
buffer, you also need to serialize it properly. The most common serializer used
in Solana is [Borsh](https://borsh.io). Per the website:

> Borsh stands for Binary Object Representation Serializer for Hashing. It is
> meant to be used in security-critical projects as it prioritizes consistency,
> safety, speed; and comes with a strict specification.

Borsh maintains a [JS library](https://github.com/near/borsh-js) that handles
serializing common types into a buffer. There are also other packages built on
top of Borsh that try to make this process even easier. We'll be using the
`@coral-xyz/borsh` library which can be installed using `npm`.

Building off of the previous game inventory example, let's look at a
hypothetical scenario where we are instructing the program to equip a player
with a given item. Assume the program is designed to accept a buffer that
represents a struct with the following properties:

1. `variant` is an unsigned, 8-bit integer that instructs the program which
   instruction, or function, to execute.
2. `playerId` is an unsigned, 16-bit integer that represents the player ID of
   the player who is to be equipped with the given item.
3. `itemId` is an unsigned, 256-bit integer that represents the item ID of the
   item that will be equipped for the given player.

All of this will be passed as a byte buffer that will be read in order, so
ensuring proper buffer layout order is crucial. You would create the buffer
layout schema or template for the above as follows:

```typescript
import * as borsh from "@coral-xyz/borsh";

const equipPlayerSchema = borsh.struct([
  borsh.u8("variant"),
  borsh.u16("playerId"),
  borsh.u256("itemId"),
]);
```

You can then encode data using this schema with the `encode` method. This method
accepts as arguments an object representing the data to be serialized and a
buffer. In the below example, we allocate a new buffer that's much larger than
needed, then encode the data into that buffer and slice the original buffer down
into a new buffer that's only as large as needed.

```typescript
import * as borsh from "@coral-xyz/borsh";

const equipPlayerSchema = borsh.struct([
  borsh.u8("variant"),
  borsh.u16("playerId"),
  borsh.u256("itemId"),
]);

const buffer = Buffer.alloc(1000);
equipPlayerSchema.encode(
  { variant: 2, playerId: 1435, itemId: 737498 },
  buffer,
);

const instructionBuffer = buffer.subarray(0, equipPlayerSchema.getSpan(buffer));
```

Once a buffer is properly created and the data serialized, all that's left is
building the transaction. This is similar to what you've done in previous
lessons. The example below assumes that:

- `player`, `playerInfoAccount`, and `PROGRAM_ID` are already defined somewhere
  outside the code snippet
- `player` is a user's public key
- `playerInfoAccount` is the public key of the account where inventory changes
  will be written
- `SystemProgram` will be used in the process of executing the instruction.

```typescript
import * as borsh from "@coral-xyz/borsh";
import {
  clusterApiUrl,
  Connection,
  SystemProgram,
  Transaction,
  TransactionInstruction,
  sendAndConfirmTransaction,
} from "@solana/web3.js";

const equipPlayerSchema = borsh.struct([
  borsh.u8("variant"),
  borsh.u16("playerId"),
  borsh.u256("itemId"),
]);

const buffer = Buffer.alloc(1000);
equipPlayerSchema.encode(
  { variant: 2, playerId: 1435, itemId: 737498 },
  buffer,
);

const instructionBuffer = buffer.subarray(0, equipPlayerSchema.getSpan(buffer));

const endpoint = clusterApiUrl("devnet");
const connection = new Connection(endpoint);

const transaction = new Transaction();
const instruction = new TransactionInstruction({
  keys: [
    {
      pubkey: player.publicKey,
      isSigner: true,
      isWritable: false,
    },
    {
      pubkey: playerInfoAccount,
      isSigner: false,
      isWritable: true,
    },
    {
      pubkey: SystemProgram.programId,
      isSigner: false,
      isWritable: false,
    },
  ],
  data: instructionBuffer,
  programId: PROGRAM_ID,
});

transaction.add(instruction);

try {
  const transactionId = await sendAndConfirmTransaction(
    connection,
    transaction,
    [player],
  );
  const explorerLink = getExplorerLink("transaction", transactionId, "devnet");
  console.log(`Transaction submitted: ${explorerLink}`);
} catch (error) {
  alert(error);
}
```

## Lab

Let's practice this together by building a Movie Review app that lets users
submit a movie review and have it stored on Solana's network. We'll build this
app a little bit at a time over the next few lessons, adding new functionality
each lesson.

![Movie review frontend](/assets/courses/movie-review-dapp.png)

Here's a quick diagram of the program we'll build:

![Solana stores data items in PDAs, which can be found using their seeds](/assets/courses/unboxed/movie-review-program.svg)

The public key of the Solana program we'll use for this application is
`CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN`.

#### 1. Download the starter code

Before we get started, go ahead and download the
[starter code](https://github.com/solana-developers/movie-review-frontend/tree/starter).

The project is a fairly simple Next.js application. It includes the
`WalletContextProvider` we created in the Wallets lesson, a `Card` component for
displaying a movie review, a `MovieList` component that displays reviews in a
list, a `Form` component for submitting a new review, and a `Movie.ts` file that
contains a class definition for a `Movie` object.

Note that for now, the movies displayed on the page when you run `npm run dev`
are mocks. In this lesson, we'll focus on adding a new review but we won't be
able to see that review displayed. Next lesson, we'll focus on deserializing
custom data from onchain accounts.

#### 2. Create the buffer layout

Remember that to properly interact with a Solana program, you need to know how
it expects data to be structured. Our Movie Review program expects instruction
data to contain:

1. `variant` as an unsigned, 8-bit integer representing which instruction should
   be executed (in other words which function on the program should be called).
2. `title` as a string representing the title of the movie that you are
   reviewing.
3. `rating` as an unsigned, 8-bit integer representing the rating out of 5 that
   you are giving to the movie you are reviewing.
4. `description` as a string representing the written portion of the review you
   are leaving for the movie.

Let's configure a `borsh` layout in the `Movie` class. Start by importing
`@coral-xyz/borsh`. Next, create a `borshInstructionSchema` property and set it
to the appropriate `borsh` struct containing the properties listed above.

```typescript
import * as borsh from '@coral-xyz/borsh'

export class Movie {
  title: string;
  rating: number;
  description: string;

  ...

  borshInstructionSchema = borsh.struct([
    borsh.u8('variant'),
    borsh.str('title'),
    borsh.u8('rating'),
    borsh.str('description'),
  ])
}
```

Keep in mind that _order matters_. If the order of properties here differs from
how the program is structured, the transaction will fail.

#### 3. Create a method to serialize data

Now that we have the buffer layout set up, let's create a method in `Movie`
called `serialize()` that will return a `Buffer` with a `Movie` object's
properties encoded into the appropriate layout.

Instead of allocating a fixed buffer size, we'll calculate the size dynamically
using known constants for the space required by each field in the `Movie`
object. Specifically, we'll use `INIT_SPACE` (to account for string length
metadata) and `ANCHOR_DISCRIMINATOR` (to account for the 8-byte discriminator
used by Anchor).

```typescript
import * as borsh from "@coral-xyz/borsh";

// Constants for size calculations
const ANCHOR_DISCRIMINATOR = 8; // 8 bytes for the account discriminator used by Anchor
const STRING_LENGTH_SPACE = 4; // 4 bytes to store the length of each string

// Specific sizes for 'title' and 'description' strings
const TITLE_SIZE = 100; // Allocate 100 bytes for the 'title'
const DESCRIPTION_SIZE = 500; // Allocate 500 bytes for the 'description'

// Total space calculation for the Movie review structure
const MOVIE_REVIEW_SPACE =
  ANCHOR_DISCRIMINATOR + // 8 bytes for the account discriminator
  STRING_LENGTH_SPACE +
  TITLE_SIZE + // 4 bytes for the title length + 100 bytes for the title
  STRING_LENGTH_SPACE +
  DESCRIPTION_SIZE + // 4 bytes for the description length + 500 bytes for the description
  1 + // 1 byte for 'variant'
  1; // 1 byte for 'rating'

export class Movie {
  title: string;
  rating: number;
  description: string;

  constructor(title: string, rating: number, description: string) {
    // Enforce specific sizes for title and description
    if (title.length > TITLE_SIZE) {
      throw new Error(`Title cannot exceed ${TITLE_SIZE} characters.`);
    }
    if (description.length > DESCRIPTION_SIZE) {
      throw new Error(
        `Description cannot exceed ${DESCRIPTION_SIZE} characters.`,
      );
    }

    this.title = title;
    this.rating = rating;
    this.description = description;
  }

  borshInstructionSchema = borsh.struct([
    borsh.u8("variant"),
    borsh.str("title"),
    borsh.u8("rating"),
    borsh.str("description"),
  ]);

  serialize(): Buffer {
    try {
      // Allocate a buffer with the exact space needed
      const buffer = Buffer.alloc(MOVIE_REVIEW_SPACE);
      this.borshInstructionSchema.encode({ ...this, variant: 0 }, buffer);
      return buffer.subarray(0, this.borshInstructionSchema.getSpan(buffer));
    } catch (error) {
      console.error("Serialization error:", error);
      return Buffer.alloc(0);
    }
  }
}
```

The method shown above first creates a large enough buffer for our object, then
encodes `{ ...this, variant: 0 }` into the buffer. Because the `Movie` class
definition contains 3 of the 4 properties required by the buffer layout and uses
the same naming, we can use it directly with the spread operator and just add
the `variant` property. Finally, the method returns a new buffer that leaves off
the unused portion of the original.

#### 4. Send a transaction when the user submits the form

Now that we have the building blocks for the instruction data, we can create and
send the transaction when a user submits the form. Open `Form.tsx` and locate
the `handleTransactionSubmit` function. This gets called by `handleSubmit` each
time a user submits the Movie Review form.

Inside this function, we'll be creating and sending the transaction that
contains the data submitted through the form.

Start by importing `@solana/web3.js` and importing `useConnection` and
`useWallet` from `@solana/wallet-adapter-react`.

```tsx
import { FC } from "react";
import { Movie } from "../models/Movie";
import { useState } from "react";
import {
  Connection,
  PublicKey,
  SystemProgram,
  Transaction,
  TransactionInstruction,
} from "@solana/web3.js";
import { useConnection, useWallet } from "@solana/wallet-adapter-react";
```

Next, before the `handleSubmit` function, call `useConnection()` to get a
`connection` object and call `useWallet()` to get `publicKey` and
`sendTransaction`.

```tsx
import { FC } from 'react'
import { Movie } from '../models/Movie'
import { useState } from 'react'
import {
  Connection,
  PublicKey,
  SystemProgram,
  Transaction,
  TransactionInstruction,
} from "@solana/web3.js"
import { useConnection, useWallet } from '@solana/wallet-adapter-react'
import { getExplorerLink } from "@solana-developers/helpers";

const MOVIE_REVIEW_PROGRAM_ID = 'CenYq6bDRB7p73EjsPEpiYN7uveyPUTdXkDkgUduboaN'

export const Form: FC = () => {
  const [title, setTitle] = useState('')
  const [rating, setRating] = useState(0)
  const [message, setMessage] = useState('')

  const { connection } = useConnection();
  const { publicKey, sendTransaction } = useWallet();

  const handleSubmit = (event: any) => {
    event.preventDefault()
    const movie = new Movie(title, rating, description)
    handleTransactionSubmit(movie)
  }

  ...
}
```

Before we implement `handleTransactionSubmit`, let's talk about what needs to be
done. We need to:

1. Check that `publicKey` exists to ensure that the user has connected their
   wallet.
2. Call `serialize()` on `movie` to get a buffer representing the instruction
   data.
3. Create a new `Transaction` object.
4. Get all of the accounts that the transaction will read or write.
5. Create a new `Instruction` object that includes all of these accounts in the
   `keys` argument, includes the buffer in the `data` argument, and includes the
   program's public key in the `programId` argument.
6. Add the instruction from the last step to the transaction.
7. Call `sendTransaction`, passing in the assembled transaction.

That's quite a lot to process! But don't worry, it gets easier the more you do
it. Let's start with the first 3 steps from above:

```typescript
const handleTransactionSubmit = async (movie: Movie) => {
  if (!publicKey) {
    alert("Please connect your wallet!");
    return;
  }

  const buffer = movie.serialize();
  const transaction = new Transaction();
};
```

The next step is to get all of the accounts that the transaction will read or
write. In past lessons, the account where data will be stored has been given to
you. This time, the account's address is more dynamic, so it needs to be
computed. We'll cover this in-depth in the next lesson, but for now, you can use
the following, where `pda` is the address to the account where data will be
stored:

```typescript
const [pda] = await PublicKey.findProgramAddressSync(
  [publicKey.toBuffer(), Buffer.from(movie.title)],
  new PublicKey(MOVIE_REVIEW_PROGRAM_ID),
);
```

In addition to this account, the program will also need to read from
`SystemProgram`, so our array needs to include `SystemProgram.programId` as
well.

With that, we can finish the remaining steps:

```typescript
const handleTransactionSubmit = async (movie: Movie) => {
  if (!publicKey) {
    alert("Please connect your wallet!");
    return;
  }

  const buffer = movie.serialize();
  const transaction = new Transaction();

  const [pda] = await PublicKey.findProgramAddressSync(
    [publicKey.toBuffer(), new TextEncoder().encode(movie.title)],
    new PublicKey(MOVIE_REVIEW_PROGRAM_ID),
  );

  const instruction = new TransactionInstruction({
    keys: [
      {
        pubkey: publicKey,
        isSigner: true,
        isWritable: false,
      },
      {
        pubkey: pda,
        isSigner: false,
        isWritable: true,
      },
      {
        pubkey: SystemProgram.programId,
        isSigner: false,
        isWritable: false,
      },
    ],
    data: buffer,
    programId: new PublicKey(MOVIE_REVIEW_PROGRAM_ID),
  });

  transaction.add(instruction);

  try {
    let transactionId = await sendTransaction(transaction, connection);
    const explorerLink = getExplorerLink(
      "transaction",
      transactionId,
      "devnet",
    );
    console.log(`Transaction submitted: ${explorerLink}`);
  } catch (error) {
    alert(error);
  }
};
```

And that's it! You should now be able to use the form on the site to submit a
movie review. While you won't see the UI update to reflect the new review, you
can look at the transaction's program logs on Solana Explorer to see that it was
successful.

If you need a bit more time with this project to feel comfortable, have a look
at the complete
[solution code](https://github.com/solana-developers/movie-review-frontend/tree/solution-serialize-instruction-data).

## Challenge

Now it's your turn to build something independently. Create an application that
lets students of this course introduce themselves! The Solana program that
supports this is at `HdE95RSVsdb315jfJtaykXhXY478h53X6okDupVfY9yf`.

![Student Intros frontend](/assets/courses/student-intros-frontend.png)

1. You can build this from scratch or you can
   [download the starter code](https://github.com/solana-developers/solana-student-intro-frontend/tree/starter).
2. Create the instruction buffer layout in `StudentIntro.ts`. The program
   expects instruction data to contain:
   1. `variant` as an unsigned, 8-bit integer representing the instruction to
      run (should be 0).
   2. `name` as a string representing the student's name.
   3. `message` as a string representing the message the student is sharing
      about their Solana journey.
3. Create a method in `StudentIntro.ts` that will use the buffer layout to
   serialize a `StudentIntro` object.
4. In the `Form` component, implement the `handleTransactionSubmit` function so
   that it serializes a `StudentIntro`, builds the appropriate transaction and
   transaction instructions, and submits the transaction to the user's wallet.
5. You should now be able to submit introductions and have the information
   stored onchain! Be sure to log the transaction ID and look at it in Solana
   Explorer to verify that it worked.

If you get stumped, you can
[check out the solution code](https://github.com/solana-developers/solana-student-intro-frontend/tree/solution-serialize-instruction-data).

Feel free to get creative with these challenges and take them even further. The
instructions aren't here to hold you back!

<Callout type="info" title="Completed the lab?">

Push your code to GitHub and
[tell us what you thought of this lesson](https://form.typeform.com/to/IPH0UGz7#answers-lesson=6cb40094-3def-4b66-8a72-dd5f00298f61)!

</Callout>
