libp2p-kad-dht
0.15.4

Intro

Installable via npm install --save libp2p-kad-dht, it can also be used directly in the browser.

Download

The source is available for download from GitHub. Alternatively, you can install using npm:

$ npm install --save libp2p-kad-dht

You can then require() libp2p-kad-dht as normal:

const libp2PKadDht = require('libp2p-kad-dht')

In the Browser

Libp2p-kad-dht should work in any ES2015 environment out of the box.

Usage:

<script type="text/javascript" src="index.js"></script>

The portable versions of libp2p-kad-dht, including index.js and index.min.js, are included in the /dist folder. Libp2p-kad-dht can also be found on unpkg.com under

KadDHT

A DHT implementation modeled after Kademlia with S/Kademlia modifications.

Original implementation in go: https://github.com/libp2p/go-libp2p-kad-dht.

Extends EventEmitter.

Parameters

  1. sw: Switch:  
    libp2p-switch instance
  2. options: object:  
    DHT options

instance

KadDHT.prototype.switch

switch: Switch

Local reference to the libp2p-switch instance

KadDHT.prototype.kBucketSize

kBucketSize: number

k-bucket size, defaults to 20

KadDHT.prototype.concurrency

concurrency: number

ALPHA concurrency at which each query path with run, defaults to 3

KadDHT.prototype.disjointPaths

disjointPaths: number

Number of disjoint query paths to use This is set to kBucketSize/2 per the S/Kademlia paper

KadDHT.prototype.routingTable

routingTable: RoutingTable

The routing table.

KadDHT.prototype.datastore

datastore: Datastore

Reference to the datastore, uses an in-memory store if none given.

KadDHT.prototype.providers

providers: Providers

Provider management

KadDHT.prototype.randomWalk

randomWalk: RandomWalk

Random walk management

KadDHT.prototype._queryManager

_queryManager: QueryManager

Keeps track of running queries

KadDHT.prototype.isStarted

isStarted: bool

Is this DHT running.

KadDHT.prototype.start

start(callback: function (Error)): void

Start listening to incoming connections.

Parameters

  1. callback: function (Error):  

Returns

void

KadDHT.prototype.stop

stop(callback: function (Error)): void

Stop accepting incoming connections and sending outgoing messages.

Parameters

  1. callback: function (Error):  

Returns

void

KadDHT.prototype.peerInfo

peerInfo: PeerInfo

Local peer (yourself)

KadDHT.prototype.put

put(key: Buffer, value: Buffer, options: Object, callback: function (Error)): void

Store the given key/value pair in the DHT.

Parameters

  1. key: Buffer:  
  2. value: Buffer:  
  3. options: Object:  
    get options
  4. options.minPeers: number:  
    minimum peers that must be put to to consider this a successful operation (default: closestPeers.length)
  5. callback: function (Error):  

Returns

void

KadDHT.prototype.get

get(key: Buffer, options: Object, callback: function (Error, Buffer)): void

Get the value to the given key. Times out after 1 minute.

Parameters

  1. key: Buffer:  
  2. options: Object:  
    get options
  3. options.timeout: number:  
    optional timeout (default: 60000)
  4. callback: function (Error, Buffer):  

Returns

void

KadDHT.prototype.getMany

getMany(key: Buffer, nvals: number, options: Object, callback: function (Error, Array<{from: PeerId, val: Buffer}>)): void

Get the n values to the given key without sorting.

Parameters

  1. key: Buffer:  
  2. nvals: number:  
  3. options: Object:  
    get options
  4. options.timeout: number:  
    optional timeout (default: 60000)
  5. callback: function (Error, Array<{from: PeerId, val: Buffer}>):  

Returns

void

KadDHT.prototype.getClosestPeers

getClosestPeers(key: Buffer, options: Object, callback: function (Error, Array<PeerId>)): void

Kademlia 'node lookup' operation.

Parameters

  1. key: Buffer:  
  2. options: Object:  
  3. options.shallow: boolean:  
    shallow query
  4. callback: function (Error, Array<PeerId>):  

Returns

void

KadDHT.prototype.getPublicKey

getPublicKey(peer: PeerId, callback: function (Error, PubKey)): void

Get the public key for the given peer id.

Parameters

  1. peer: PeerId:  
  2. callback: function (Error, PubKey):  

Returns

void

KadDHT.prototype.findPeerLocal

findPeerLocal(peer: PeerId, callback: function (Error, PeerInfo)): void

Look if we are connected to a peer with the given id. Returns the PeerInfo for it, if found, otherwise undefined.

Parameters

  1. peer: PeerId:  
  2. callback: function (Error, PeerInfo):  

Returns

void

KadDHT.prototype.provide

provide(key: CID, callback: function (Error)): void

Announce to the network that we can provide given key's value.

Parameters

  1. key: CID:  
  2. callback: function (Error):  

Returns

void

KadDHT.prototype.findProviders

findProviders(key: CID, options: Object, callback: function (Error, Array<PeerInfo>)): void

Search the dht for up to K providers of the given CID.

Parameters

  1. key: CID:  
  2. options: Object:  
    findProviders options
  3. options.timeout: number:  
    how long the query should maximally run, in milliseconds (default: 60000)
  4. options.maxNumProviders: number:  
    maximum number of providers to find
  5. callback: function (Error, Array<PeerInfo>):  

Returns

void

KadDHT.prototype.findPeer

findPeer(id: PeerId, options: Object, callback: function (Error, PeerInfo)): void

Search for a peer with the given ID.

Parameters

  1. id: PeerId:  
  2. options: Object:  
    findPeer options
  3. options.timeout: number:  
    how long the query should maximally run, in milliseconds (default: 60000)
  4. callback: function (Error, PeerInfo):  

Returns

void

RoutingTable

A wrapper around k-bucket, to provide easy store and retrival for peers.

Parameters

  1. self: PeerId:  
  2. kBucketSize: number:  

instance

RoutingTable.prototype.size

size: number

Amount of currently stored peers.

RoutingTable.prototype.find

find(peer: PeerId, callback: function (Error, PeerId)): void

Find a specific peer by id.

Parameters

  1. peer: PeerId:  
  2. callback: function (Error, PeerId):  

Returns

void

RoutingTable.prototype.closestPeer

closestPeer(key: Buffer, count: number): (PeerId | undefined)

Retrieve the closest peers to the given key.

Parameters

  1. key: Buffer:  
  2. count: number:  

Returns

(PeerId | undefined)

RoutingTable.prototype.closestPeers

closestPeers(key: Buffer, count: number): Array<PeerId>

Retrieve the count-closest peers to the given key.

Parameters

  1. key: Buffer:  
  2. count: number:  

Returns

Array<PeerId>

RoutingTable.prototype.add

add(peer: PeerId, callback: function (Error)): undefined

Add or update the routing table with the given peer.

Parameters

  1. peer: PeerId:  
  2. callback: function (Error):  

Returns

RoutingTable.prototype.remove

remove(peer: PeerId, callback: function (Error)): undefined

Remove a given peer from the table.

Parameters

  1. peer: PeerId:  
  2. callback: function (Error):  

Returns

convertBuffer

Creates a DHT ID by hashing a given buffer.

Parameters

  1. buf: Buffer:  
  2. callback: function (Error, Buffer):  

Returns

void

convertPeerId

Creates a DHT ID by hashing a Peer ID

Parameters

  1. peer: PeerId:  
  2. callback: function (Error, Buffer):  

Returns

void

bufferToKey

Convert a buffer to their SHA2-256 hash.

Parameters

  1. buf: Buffer:  

Returns

Key

keyForPublicKey

Generate the key for a public key.

Parameters

  1. peer: PeerId:  

Returns

now

Get the current time as timestamp.

Returns

encodeBase32

Encode a given buffer into a base32 string.

Parameters

  1. buf: Buffer:  

Returns

decodeBase32

Decode a given base32 string into a buffer.

Parameters

  1. raw: string:  

Returns

sortClosestPeers

Sort peers by distance to the given target.

Parameters

  1. peers: Array<PeerId>:  
  2. target: Buffer:  
  3. callback: function (Error, Array<PeerId>):  

Returns

void

xorCompare

Compare function to sort an array of elements which have a distance property which is the xor distance to a given element.

Parameters

  1. a: Object:  
  2. b: Object:  

Returns

pathSize

Computes how many results to collect on each disjoint path, rounding up. This ensures that we look for at least one result per path.

Parameters

  1. resultsWanted: number:  
  2. numPaths: number:  
    total number of paths

Returns

createPutRecord

Create a new put record, encodes and signs it if enabled.

Parameters

  1. key: Buffer:  
  2. value: Buffer:  
  3. callback: function (Error, Buffer):  

Returns

void

Query

Divide peers up into disjoint paths (subqueries). Any peer can only be used once over all paths. Within each path, query peers from closest to farthest away.

Parameters

  1. dht: DHT:  
    DHT instance
  2. key: Buffer:  
  3. makePath: makePath:  
    Called to set up each disjoint path. Must return the query function.

instance

Query.prototype.run

run(peers: Array<PeerId>): Promise

Run this query, start with the given list of peers first.

Parameters

  1. peers: Array<PeerId>:  

Returns

Query.prototype._onStart

_onStart()

Called when the run starts.

Query.prototype._onComplete

_onComplete()

Called when the run completes (even if there's an error).

Query.prototype.stop

stop()

Stop the query.

Run

Manages a single run of the query.

Extends EventEmitter.

Parameters

  1. query: Query:  

instance

Run.prototype.stop

stop()

Stop all the workers

Run.prototype.execute

execute(peers: Array<PeerId>): Promise

Execute the run with the given initial set of peers.

Parameters

  1. peers: Array<PeerId>:  

Returns

Run.prototype.executePaths

executePaths(paths: Array<Path>): Promise<void>

Execute all paths through the DHT.

Parameters

  1. paths: Array<Path>:  

Returns

Promise<void>

Run.prototype.workerQueue

workerQueue(path: Path): Promise<void>

Initialize the list of queried peers, then start a worker queue for the given path.

Parameters

  1. path: Path:  

Returns

Promise<void>

Run.prototype.startWorker

startWorker(path: Path): Promise<void>

Create and start a worker queue for a particular path.

Parameters

  1. path: Path:  

Returns

Promise<void>

Run.prototype.init

init(): Promise<void>

Initialize the list of closest peers we've queried - this is shared by all paths in the run.

Returns

Promise<void>

Run.prototype.continueQuerying

continueQuerying(worker: WorkerQueue): Promise<Boolean>

If we've queried K peers, and the remaining peers in the given worker's queue are all further from the key than the peers we've already queried, then we should stop querying on that worker.

Parameters

  1. worker: WorkerQueue:  

Returns

PeerDistanceList

Maintains a list of peerIds sorted by distance from a DHT key.

Parameters

  1. originDhtKey: Buffer:  
    the DHT key from which distance is calculated
  2. capacity: number:  
    the maximum size of the list

instance

PeerDistanceList.prototype.length

length

The length of the list

PeerDistanceList.prototype.peers

peers

The peerIds in the list, in order of distance from the origin key

PeerDistanceList.prototype.add

add(peerId: PeerId, callback: function (Error)): void

Add a peerId to the list.

Parameters

  1. peerId: PeerId:  
  2. callback: function (Error):  

Returns

void

PeerDistanceList.prototype.anyCloser

anyCloser(peerIds: Array<PeerId>, callback: function (Error, Boolean)): void

Indicates whether any of the peerIds passed as a parameter are closer to the origin key than the furthest peerId in the PeerDistanceList.

Parameters

  1. peerIds: Array<PeerId>:  
  2. callback: function (Error, Boolean):  

Returns

void

Path

Manages a single Path through the DHT.

Parameters

  1. run: Run:  
  2. queryFunc: queryFunc:  

instance

Path.prototype.initialPeers

initialPeers: Array<PeerId>

Path.prototype.peersToQuery

peersToQuery: PeerQueue

Path.prototype.addInitialPeer

addInitialPeer(peer: PeerId)

Add a peer to the set of peers that are used to intialize the path.

Parameters

  1. peer: PeerId:  

Path.prototype.execute

execute(): Promise

Execute the path.

Returns

Path.prototype.addPeerToQuery

addPeerToQuery(peer: PeerId): Promise<void>

Add a peer to the peers to be queried.

Parameters

  1. peer: PeerId:  

Returns

Promise<void>

PeerQueue

PeerQueue is a heap that sorts its entries (PeerIds) by their xor distance to the inital provided key.

Parameters

  1. from: Buffer:  
    The sha2-256 encoded peer id

static

PeerQueue.fromPeerId

fromPeerId(id: PeerId): Promise<PeerQueue>

Create from a given peer id.

Parameters

  1. id: PeerId:  

PeerQueue.fromKey

fromKey(keyBuffer: Buffer): Promise<PeerQueue>

Create from a given buffer.

Parameters

  1. keyBuffer: Buffer:  

instance

PeerQueue.prototype.enqueue

enqueue(id: PeerId): Promise

Add a new PeerId to the queue.

Parameters

  1. id: PeerId:  

Returns

PeerQueue.prototype.dequeue

dequeue(): PeerId

Returns the closest peer to the from peer.

Returns

PeerId

constructor

Creates a new WorkerQueue.

Parameters

  1. dht: DHT:  
  2. run: Run:  
  3. path: Object:  
  4. log: function:  

setupQueue

Create the underlying async queue.

Returns

stop

Stop the worker, optionally providing an error to pass to the worker's callback.

Parameters

  1. err: Error:  

execute

Use the queue from async to keep concurrency amount items running per path.

Returns

Promise<void>

fill

Add peers to the worker queue until there are enough to satisfy the worker queue concurrency. Note that we don't want to take any more than those required to satisfy concurrency from the peers-to-query queue, because we always want to query the closest peers to the key first, and new peers are continously being added to the peers-to-query queue.

processNext

Process the next peer in the queue

Parameters

  1. peer: PeerId:  

Returns

Promise<void>

Network

Handle network operations for the dht

Parameters

  1. self: KadDHT:  

instance

Network.prototype.start

start(callback: function (Error)): void

Start the network.

Parameters

  1. callback: function (Error):  

Returns

void

Network.prototype.stop

stop(callback: function (Error)): void

Stop all network activity.

Parameters

  1. callback: function (Error):  

Returns

void

Network.prototype.isStarted

isStarted: bool

Is the network online?

Network.prototype.isConnected

isConnected: bool

Are all network components there?

Network.prototype.sendRequest

sendRequest(to: PeerId, msg: Message, callback: function (Error, Message)): void

Send a request and record RTT for latency measurements.

Parameters

  1. to: PeerId:  
    The peer that should receive a message
  2. msg: Message:  
    The message to send.
  3. callback: function (Error, Message):  

Returns

void

Network.prototype.sendMessage

sendMessage(to: PeerId, msg: Message, callback: function (Error)): void

Sends a message without expecting an answer.

Parameters

  1. to: PeerId:  
  2. msg: Message:  
  3. callback: function (Error):  

Returns

void

protocolHandler

Handle incoming streams from the Switch, on the dht protocol.

Parameters

  1. protocol: string:  
  2. conn: Connection:  

Returns

getValue

Process GetValue DHT messages.

Parameters

  1. peer: PeerInfo:  
  2. msg: Message:  
  3. callback: function (Error, Message):  

Returns

putValue

Process PutValue DHT messages.

Parameters

  1. peer: PeerInfo:  
  2. msg: Message:  
  3. callback: function (Error, Message):  

Returns

findNode

Process FindNode DHT messages.

Parameters

  1. peer: PeerInfo:  
  2. msg: Message:  
  3. callback: function (Error, Message):  

Returns

addProvider

Process AddProvider DHT messages.

Parameters

  1. peer: PeerInfo:  
  2. msg: Message:  
  3. callback: function (Error):  

Returns

getProviders

Process GetProviders DHT messages.

Parameters

  1. peer: PeerInfo:  
  2. msg: Message:  
  3. callback: function (Error, Message):  

Returns

ping

Process Ping DHT messages.

Parameters

  1. peer: PeerInfo:  
  2. msg: Message:  
  3. callback: function (Error, Message):  

Returns

LimitedPeerList

Like PeerList but with a length restriction.

Extends PeerList.

Parameters

  1. limit: number:  

instance

LimitedPeerList.prototype.push

push(info: PeerInfo): bool

Add a PeerInfo if it fits in the list

Parameters

  1. info: PeerInfo:  

Returns

bool

PeerList

A list of unique peer infos.

instance

PeerList.prototype.push

push(info: PeerInfo): bool

Add a new info. Returns true if it was a new one

Parameters

  1. info: PeerInfo:  

Returns

bool

PeerList.prototype.has

has(info: PeerInfo): bool

Check if this PeerInfo is already in here.

Parameters

  1. info: PeerInfo:  

Returns

bool

PeerList.prototype.toArray

toArray(): Array<PeerInfo>

Get the list as an array.

Returns

Array<PeerInfo>

PeerList.prototype.pop

pop(): PeerInfo

Remove the last element

Returns

PeerInfo

PeerList.prototype.length

length: number

The length of the list

Providers

This class manages known providers. A provider is a peer that we know to have the content for a given CID.

Every cleanupInterval providers are checked if they are still valid, i.e. younger than the provideValidity. If they are not, they are deleted.

To ensure the list survives restarts of the daemon, providers are stored in the datastore, but to ensure access is fast there is an LRU cache in front of that.

Parameters

  1. datastore: Object:  
  2. self: PeerId?:  
  3. cacheSize: number (=256):  

instance

Providers.prototype.cleanupInterval

cleanupInterval: number

How often invalid records are cleaned. (in seconds)

Providers.prototype.provideValidity

provideValidity: number

How long is a provider valid for. (in seconds)

Providers.prototype.lruCacheSize

lruCacheSize: number

LRU cache size

Providers.prototype.stop

stop(): undefined

Release any resources.

Returns

Providers.prototype.addProvider

addProvider(cid: CID, provider: PeerId): Promise

Add a new provider for the given CID.

Parameters

  1. cid: CID:  
  2. provider: PeerId:  

Returns

Providers.prototype.getProviders

getProviders(cid: CID): Promise<Array<PeerId>>

Get a list of providers for the given CID.

Parameters

  1. cid: CID:  

Returns

Promise<Array<PeerId>>

Message

Represents a single DHT control message.

Parameters

  1. type: MessageType:  
  2. key: Buffer:  
  3. level: number:  

static

Message.deserialize

deserialize(raw: Buffer): Message

Decode from protobuf

Parameters

  1. raw: Buffer:  

Returns

instance

Message.prototype.clusterLevel

clusterLevel: number

Message.prototype.serialize

serialize(): Buffer

Encode into protobuf

Returns

constructor

Parameters

  1. dht: DHT:  
  2. options: object:  
  3. options.enabled: randomWalkOptions.enabled:  
  4. options.queriesPerPeriod: randomWalkOptions.queriesPerPeriod:  
  5. options.interval: randomWalkOptions.interval:  
  6. options.timeout: randomWalkOptions.timeout:  
  7. options.delay: randomWalkOptions.delay:  
  8. options.dht: DHT:  

start

Start the Random Walk process. This means running a number of queries every interval requesting random data. This is done to keep the dht healthy over time.

Returns

void

stop

Stop the random-walk process. Any active queries will be aborted.

Returns

void

QueryManager

Keeps track of all running queries.

instance

QueryManager.prototype.queryStarted

queryStarted(query: Query)

Called when a query is started.

Parameters

  1. query: Query:  

QueryManager.prototype.queryCompleted

queryCompleted(query: Query)

Called when a query completes.

Parameters

  1. query: Query:  

QueryManager.prototype.start

start()

Starts the query manager.

QueryManager.prototype.stop

stop()

Stops all queries.