libp2p-kad-dht

0.16.1

Intro

Installable via npm install --save libp2p-kad-dht, it can also be used directly in the browser.

Download

The source is available for download from GitHub. Alternatively, you can install using npm:

$ npm install --save libp2p-kad-dht

You can then require() libp2p-kad-dht as normal:

const libp2PKadDht = require('libp2p-kad-dht')

In the Browser

Libp2p-kad-dht should work in any ES2015 environment out of the box.

Usage:

<script type="text/javascript" src="index.js"></script>

The portable versions of libp2p-kad-dht, including index.js and index.min.js, are included in the /dist folder. Libp2p-kad-dht can also be found on unpkg.com under

A DHT implementation modeled after Kademlia with S/Kademlia modifications.

Original implementation in go: https://github.com/libp2p/go-libp2p-kad-dht.

new KadDHT(sw: Switch, options: object)

Extends EventEmitter

Parameters
sw (Switch) libp2p-switch instance
options (object) DHT options
Instance Members
switch
kBucketSize
concurrency
disjointPaths
routingTable
datastore
providers
randomWalk
_queryManager
isStarted
start(callback)
stop(callback)
peerInfo
put(key, value, options, callback)
get(key, options, callback)
getMany(key, nvals, options, callback)
getClosestPeers(key, options, callback)
getPublicKey(peer, callback)
findPeerLocal(peer, callback)
provide(key, callback)
findProviders(key, options, callback)
findPeer(id, options, callback)

RoutingTable

src/routing.js

A wrapper around k-bucket, to provide easy store and retrival for peers.

new RoutingTable(self: PeerId, kBucketSize: number)
Parameters
self (PeerId)
kBucketSize (number)
Instance Members
size
find(peer, callback)
closestPeer(key, count)
closestPeers(key, count)
add(peer, callback)
remove(peer, callback)

convertBuffer

src/utils.js

Creates a DHT ID by hashing a given buffer.

convertBuffer(buf: Buffer, callback: function (Error, Buffer)): void
Parameters
buf (Buffer)
callback (function (Error, Buffer))
Returns
void:

convertPeerId

src/utils.js

Creates a DHT ID by hashing a Peer ID

convertPeerId(peer: PeerId, callback: function (Error, Buffer)): void
Parameters
peer (PeerId)
callback (function (Error, Buffer))
Returns
void:

bufferToKey

src/utils.js

Convert a buffer to their SHA2-256 hash.

bufferToKey(buf: Buffer): Key
Parameters
buf (Buffer)
Returns
Key:

keyForPublicKey

src/utils.js

Generate the key for a public key.

keyForPublicKey(peer: PeerId): Buffer
Parameters
peer (PeerId)
Returns
Buffer:

Get the current time as timestamp.

now(): number
Returns
number:

encodeBase32

src/utils.js

Encode a given buffer into a base32 string.

encodeBase32(buf: Buffer): string
Parameters
buf (Buffer)
Returns
string:

decodeBase32

src/utils.js

Decode a given base32 string into a buffer.

decodeBase32(raw: string): Buffer
Parameters
raw (string)
Returns
Buffer:

sortClosestPeers

src/utils.js

Sort peers by distance to the given target.

sortClosestPeers(peers: Array<PeerId>, target: Buffer, callback: function (Error, Array<PeerId>)): void
Parameters
peers (Array<PeerId>)
target (Buffer)
callback (function (Error, Array<PeerId>))
Returns
void:

xorCompare

src/utils.js

Compare function to sort an array of elements which have a distance property which is the xor distance to a given element.

xorCompare(a: Object, b: Object): number
Parameters
a (Object)
b (Object)
Returns
number:

Computes how many results to collect on each disjoint path, rounding up. This ensures that we look for at least one result per path.

pathSize(resultsWanted: number, numPaths: number): number
Parameters
resultsWanted (number)
numPaths (number) total number of paths
Returns
number:

createPutRecord

src/utils.js

Create a new put record, encodes and signs it if enabled.

createPutRecord(key: Buffer, value: Buffer, callback: function (Error, Buffer)): void
Parameters
key (Buffer)
value (Buffer)
callback (function (Error, Buffer))
Returns
void:

Divide peers up into disjoint paths (subqueries). Any peer can only be used once over all paths. Within each path, query peers from closest to farthest away.

new Query(dht: DHT, key: Buffer, makePath: makePath)
Parameters
dht (DHT) DHT instance
key (Buffer)
makePath (makePath) Called to set up each disjoint path. Must return the query function.
Instance Members
run(peers)
_onStart()
_onComplete()
stop()

Manages a single run of the query.

new Run(query: Query)

Extends EventEmitter

Parameters
query (Query)
Instance Members
stop()
execute(peers)
executePaths(paths)
workerQueue(path)
startWorker(path)
init()
continueQuerying(worker)

Maintains a list of peerIds sorted by distance from a DHT key.

new PeerDistanceList(originDhtKey: Buffer, capacity: number)
Parameters
originDhtKey (Buffer) the DHT key from which distance is calculated
capacity (number) the maximum size of the list
Instance Members
length
peers
add(peerId, callback)
anyCloser(peerIds, callback)

Manages a single Path through the DHT.

new Path(run: Run, queryFunc: queryFunc)
Parameters
run (Run)
queryFunc (queryFunc)
Instance Members
initialPeers
peersToQuery
addInitialPeer(peer)
execute()
addPeerToQuery(peer)

PeerQueue is a heap that sorts its entries (PeerIds) by their xor distance to the inital provided key.

new PeerQueue(from: Buffer)
Parameters
from (Buffer) The sha2-256 encoded peer id
Static Members
fromPeerId(id)
fromKey(keyBuffer)
Instance Members
enqueue(id)
dequeue()

Creates a new WorkerQueue.

constructor(dht: DHT, run: Run, path: Object, log: function)
Parameters
dht (DHT)
run (Run)
path (Object)
log (function)

Create the underlying async queue.

setupQueue(): Object
Returns
Object:

Stop the worker, optionally providing an error to pass to the worker's callback.

stop(err: Error)
Parameters
err (Error)

Use the queue from async to keep concurrency amount items running per path.

execute(): Promise<void>
Returns
Promise<void>:

Add peers to the worker queue until there are enough to satisfy the worker queue concurrency. Note that we don't want to take any more than those required to satisfy concurrency from the peers-to-query queue, because we always want to query the closest peers to the key first, and new peers are continously being added to the peers-to-query queue.

fill()

Process the next peer in the queue

processNext(peer: PeerId): Promise<void>
Parameters
peer (PeerId)
Returns
Promise<void>:

Handle network operations for the dht

new Network(self: KadDHT)
Parameters
self (KadDHT)
Instance Members
start(callback)
stop(callback)
isStarted
isConnected
sendRequest(to, msg, callback)
sendMessage(to, msg, callback)

protocolHandler

src/rpc/index.js

Handle incoming streams from the Switch, on the dht protocol.

protocolHandler
Parameters
protocol (string)
conn (Connection)
Returns
undefined:

Process GetValue DHT messages.

getValue
Parameters
peer (PeerInfo)
msg (Message)
callback (function (Error, Message))
Returns
undefined:

Process PutValue DHT messages.

putValue
Parameters
peer (PeerInfo)
msg (Message)
callback (function (Error, Message))
Returns
undefined:

Process FindNode DHT messages.

findNode
Parameters
peer (PeerInfo)
msg (Message)
callback (function (Error, Message))
Returns
undefined:

Process AddProvider DHT messages.

addProvider
Parameters
peer (PeerInfo)
msg (Message)
callback (function (Error))
Returns
undefined:

Process GetProviders DHT messages.

getProvidersAsync(peer: PeerInfo, msg: Message): Promise<Message>
Parameters
peer (PeerInfo)
msg (Message)
Returns
Promise<Message>: Resolves a Message response

Process GetProviders DHT messages.

getProviders
Parameters
peer (PeerInfo)
msg (Message)
callback (function (Error, Message))
Returns
undefined:

Process Ping DHT messages.

ping
Parameters
peer (PeerInfo)
msg (Message)
callback (function (Error, Message))
Returns
undefined:

Like PeerList but with a length restriction.

new LimitedPeerList(limit: number)

Extends PeerList

Parameters
limit (number)
Instance Members
push(info)

A list of unique peer infos.

new PeerList()
Instance Members
push(info)
has(info)
toArray()
pop()
length

This class manages known providers. A provider is a peer that we know to have the content for a given CID.

Every cleanupInterval providers are checked if they are still valid, i.e. younger than the provideValidity. If they are not, they are deleted.

To ensure the list survives restarts of the daemon, providers are stored in the datastore, but to ensure access is fast there is an LRU cache in front of that.

new Providers(datastore: Object, self: PeerId?, cacheSize: number)
Parameters
datastore (Object)
self (PeerId?)
cacheSize (number = 256)
Instance Members
cleanupInterval
provideValidity
lruCacheSize
stop()
addProvider(cid, provider)
getProviders(cid)

Represents a single DHT control message.

new Message(type: MessageType, key: Buffer, level: number)
Parameters
type (MessageType)
key (Buffer)
level (number)
Static Members
deserialize(raw)
Instance Members
clusterLevel
serialize()
new constructor(dht: DHT, options: object)
Parameters
dht (DHT)
options (object)
Name Description
options.enabled randomWalkOptions.enabled
options.queriesPerPeriod randomWalkOptions.queriesPerPeriod
options.interval randomWalkOptions.interval
options.timeout randomWalkOptions.timeout
options.delay randomWalkOptions.delay
options.dht DHT

Start the Random Walk process. This means running a number of queries every interval requesting random data. This is done to keep the dht healthy over time.

start(): void
Returns
void:

Stop the random-walk process. Any active queries will be aborted.

stop(): void
Returns
void:

Keeps track of all running queries.

new QueryManager()
Instance Members
queryStarted(query)
queryCompleted(query)
start()
stop()