libp2p-kad-dht

0.19.6

Intro

Installable via npm install --save libp2p-kad-dht, it can also be used directly in the browser.

Download

The source is available for download from GitHub. Alternatively, you can install using npm:

$ npm install --save libp2p-kad-dht

You can then require() libp2p-kad-dht as normal:

const libp2PKadDht = require('libp2p-kad-dht')

In the Browser

libp2p-kad-dht should work in any ECMAScript 2018 environment out of the box.

Usage:

<script type="text/javascript" src="index.js"></script>

The portable versions of libp2p-kad-dht, including index.js and index.min.js, are included in the /dist folder. libp2p-kad-dht can also be found on unpkg.com under

A DHT implementation modeled after Kademlia with S/Kademlia modifications. Original implementation in go: https://github.com/libp2p/go-libp2p-kad-dht.

new KadDHT($0: Object, props: Object)

Extends EventEmitter

Parameters
$0 (Object)
Name Description
$0.dialer any
$0.peerId any
$0.peerStore any
$0.registrar any
$0.datastore any (default new MemoryDatastore())
$0.kBucketSize any (default c.K)
$0.concurrency any (default c.ALPHA)
$0.validators any (default {})
$0.selectors any (default {})
$0.randomWalk any (default {})
props (Object)
Instance Members
dialer
peerId
peerStore
registrar
kBucketSize
concurrency
disjointPaths
routingTable
datastore
providers
randomWalk
_queryManager
isStarted
start()
stop()
put(key, value, options = {})
get(key, options = {})
getMany(key, nvals, options = {})
provide(key)
findProviders(key, options)
findPeer(id, options)
getClosestPeers(key, options = {shallow:false})
getPublicKey(peer)

RoutingTable

src/routing.js

A wrapper around k-bucket, to provide easy store and retrival for peers.

new RoutingTable(self: PeerId, kBucketSize: number)
Parameters
self (PeerId)
kBucketSize (number)
Instance Members
size
find(peer)
closestPeer(key)
closestPeers(key, count)
add(peer)
remove(peer)

convertBuffer

src/utils.js

Creates a DHT ID by hashing a given buffer.

convertBuffer(buf: Buffer): Promise<Buffer>
Parameters
buf (Buffer)
Returns
Promise<Buffer>:

convertPeerId

src/utils.js

Creates a DHT ID by hashing a Peer ID

convertPeerId(peer: PeerId): Promise<Buffer>
Parameters
peer (PeerId)
Returns
Promise<Buffer>:

bufferToKey

src/utils.js

Convert a buffer to their SHA2-256 hash.

bufferToKey(buf: Buffer): Key
Parameters
buf (Buffer)
Returns
Key:

keyForPublicKey

src/utils.js

Generate the key for a public key.

keyForPublicKey(peer: PeerId): Buffer
Parameters
peer (PeerId)
Returns
Buffer:

Get the current time as timestamp.

now(): number
Returns
number:

encodeBase32

src/utils.js

Encode a given buffer into a base32 string.

encodeBase32(buf: Buffer): string
Parameters
buf (Buffer)
Returns
string:

decodeBase32

src/utils.js

Decode a given base32 string into a buffer.

decodeBase32(raw: string): Buffer
Parameters
raw (string)
Returns
Buffer:

sortClosestPeers

src/utils.js

Sort peers by distance to the given target.

sortClosestPeers(peers: Array<PeerId>, target: Buffer): Array<PeerId>
Parameters
peers (Array<PeerId>)
target (Buffer)
Returns
Array<PeerId>:

xorCompare

src/utils.js

Compare function to sort an array of elements which have a distance property which is the xor distance to a given element.

xorCompare(a: Object, b: Object): number
Parameters
a (Object)
b (Object)
Returns
number:

Computes how many results to collect on each disjoint path, rounding up. This ensures that we look for at least one result per path.

pathSize(resultsWanted: number, numPaths: number): number
Parameters
resultsWanted (number)
numPaths (number) total number of paths
Returns
number:

createPutRecord

src/utils.js

Create a new put record, encodes and signs it if enabled.

createPutRecord(key: Buffer, value: Buffer): Buffer
Parameters
key (Buffer)
value (Buffer)
Returns
Buffer:

Handle network operations for the dht

new Network(self: KadDHT)
Parameters
self (KadDHT)
Instance Members
start()
stop()
isStarted
isConnected
sendRequest(to, msg)
sendMessage(to, msg)

onIncomingStream

src/rpc/index.js

Handle incoming streams on the dht protocol.

onIncomingStream
Parameters
props (Object)
Name Description
props.stream DuplexStream
props.connection Connection connection
Returns
Promise<void>:

Process GetValue DHT messages.

getValue
Parameters
peerId (PeerId)
msg (Message)
Returns
Promise<Message>:

Process PutValue DHT messages.

putValue
Parameters
peerId (PeerId)
msg (Message)
Returns
Promise<Message>:

Process FindNode DHT messages.

findNode
Parameters
peerId (PeerId)
msg (Message)
Returns
Promise<Message>:

Process AddProvider DHT messages.

addProvider
Parameters
peerId (PeerId)
msg (Message)
Returns
Promise<void>:

Process GetProviders DHT messages.

getProviders
Parameters
peerId (PeerId)
msg (Message)
Returns
Promise<Message>:

Process Ping DHT messages.

ping
Parameters
peerId (PeerId)
msg (Message)
Returns
Message:

Store the given key/value pair in the DHT.

put(key: Buffer, value: Buffer, options: Object?): Promise<void>
Parameters
key (Buffer)
value (Buffer)
options (Object? = {}) put options
Name Description
options.minPeers number? minimum number of peers required to successfully put (default: closestPeers.length)
Returns
Promise<void>:

Get the value to the given key. Times out after 1 minute by default.

get(key: Buffer, options: Object?): Promise<Buffer>
Parameters
key (Buffer)
options (Object? = {}) get options
Name Description
options.timeout number? optional timeout (default: 60000)
Returns
Promise<Buffer>:

Get the n values to the given key without sorting.

getMany(key: Buffer, nvals: number, options: Object?): Promise<Array<{from: PeerId, val: Buffer}>>
Parameters
key (Buffer)
nvals (number)
options (Object? = {}) get options
Name Description
options.timeout number? optional timeout (default: 60000)
Returns
Promise<Array<{from: PeerId, val: Buffer}>>:

Divide peers up into disjoint paths (subqueries). Any peer can only be used once over all paths. Within each path, query peers from closest to farthest away.

new Query(dht: DHT, key: Buffer, makePath: makePath)
Parameters
dht (DHT) DHT instance
key (Buffer)
makePath (makePath) Called to set up each disjoint path. Must return the query function.
Instance Members
run(peers)
_onStart()
_onComplete()
stop()

Manages a single run of the query.

new Run(query: Query)

Extends EventEmitter

Parameters
query (Query)
Instance Members
stop()
execute(peers)
executePaths(paths)
workerQueue(path)
startWorker(path)
init()
continueQuerying(worker)

Maintains a list of peerIds sorted by distance from a DHT key.

new PeerDistanceList(originDhtKey: Buffer, capacity: number)
Parameters
originDhtKey (Buffer) the DHT key from which distance is calculated
capacity (number) the maximum size of the list
Instance Members
length
peers
add(peerId)
anyCloser(peerIds)

Manages a single Path through the DHT.

new Path(run: Run, queryFunc: queryFunc)
Parameters
run (Run)
queryFunc (queryFunc)
Instance Members
initialPeers
peersToQuery
addInitialPeer(peer)
execute()
addPeerToQuery(peer)

PeerQueue is a heap that sorts its entries (PeerIds) by their xor distance to the inital provided key.

new PeerQueue(from: Buffer)
Parameters
from (Buffer) The sha2-256 encoded peer id
Static Members
fromPeerId(id)
fromKey(keyBuffer)
Instance Members
enqueue(id)
dequeue()

Creates a new WorkerQueue.

constructor(dht: DHT, run: Run, path: Object, log: function)
Parameters
dht (DHT)
run (Run)
path (Object)
log (function)

Create the underlying async queue.

setupQueue(): Object
Returns
Object:

Stop the worker, optionally providing an error to pass to the worker's callback.

stop(err: Error)
Parameters
err (Error)

Use the queue from async to keep concurrency amount items running per path.

execute(): Promise<void>
Returns
Promise<void>:

Add peers to the worker queue until there are enough to satisfy the worker queue concurrency. Note that we don't want to take any more than those required to satisfy concurrency from the peers-to-query queue, because we always want to query the closest peers to the key first, and new peers are continously being added to the peers-to-query queue.

fill()

Process the next peer in the queue

processNext(peer: PeerId): Promise<void>
Parameters
peer (PeerId)
Returns
Promise<void>:

Announce to the network that we can provide given key's value.

provide(key: CID): Promise<void>
Parameters
key (CID)
Returns
Promise<void>:

Search the dht for up to K providers of the given CID.

findProviders(key: CID, options: Object): AsyncIterable<{id: PeerId, multiaddrs: Array<Multiaddr>}>
Parameters
key (CID)
options (Object = {}) findProviders options
Name Description
options.timeout number how long the query should maximally run, in milliseconds (default: 60000)
options.maxNumProviders number maximum number of providers to find
Returns
AsyncIterable<{id: PeerId, multiaddrs: Array<Multiaddr>}>:

Like PeerList but with a length restriction.

new LimitedPeerList(limit: number)

Extends PeerList

Parameters
limit (number)
Instance Members
push(peerData)

A list of unique peers.

new PeerList()
Instance Members
push(peerData)
has(peerId)
toArray()
pop()
length

Look if we are connected to a peer with the given id. Returns its id and addresses, if found, otherwise undefined.

findPeerLocal
Parameters
peer (PeerId)
Returns
Promise<{id: PeerId, multiaddrs: Array<Multiaddr>}>:

Search for a peer with the given ID.

findPeer(id: PeerId, options: Object): Promise<{id: PeerId, multiaddrs: Array<Multiaddr>}>
Parameters
id (PeerId)
options (Object = {}) findPeer options
Name Description
options.timeout number how long the query should maximally run, in milliseconds (default: 60000)
Returns
Promise<{id: PeerId, multiaddrs: Array<Multiaddr>}>:

Kademlia 'node lookup' operation.

getClosestPeers(key: Buffer, options: Object?): AsyncIterable<PeerId>
Parameters
key (Buffer)
options (Object? = {shallow:false})
Name Description
options.shallow boolean? shallow query (default: false)
Returns
AsyncIterable<PeerId>:

Get the public key for the given peer id.

getPublicKey(peer: PeerId): Promise<PubKey>
Parameters
peer (PeerId)
Returns
Promise<PubKey>:

Represents a single DHT control message.

new Message(type: MessageType, key: Buffer, level: number)
Parameters
type (MessageType)
key (Buffer)
level (number)
Static Members
deserialize(raw)
Instance Members
clusterLevel
serialize()

This class manages known providers. A provider is a peer that we know to have the content for a given CID.

Every cleanupInterval providers are checked if they are still valid, i.e. younger than the provideValidity. If they are not, they are deleted.

To ensure the list survives restarts of the daemon, providers are stored in the datastore, but to ensure access is fast there is an LRU cache in front of that.

new Providers(datastore: Object, self: PeerId?, cacheSize: number)
Parameters
datastore (Object)
self (PeerId?)
cacheSize (number = 256)
Instance Members
cleanupInterval
provideValidity
lruCacheSize
start()
stop()
addProvider(cid, provider)
getProviders(cid)
new constructor(dht: DHT, options: object)
Parameters
dht (DHT)
options (object)
Name Description
options.enabled randomWalkOptions.enabled
options.queriesPerPeriod randomWalkOptions.queriesPerPeriod
options.interval randomWalkOptions.interval
options.timeout randomWalkOptions.timeout
options.delay randomWalkOptions.delay
options.dht DHT

Start the Random Walk process. This means running a number of queries every interval requesting random data. This is done to keep the dht healthy over time.

start(): void
Returns
void:

Stop the random-walk process. Any active queries will be aborted.

stop(): void
Returns
void:

Keeps track of all running queries.

new QueryManager()
Instance Members
queryStarted(query)
queryCompleted(query)
start()
stop()