{"_id":"@bobanetwork/turing-hybrid-compute","name":"@bobanetwork/turing-hybrid-compute","dist-tags":{"0.2.0":"0.2.0","latest":"0.2.0"},"versions":{"0.2.0":{"name":"@bobanetwork/turing-hybrid-compute","version":"0.2.0","description":"Hybrid Compute for Ethereum prototype","main":"index.js","repository":{"type":"git","url":"git@github.com:bobanetwork/boba/packages/boba/turing"},"author":{"name":"Michael Montour","email":"mmontour@enya.ai"},"license":"Internal use only","private":false,"scripts":{"clean":"rm -rf ./artifacts ./cache","build":"yarn build:contracts","build:contracts":"hardhat compile","test:local":"hardhat --network boba_local test","test:mainnet":"hardhat --network boba_mainnet test"},"devDependencies":{"@ethersproject/address":"^5.5.0","@ethersproject/contracts":"^5.5.0","@ethersproject/networks":"^5.5.0","@ethersproject/providers":"^5.5.0","@ethersproject/solidity":"^5.5.0","@nomiclabs/hardhat-ethers":"^2.0.2","@openzeppelin/contracts":"4.3.2","@types/mocha":"^8.2.2","chai":"^4.3.6","ethers":"^5.5.4","hardhat":"^2.12.5","mocha":"^8.3.1","ts-node":"10.9.1","typescript":"^4.3.5"},"dependencies":{"@uniswap/sdk":"^3.0.3","ip":"^1.1.5","web3":"^1.6.1","web3-eth-abi":"^1.6.1"},"_id":"@bobanetwork/turing-hybrid-compute@0.2.0","dist":{"shasum":"b84600a2faa13e9dbcfca50178f1a834893bbb3d","integrity":"sha512-KQ4RnCfgqqF+cF4E207VulJNH+GqlRgv+0qxLda8jOcOlCSbikY7t5I1iasG7s8eOeDvElShizC25gScBng3Sw==","tarball":"https://registry.npmjs.org/@bobanetwork/turing-hybrid-compute/-/turing-hybrid-compute-0.2.0.tgz","fileCount":31,"unpackedSize":417069,"signatures":[{"keyid":"SHA256:jl3bwswu80PjjokCgh0o2w5c2U4LhQAE57gj9cz1kzA","sig":"MEUCIQDFfh9aW0YgCxdk1kWA/2Qp/FsMxxnX8+7hNkRrkBZViAIgXMevC9EbUkHkcaLWDJk73a5zgMkXWPNFVWUfx4DIibM="}]},"_npmUser":{"name":"boba.network","email":"chaininfra@boba.foundation"},"directories":{},"maintainers":[{"name":"boba.network","email":"chaininfra@boba.foundation"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages","tmp":"tmp/turing-hybrid-compute_0.2.0_1695734050249_0.7560822229787432"},"_hasShrinkwrap":false}},"time":{"created":"2023-09-26T13:14:10.174Z","0.2.0":"2023-09-26T13:14:10.472Z","modified":"2023-09-26T13:14:10.803Z"},"maintainers":[{"name":"boba.network","email":"chaininfra@boba.foundation"}],"description":"Hybrid Compute for Ethereum prototype","repository":{"type":"git","url":"git@github.com:bobanetwork/boba/packages/boba/turing"},"author":{"name":"Michael Montour","email":"mmontour@enya.ai"},"license":"Internal use only","readme":"---\ndescription: Learn how to use Turing hybrid compute\n---\n\nTuring is a system for interacting with the outside world from within solidity smart contracts. All data returned from external APIs, such as social networking or weather data are deposited into a public data-storage contract on Ethereum Mainnet. This extra data allows replicas, verifiers, and fraud-detectors to reproduce and validate the Boba L2 blockchain, block by block.\n\nUsing Turing is as easy as calling specific functions from inside your smart contract. For example, to obtain a random number for minting NFTs, call:\n\n```javascript\n\n  // ERC721.sol\n  random_number = turing.getRandom()\n\n  // Test/Debug Response\n  Turing NFT Random 256\n    256 bit random number as a BigInt = 61245594159531997717158776666900035572992757857563713350570408643552830626492n\n    Minted an NFT with Attribute A = 135 and Attribute B = 103\n    Minted a pirate with a green hat\n    ✓ should mint an NFT with random attributes (65ms)\n\n```\n\nTo obtain **Twitter** or **Spotify** data you could set up a system like this:\n\n```javascript\n\n  urlStr = 'https://_myAPIURL_/social'\n  likes = social.getCurrentLikes(tweetUniqueID)\n\n  // Test/Debug response\n  Tweet 123456789 had: 18 likes by time: 1650534735\n\n```\n\n## Feature Highlight 1: Using Turing to mint an NFT with 256 random attributes in a single transaction\n\nWith Turing, your ERC721 contract can generate a cryptographically strong 256 bit random number immediately prior to the execution flow moving to the `mint` function. This is an _atomic_ transaction - everything takes places within one transaction:\n\n```javascript\n\n  // modified mint function in a standard ERC721.sol\n  function mint(address to, uint256 tokenId) public {\n    uint256 result = myHelper.TuringRandom();\n    bytes memory i_bytes = abi.encodePacked(result);\n    uint8 attribute_1  = uint8(i_bytes[ 0]);\n    uint8 attribute_2  = uint8(i_bytes[ 1]);\n    ...\n    uint8 attribute_32 = uint8(i_bytes[31]);\n    // use the attributes here to e.g. set URI/Attributes etc\n    _mint(to, tokenId);\n    emit MintedRandom(result, attribute_1, attribute_2, ...);\n  }\n\n  // pseudocode transaction response from test system (see boba/turing/test/006_NFT_random.ts)\n  256 bit random number as a BigInt = 61245594159531997717158776666900035572992757857563713350570408643552830626492n\n  Minted an NFT with Attribute A = 135 and Attribute B = 103\n  Minted a pirate with a green hat\n  ✓ should mint an NFT with random attributes (65ms)\n\n```\n\nTo use this functionality, deploy your `TuringHelper` contract, provide its address to your ERC721 contract, and make the `TuringHelper` aware of the new caller:\n\n```javascript\n\n  // deploy your Turing helper\n  myTuringHelper = await Factory__Helper.deploy()\n\n  // deploy your ERC721 contract with the\n  erc721 = await Factory__ERC721.deploy(\"RandomERC721\", \"RER\", myTuringHelper.address)\n\n  // restrict your myHelper to accept only requests from your ERC721\n  await myTuringHelper.addPermittedCaller(erc721.address)\n\n```\n\nThen, register and fund your Turing Credit account:\n\n```javascript\n\n  const ONE_BOBA = utils.parseEther('1')\n  await turingCredit.addBalanceTo(ONE_BOBA, myTuringHelper.address)\n\n```\n\n**All done**! Each Turing request costs 0.01 BOBA, so 1 BOBA is enough for 100 Turing requests. Have fun. You can find [example code and an ERC721 that uses Turing here](./test/006_NFT_random.ts) and a [fully-featured Turing-ready NFT system here](../../boba_community/turing-monsters/README.md).\n\n## Feature Highlight 2: Using Turing to access APIs from within your solidity smart contract\n\nYou can use Turing as a pipe to any other computer, such as APIs for social networks, weather and location data, or market data. Please keep in mind however that Turing differs sharply from established providers of market trading data, in particular, since **Turing does not provide a decentralized mechanism to verify the accuracy of the data**. **You should therefore not use Turing for production trading or lending use, but should use proven, decentralized data oracles**.\n\n**Data/Oracle best practices** Regardless of your specific use case, minimally, you will need to secure your pipe/contract against data outliers, temporary lack of data, and malicious attempts to distort the data. For example, you could average over multiple on-chain oracles and/or off-chain sources - in this case, the role of Turing could be to 'augment' or separately estimate the reliability and timeliness of on-chain oracles.\n\n**Note - Boba does not provide endpoints for you** You are responsible for setting up an endpoint that Turing can access - read on for more information and example code. Assume you have an API access key to a provider of weather data. First, set up a server or endpoint that queries this API, and stores and analyzes the data, if needed. Your own server/endpoint contains your secrets and API access keys. Next, add a simple interface to allow Turing to interact with your server. Turing calls to your server  contain the address of the calling contract and there are multiple ways to control access to your server in very granular manner, if desired. See `.packages/boba/turing/AWS_code/turing_oracle.py` for a copy-paste example for querying data APIs via a wrapper at AWS Lambda:\n\n```python\n/AWS_code/turing_oracle.py\n\n# Note - This code is running on YOUR server\n\n...\n  api_key = 'YOUR_API_KEY' # Insert your API key here\n\n  authorized_contract = None # for open access\n  # or...\n  authorized_contract = '0xOF_YOUR_HELPER_CONTRACT' # to restrict access to only your smart contract\n...\n\n```\n\nYou should lock down your off-chain endpoint to only accept queries from your smart contract. To do this, designate your smart contract's address on Boba as the `authorized_contract`. If you wish to allow open access, set this variable to `None`. You can then call this API in your smart contract:\n\n```javascript\n\n  urlStr = 'https://_myAPIURL_/social'\n  likes = social.getCurrentLikes(tweetUniqueID)\n\n    // Test/Debug response\n    Tweet 123456789 had: 18 likes by time: 1650534735\n\n```\n\n## AWS and Google Cloud Function Examples\n\nYour external API will need to accept calls from the L2Geth and return data in a way that can be understood by the L2Geth. Examples are provided in `./packages/boba/turing/AWS_code`. Specific instructions for setting up AWS lambda endpoints are [here](./AWS_code/AWS_lambda_setup.md) - note that _all_ APIs can be used, not just AWS Lambda endpoints.\n\n## Important Properties of Turing\n\n* Strings returned from external endpoints are limited to 322 characters (`5*64+2=322`)\n* Only one Turing call per execution\n* There is **1200 ms timeout** on API responses. Please make sure that your API responds promptly. If you are using AWS, note that some of their services take several seconds to spin up from a 'coldstart', resulting in persistent failure of your first call to your endpoint.\n\n### String length limit\n\nThe string length cap of 322 is large enough to return, for example, four `uint256` from the external api:\n\n```javascript\n//example: returing 4 unit264\n\n  // 0x\n  // 0000000000000000000000000000000000000000000000000000000000000080 ** length of the dynamic bytes\n  // 0000000000000000000000000000000000000000000000000000000000418b95 ** first uint256\n  // 0000000000000000000000000000000000000000000000000000017e60d3b45f **\n  // 0000000000000000000000000000000000000000000000000000000000eb7ca3 **\n  // 00000000000000000000000000000000000000000000000000000000004c788f ** fourth unit265\n\n```\n\nYou can return anything you want - e.g. numbers, strings, ... - and this information will then later be decoded per your `abi.decode`. For example, if the external API sends two `unit256`:\n\n```javascript\n\n  // Payload from the external API\n  // 0x\n  // 0000000000000000000000000000000000000000000000000000000000000040 ** length of the dynamic bytes\n  // 0000000000000000000000000000000000000000000000000000000000418b95 ** first uint256\n  // 0000000000000000000000000000000000000000000000000000017e60d3b45f ** second uint256\n\n  // decoding of those data within the smart contract\n  (uint256 market_price, uint256 time) = abi.decode(encResponse,(uint256,uint256));\n\n```\n\n### One Turing call per Transaction\n\nAt present, you can only have one Turing call per transaction, i.e. a Turing call cannot call other contracts that invoke Turing as well. Transactions that result in multiple Turing calls in the call stack will revert.\n\n## Turing Architecture\n\nThe modified Turing L2Geth, `L2TGeth`, monitors calldata for particular Keccak methodIDs of functions such as `GetRandom(uint32 rType, uint256 _random)` and `GetResponse(uint32 rType, string memory _url, bytes memory _payload)`. Upon finding such methodIDs in the execution flow, at any level, L2TGeth parses the calldata for additional information, such as external URLs, and uses that information to either directly prepare a response (e.g. generate a random number) or to call an external API. After new information is generated (or has returned from the external API), L2TGeth then runs the function with updated inputs, such that the new information flows back to the caller (via overloaded variables and a system for conditionally bypassing `requires`). Put simply, L2TGeth intercepts function calls, adds new information to the inputs, and then runs the function with the updated inputs.\n\nIn general, this system would lead to disagreement about the correct state of the underlying blockchain. For example, if replicas and verifiers simply ingested the transactions and re-executed them, then every blockchain would differ, destroying the entire system. Thus, a new data field called `Turing` (aka `turing`, `l1Turing` or `L1Turing` depending on context) has been added to the L2Geth `transactions`,`messages`, `receipts`, `blocks`, `evm.contexts`, and various `codecs` and `encoders/decoders`. This new data field is understood by `core-utils` as well as the `data-translation-layer` and the `batch-submitter`, and allows Turing data to be pushed into, and recovered from, the `CanonicalTransactionChain` (CTC). This extra information allows all verifiers and replicas to enter a new **replay** mode, where instead of generating new random numbers (or calling off-chain for new data), they use the Turing data stored in the CTC (or in the L2 blocks as part of the transaction metadata) to generate a faithful copy of the main Boba L2 blockchain. Thus, the overall system works as before, with all the information needed for restoring the Boba L2 and, just as critically, for public fraud detection, being publicly deposited into Ethereum.\n\n## Quickstart for Turing Developers\n\nOpen a terminal window and from the top level:\n\n```bash\n$ yarn\n$ yarn build\n$ cd ops\n$ BUILD=1 DAEMON=0 ./up_local.sh\n```\n\nThis will spin up the stack. Then, open a second terminal window and:\n\n```bash\n$ cd packages/boba/turing\n$ yarn test:local\n```\n\n**Note: Testing on Goerli**\n\nTo test on Goerli, you need a private key with both ETH and BOBA on the Boba L2; the private key needs to be provided in `hardhat.config.js`. Just replace all the zeros with your key:\n\n```javascript\n    boba_goerli: {\n      url: 'https://goerli.boba.network',\n      accounts: ['0x0000000000000000000000000000000000000000000000000000000000000000']\n    },\n```\n\nThen, run:\n\n```bash\n$ cd packages/boba/turing\n$ yarn test:goerli\n```\n\nThe tests will perform some basic floating point math, provide some random numbers, and get the latest BTC-USD exchange rate:\n\n```bash\nyarn run v1.22.15\n$ hardhat --network boba_local test\n\n  Stableswap at AWS Lambda\n    URL set to https://i9iznmo33e.execute-api.us-east-1.amazonaws.com/swapy\n    Helper contract deployed as 0x8e264821AFa98DD104eEcfcfa7FD9f8D8B320adA\n    Stableswap contract deployed as 0x871ACbEabBaf8Bed65c22ba7132beCFaBf8c27B5\n    addingPermittedCaller to TuringHelper 0x000000000000000000000000871acbeabbaf8bed65c22ba7132becfabf8c27b5\n    Test contract whitelisted in TuringHelper (1 = yes)? 1\n    ✓ contract should be whitelisted (50ms)\n    Credit Prebalance 0\n    BOBA Balance in your account 300000000000000000000\n    ✓ Should register and fund your Turing helper contract in turingCredit (172ms)\n    ✓ should return the helper address (116ms)\n      result of x_in 12 -> y_out = 50\n    ✓ should correctly swap X in for Y out (202ms)\n\n  Turing 256 Bit Random Number\n    Helper contract deployed at 0xb185E9f6531BA9877741022C92CE858cDCc5760E\n    Test contract deployed at 0xAe120F0df055428E45b264E7794A18c54a2a3fAF\n    addingPermittedCaller to TuringHelper 0x000000000000000000000000ae120f0df055428e45b264e7794a18c54a2a3faf\n    Test contract whitelisted in TuringHelper (1 = yes)? 1\n    ✓ contract should be whitelisted (51ms)\n    Credit Prebalance 0\n    BOBA Balance in your account 290000000000000000000\n    ✓ Should register and fund your Turing helper contract in turingCredit (174ms)\n    Turing 42 = 42\n    ✓ should get the number 42 (91ms)\n    Turing VRF 256 = 11642062518220346831211086370276871135010213271872466428492348202384902597141n\n    ✓ should get a 256 bit random number (83ms)\n    Turing VRF 256 = 39492154036951735205025381980653780356965271743173916331971607322325246415525n\n    ✓ should get a 256 bit random number (83ms)\n\n  Pull Bitcoin - USD quote\n    URL set to https://i9iznmo33e.execute-api.us-east-1.amazonaws.com/quote\n    Helper contract deployed as 0x7C8BaafA542c57fF9B2B90612bf8aB9E86e22C09\n    Lending contract deployed as 0x0a17FabeA4633ce714F1Fa4a2dcA62C3bAc4758d\n    addingPermittedCaller to TuringHelper 0x0000000000000000000000000a17fabea4633ce714f1fa4a2dca62c3bac4758d\n    Test contract whitelisted in TuringHelper (1 = yes)? 1\n    ✓ contract should be whitelisted (53ms)\n    ✓ should return the helper address\n    Credit Prebalance 0\n    BOBA Balance in your account 280000000000000000000\n    ✓ Should register and fund your Turing helper contract in turingCredit (176ms)\n    Bitcoin to USD price is 36654.89\n    timestamp 1643158948154\n    ✓ should get the current Bitcoin - USD price (305ms)\n\n  Turing NFT Random 256\n    Turing Helper contract deployed at 0xd9fEc8238711935D6c8d79Bef2B9546ef23FC046\n    ERC721 contract deployed at 0xd3FFD73C53F139cEBB80b6A524bE280955b3f4db\n    adding your ERC721 as PermittedCaller to TuringHelper 0x000000000000000000000000d3ffd73c53f139cebb80b6a524be280955b3f4db\n    Credit Prebalance 0\n    BOBA Balance in your account 270000000000000000000\n    ✓ Should register and fund your Turing helper contract in turingCredit (122ms)\n    ERC721 contract whitelisted in TuringHelper (1 = yes)? 1\n    ✓ Your ERC721 contract should be whitelisted\n    256 bit random number as a BigInt = 61245594159531997717158776666900035572992757857563713350570408643552830626492n\n    Minted an NFT with Attribute A = 135 and Attribute B = 103\n    Minted a pirate with a green hat\n    ✓ should mint an NFT with random attributes (65ms)\n\n\n  22 passing (3s)\n\n✨  Done in 6.67s.\n```\n\n## Technical Appendix: Implementation Details\n\n### Step 1: Invoking Turing for inside a Smart contract\n\nA Turing cycle starts with specific function calls inside solidity smart contracts deployed on Boba:\n\n```javascript\n\n  random_number = turing.getRandom()\n\n```\n\nThe modified `L2TGeth` detects these function calls, intercepts them, and obtains requested data from other sources (strong random number generators, off-chain APIs and datafeeds, ...).\n\n```go\n/l2geth/core/vm/evm.go\n\n// Call executes the contract associated with the addr with the given input as\n// parameters. It also handles any necessary value transfer required and takes\n// the necessary steps to create accounts and reverses the state in case of an\n// execution error or failed value transfer.\nfunc (evm *EVM) Call(caller ContractRef, addr common.Address, input []byte, gas uint64, value *big.Int) (ret []byte, leftOverGas uint64, err error) {\n\n...\n\n  //methodID for GetResponse is 7d93616c -> [125 147 97 108]\n  isTuring2 := bytes.Equal(input[:4], []byte{125, 147, 97, 108})\n\n  //methodID for GetRandom is 493d57d6 -> [73 61 87 214]\n  isGetRand2 := bytes.Equal(input[:4], []byte{73, 61, 87, 214})\n\n  // TuringCall takes the original calldata, figures out what needs\n  // to be done, and then synthesizes a 'updated_input' calldata\n  var updated_input hexutil.Bytes\n\n  if isTuring2 {\n    if len(evm.Context.Turing) < 3 {\n      // This is the first run of Turing for this transaction\n      // We sometimes use a short evm.Context.Turing payload for debug purposes.\n      // A real modified callData is always much much > 2 bytes\n      // This case _should_ never happen in Verifier/Replica mode, since the sequencer will already have run the Turing call\n      updated_input = bobaTuringCall(input, caller.Address())\n      ret, err = run(evm, contract, updated_input, false)\n      // and now, provide the updated_input to the context so that the data can be sent to L1 and the CTC\n      /**************** CRITICAL LINE ****************/\n      evm.Context.Turing = updated_input\n      /**************** CRITICAL LINE ****************/\n    } else {\n      // Turing for this Transaction has already been run elsewhere - replay using\n      // information from the EVM context\n      ret, err = run(evm, contract, evm.Context.Turing, false)\n    }\n  } else if isGetRand2 {\n    if len(evm.Context.Turing) < 3 {\n      // See above - they apply 1:1 here too\n      updated_input = bobaTuringRandom(input)\n      ret, err = run(evm, contract, updated_input, false)\n\n      /**************** CRITICAL LINE ****************/\n      evm.Context.Turing = updated_input\n      /**************** CRITICAL LINE ****************/\n    } else {\n      // Turing for this Transaction has already been run elsewhere - replay using\n      // information from the EVM context\n      ret, err = run(evm, contract, evm.Context.Turing, false)\n    }\n  } else {\n    ret, err = run(evm, contract, input, false)\n  }\n\n...\n\n```\n\nThe random number generation is done locally, inside the Geth, and off-chain APIs are queried with standard calls:\n\n```go\n/l2geth/core/vm/evm.go\n\n// In response to an off-chain Turing request, obtain the requested data and\n// rewrite the parameters so that the contract can be called without reverting.\nfunc bobaTuringRandom(input []byte) hexutil.Bytes {\n\n  var ret hexutil.Bytes\n\n  rest := input[4:]\n\n  //some things are easier with a hex string\n  inputHexUtil := hexutil.Bytes(input)\n\n  // If things fail, we'll return an integer parameter which will fail a\n  // \"require\" in the contract.\n  retError := make([]byte, len(inputHexUtil))\n  copy(retError, inputHexUtil)\n\n  // Check the rType\n  // 1 for Request, 2 for Response, integer >= 10 for various failures\n  rType := int(rest[31])\n  if rType != 1 {\n    log.Warn(\"TURING-1 bobaTuringRandom:Wrong state (rType != 1)\", \"rType\", rType)\n    retError[35] = 10 // Wrong input state\n    return retError\n  }\n\n  rlen := len(rest)\n  if rlen < 2*32 {\n    log.Warn(\"TURING-2 bobaTuringRandom:Calldata too short\", \"len < 2*32\", rlen)\n    retError[35] = 11 // Calldata too short\n    return retError\n  }\n\n  // Generate cryptographically strong pseudo-random int between 0 - 2^256 - 1\n  one := big.NewInt(1)\n  two := big.NewInt(2)\n  max := new(big.Int)\n  // Max random value 2^256 - 1\n  max = max.Exp(two, big.NewInt(int64(256)), nil).Sub(max, one)\n  n, err := rand.Int(rand.Reader, max)\n\n  if err != nil {\n    log.Warn(\"TURING bobaTuringRandom: Random Number Generation Failed\", \"err\", err)\n    retError[35] = 16 // RNG Failure\n    return retError\n  }\n\n  //generate a BigInt random number\n  randomBigInt := n\n\n  // build the calldata\n  methodID := make([]byte, 4)\n  copy(methodID, inputHexUtil[0:4])\n  ret = append(methodID, hexutil.MustDecode(fmt.Sprintf(\"0x%064x\", 2))...) // the usual prefix and the rType, now changed to 2\n  ret = append(ret, hexutil.MustDecode(fmt.Sprintf(\"0x%064x\", randomBigInt))...)\n\n  return ret\n}\n\n// In response to an off-chain Turing request, obtain the requested data and\n// rewrite the parameters so that the contract can be called without reverting.\nfunc bobaTuringCall(input []byte, caller common.Address) hexutil.Bytes {\n\n  var responseStringEnc string\n  var responseString []byte\n\n  rest := input[4:]\n  inputHexUtil := hexutil.Bytes(input)\n  restHexUtil := inputHexUtil[4:]\n\n  retError := make([]byte, len(inputHexUtil))\n  copy(retError, inputHexUtil)\n\n  // Check the rType\n  // 1 for Request, 2 for Response, integer >= 10 for various failures\n  rType := int(rest[31])\n  if rType != 1 {\n    retError[35] = 10 // Wrong input state\n    return retError\n  }\n\n  rlen := len(rest)\n  if rlen < 7*32 {\n    retError[35] = 11 // Calldata too short\n    return retError\n  }\n\n  // A micro-ABI decoder... this works because we know that all these numbers can never exceed 256\n  // Since the rType is 32 bytes and the three headers are 32 bytes each, the max possible value\n  // of any of these numbers is 32 + 32 + 32 + 32 + 64 = 192\n  // Thus, we only need to read one byte\n\n  // 0  -  31 = rType\n  // 32  -  63 = URL start\n  // 64  -  95 = payload start\n  // 96  - 127 = length URL string\n  // 128 - ??? = URL string\n  // ??? - ??? = payload length\n  // ??? - end = payload\n\n  startIDXurl := int(rest[63]) + 32\n  // the +32 means that we are going directly for the actual string\n  // bytes 0 to 31 are the string length\n\n  startIDXpayload := int(rest[95]) // the start of the payload\n  lengthURL := int(rest[127])      // the length of the URL string\n\n  // Check the URL length\n  // Note: we do not handle URLs that are longer than 64 characters\n  if lengthURL > 64 {\n    retError[35] = 12 // URL string > 64 bytes\n    return retError\n  }\n\n  // The URL we are going to query\n  endIDX := startIDXurl + lengthURL\n  url := string(rest[startIDXurl:endIDX])\n  // we use a specific end value (startIDXurl+lengthURL) since the URL is right-packed with zeros\n\n  // At this point, we have the API endpoint and the payload that needs to go there...\n  payload := restHexUtil[startIDXpayload:] //using hex here since that makes it easy to get the string\n\n  log.Debug(\"TURING-4 bobaTuringCall:Have URL and payload\",\n    \"url\", url,\n    \"payload\", payload)\n\n  client, err := rpc.Dial(url)\n\n  if client != nil {\n    if err := client.Call(&responseStringEnc, caller.String(), payload); err != nil {\n      retError[35] = 13 // Client Error\n      return retError\n    }\n    responseString, err = hexutil.Decode(responseStringEnc)\n    if err != nil {\n      retError[35] = 14 // Client Response Decode Error\n      return retError\n    }\n  } else {\n    retError[35] = 15 // Could not create client\n    return retError\n  }\n\n  // build the modified calldata\n  ret := make([]byte, startIDXpayload+4)\n  copy(ret, inputHexUtil[0:startIDXpayload+4]) // take the original input\n  ret[35] = 2                                  // change byte 3 + 32 = 35 (rType) to indicate a valid response\n  ret = append(ret, responseString...)         // and tack on the payload\n\n  return ret\n}\n\n```\n\n### Step 2: Flow of Turing data out of the evm.context\n\n`l2geth/core/state_processor.go:core.ApplyTransaction` moves the Turing data from `Context.Turing` into the `transaction.meta.L1Turing` byte array:\n\n```go\nl2geth/core/state_processor.go\n\n// ApplyTransaction attempts to apply a transaction to the given state database\n// and uses the input parameters for its environment. It returns the receipt\n// for the transaction, gas used and an error if the transaction failed,\n// indicating the block was invalid.\nfunc ApplyTransaction(config *params.ChainConfig, bc ChainContext, author *common.Address, gp *GasPool, statedb *state.StateDB, header *types.Header, tx *types.Transaction, usedGas *uint64, cfg vm.Config) (*types.Receipt, error) {\n...\n  109   // Apply the transaction to the current state (included in the env)\n  110   _, gas, failed, err := ApplyMessage(vmenv, msg, gp)\n  111:  // TURING Update the tx metadata, if a Turing call took place...\n  112   if len(vmenv.Context.Turing) > 1 {\n  113     tx.SetL1Turing(vmenv.Context.Turing)\n  114   }\n\n```\n\nThe Turing data are subsequently incorporated into new L2 blocks via `w.engine.FinalizeAndAssemble` - the Turing data are in the `w.current.txs` input.\n\n```go\nl2geth/miner/worker.go:\n\n// commit runs any post-transaction state modifications, assembles the final block\n// and commits new work if consensus engine is running.\nfunc (w *worker) commit(uncles []*types.Header, interval func(), start time.Time) error {\n...\n 1110   s := w.current.state.Copy()\n 1111   // log.Debug(\"TURING worker.go final block\", \"depositing_txs\", w.current.txs)\n 1112:  block, err := w.engine.FinalizeAndAssemble(w.chain, w.current.header, s, w.current.txs, uncles, w.current.receipts)\n 1113   if err != nil {\n 1114     return err\n\n```\n\nAt this point, the data are circulated to various places throughout the system as part of the block/transaction data. Notably, calls to the L2 for block/transaction data now return a new field, `l1Turing` to all callers.\n\n### Step 3: Batch submitter Turing data injection\n\nThe batch submitter receives an new block/transaction from `L2TGeth`, obtains the raw call string (`rawTransaction`) and the Turing data (`l1Turing`), and if there was a Turing event (as judged from the length of the Turing string), the modified `batch-submitter` appends those data to the `rawTransaction` string. From the perspective of the CTC, it is receiving its normal batch payload.\n\n```javascript\n// batch-submitter tx-batch-submitter.ts\n\nprivate async _getL2BatchElement(blockNumber: number): Promise<BatchElement> {\n\n  // Idea - manipulate the rawTransaction as early as possible, so we do not have to change even more of the encode/decode\n  // logic - note that this is basically adding a second encoder/decoder before the 'normal' one, which encodes total length\n  //\n  // The 'normal' one will now specify the TOTAL length (new_turing_header + rawTransaction + turing (if != 0)) rather than\n  // just remove0x(rawTransaction).length / 2\n\n...\n\n  if (this._isSequencerTx(block)) {\n    batchElement.isSequencerTx = true\n    const turing = block.transactions[0].l1Turing\n    let rawTransaction = block.transactions[0].rawTransaction\n    if (turing.length > 4) {\n      // FYI - we sometimes use short (length <= 4) non-zero Turing strings for debug purposes\n      // Chop those off at this stage\n      // Only propagate the data through the system if it's a real Turing payload\n      const headerTuringLengthField = remove0x(BigNumber.from(remove0x(turing).length / 2).toHexString()).padStart(6, '0')\n      rawTransaction = '0x' + headerTuringLengthField + remove0x(rawTransaction) + remove0x(turing)\n    } else {\n      rawTransaction = '0x' + '000000' + remove0x(rawTransaction)\n    }\n    batchElement.rawTransaction = rawTransaction\n  }\n\n```\n\n### Step 4: Writing to the CTC\n\nThe batch-submitter writes the data to the CTC as usual. **The CTC does not know about Turing** - that was one of the goals, so we do not have to modify the L1 contracts.\n\n### Step 5: DTL Turing data extraction; Reading from the CTC\n\nThe DTL reads from the CTC and unpacks the modified `rawTransaction` (which is now called `sequencerTransaction`). The DTL uses a Turing length metadata field in the `sequencerTransaction` string. Critically, the DTL writes a slightly modified `TransactionEntry` into its database, which has a new field called `turing`. When the database is queried, it thus returns the Turing data in addition to all the usual fields.\n\n```javascript\n// DTL services/l1-ingestion/handles/sequencer-batch-appended.ts\n\n        for (let j = 0; j < context.numSequencedTransactions; j++) {\n...\n\n        // need to keep track of the original length so the pointer system for accessing\n        // the individual transactions works correctly\n        const sequencerTransaction_original_length = sequencerTransaction.length\n\n        // This MIGHT have a Turing payload inside of it...\n        // First, parse the new length field...\n        const sTxHexString = toHexString(sequencerTransaction)\n        const turingLength = parseInt(remove0x(sTxHexString).slice(0,6), 16)\n\n        let turing = Buffer.from('0')\n\n        if (turingLength > 0) {\n          //we have Turing payload\n          turing = sequencerTransaction.slice(-turingLength)\n          sequencerTransaction = sequencerTransaction.slice(3, -turingLength)\n          // The `3` chops off the Turing length header field, and the `-turingLength` chops off the Turing bytes\n          console.log('Found a Turing payload at (neg) position:', {\n            turingLength,\n            turing: toHexString(turing),\n            restoredSequencerTransaction: toHexString(sequencerTransaction),\n          })\n        } else {\n          // The `3` chops off the Turing length header field, which is zero in this case (0: 00 1: 00 2: 00)\n          sequencerTransaction = sequencerTransaction.slice(3)\n        }\n\n        transactionEntries.push({\n...\n          turing: toHexString(turing),\n        })\n\n```\n\n### Step 6: Verifier data ingestion\n\nThe Verifier receives all the usual data from the DTL, but, if there was a Turing call, there is now an additional data field containing the rewritten callData as a HexString. The Turing data are obtained from incoming `json` data and are written into the transaction metadata, `meta.L1Turing = turing`:\n\n```go\n/l2geth/core/types/transaction_meta.go:\n   38   L1Timestamp     uint64          `json:\"l1Timestamp\"`\n   39:  L1Turing        []byte          `json:\"l1Turing\" gencodec:\"required\"`\n   40   L1MessageSender *common.Address `json:\"l1MessageSender\" gencodec:\"required\"`\n   ..\n   55   l1Timestamp uint64,\n   56:  l1Turing []byte,\n   57   l1MessageSender *common.Address,\n   ..\n   64       L1Timestamp:     l1Timestamp,\n   65:      L1Turing:        l1Turing,\n   66       L1MessageSender: l1MessageSender,\n   ..\n  145   }\n  146\n  147:  turing, err := common.ReadVarBytes(b, 0, 2048, \"Turing\")\n  148   if err != nil {\n  149       return nil, err\n  150   }\n  151:  if !isNullValue(turing) {\n  152:      meta.L1Turing = turing\n  153   }\n```\n\nAt this point, the Turing data can be passed into the `evm.context`, which then triggers the `else` logic in the `evm.go`:\n\n```go\n/l2geth/core/vm/evm.go\n\n// Call executes the contract associated with the addr with the given input as\n// parameters. It also handles any necessary value transfer required and takes\n// the necessary steps to create accounts and reverses the state in case of an\n// execution error or failed value transfer.\nfunc (evm *EVM) Call(caller ContractRef, addr common.Address, input []byte, gas uint64, value *big.Int) (ret []byte, leftOverGas uint64, err error) {\n\n...\n\n  //methodID for GetResponse is 7d93616c -> [125 147 97 108]\n  isTuring2 := bytes.Equal(input[:4], []byte{125, 147, 97, 108})\n\n  //methodID for GetRandom is 493d57d6 -> [73 61 87 214]\n  isGetRand2 := bytes.Equal(input[:4], []byte{73, 61, 87, 214})\n\n  // TuringCall takes the original calldata, figures out what needs\n  // to be done, and then synthesizes a 'updated_input' calldata\n  var updated_input hexutil.Bytes\n\n  if isTuring2 {\n    if len(evm.Context.Turing) < 3 {\n...\n    } else {\n      // Turing for this Transaction has already been run elsewhere - replay using\n      // information from the EVM context\n      ret, err = run(evm, contract, evm.Context.Turing, false)\n    }\n  } else if isGetRand2 {\n    if len(evm.Context.Turing) < 3 {\n...\n    } else {\n      // Turing for this Transaction has already been run elsewhere - replay using\n      // information from the EVM context\n      ret, err = run(evm, contract, evm.Context.Turing, false)\n    }\n  } else {\n    ret, err = run(evm, contract, input, false)\n  }\n\n...\n\n```\n\nThe Turing data flow from out from the `evm.context` through the rest of the system as before, so the data are incorporated into verifier and replica blocks, resulting in correct/consistent state roots and replica and verifier blocks.\n","readmeFilename":"README.md"}