1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 | 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 61 61 59 59 59 2 1 1 1 1 2 61 61 61 61 61 61 61 61 61 1 1 63 63 63 63 63 1 88 170 170 138 14 138 138 14 138 103 38 65 138 170 88 85 1 2151 1 20 1 11 1 2182 2182 1969 2182 2182 2182 2182 1958 2182 1650 2182 2157 2182 2182 2182 2192 2189 2189 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 2188 11 11 11 11 11 11 11 11 11 11 11 11 11 2177 2188 2163 2163 2188 2188 2188 446 1742 2188 2188 2188 2188 2188 428 2188 11 11 11 11 11 11 11 11 2177 20 20 20 20 20 2 20 20 20 2157 2182 2192 2192 2188 2188 2188 4 1 2189 2189 2189 1920 2189 2189 2183 2189 8 2189 2189 2189 1773 416 2189 1690 1690 1690 499 2189 2189 2189 2188 2188 2188 2188 2188 2188 2188 1 2188 2188 2188 2188 2188 84 2104 2104 1741 363 363 363 363 363 228 2736 228 228 363 363 2188 2188 1 32 32 32 32 32 2178 2188 2188 2188 2188 2188 32 32 1 2188 1 1 323 323 323 323 323 323 2446 483 483 483 483 483 160 323 323 2446 1 63 63 9 54 21 33 61 1 2530 2530 2530 2530 9 2530 2530 1 2573 55 2518 2395 2573 2573 1 61 2192 2192 2188 2188 1 165 165 165 2014 39 39 165 1 30 30 30 30 30 30 30 30 30 30 30 30 30 30 30 1 12 12 12 12 12 12 12 12 12 12 12 12 12 1 367 367 367 367 367 507 507 507 507 6 6 367 366 366 366 363 367 367 367 1 61 61 1 61 61 61 1 61 61 61 61 61 61 1 2182 2182 2171 2171 11 11 1 2192 3 2189 1877 1877 1877 1877 1 2192 2177 7 8 1 4380 4380 1 2228 2228 2228 1 | // // Copyright (c) Microsoft and contributors. All rights reserved. // // Licensed under the Apache License, Version 2.0 (the "License"); // you may not use this file except in compliance with the License. // You may obtain a copy of the License at // http://www.apache.org/licenses/LICENSE-2.0 // // Unless required by applicable law or agreed to in writing, software // distributed under the License is distributed on an "AS IS" BASIS, // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. // // See the License for the specific language governing permissions and // limitations under the License. // // Module dependencies. var request = require('request'); var url = require('url'); var util = require('util'); var xml2js = require('xml2js'); var events = require('events'); var _ = require('underscore'); var guid = require('node-uuid'); var os = require('os'); var crypto = require('crypto'); var extend = require('extend'); var azureutil = require('../util/util'); var validate = require('../util/validate'); var SR = require('../util/sr'); var WebResource = require('../http/webresource'); var ServiceSettings = require('./servicesettings'); var StorageServiceSettings = require('./storageservicesettings'); var Constants = require('../util/constants'); var StorageUtilities = require('../util/storageutilities'); var ServicePropertiesResult = require('../models/servicepropertiesresult'); var SharedKey = require('../signing/sharedkey'); var SharedAccessSignature = require('../signing/sharedaccesssignature'); var HeaderConstants = Constants.HeaderConstants; var QueryStringConstants = Constants.QueryStringConstants; var HttpResponseCodes = Constants.HttpConstants.HttpResponseCodes; var StorageServiceClientConstants = Constants.StorageServiceClientConstants; var defaultRequestLocationMode = Constants.RequestLocationMode.PRIMARY_ONLY; var RequestLocationMode = Constants.RequestLocationMode; var Logger = require('../diagnostics/logger'); /** * Creates a new StorageServiceClient object. * * @ignore * @constructor * @param {string} storageAccount The storage account. * @param {string} storageAccessKey The storage access key. * @param {object} host The host for the service. * @param {bool} usePathStyleUri Boolean value indicating wether to use path style uris. * @param {string} sasToken The Shared Access Signature token. */ function StorageServiceClient(storageAccount, storageAccessKey, host, usePathStyleUri, sasToken) { StorageServiceClient['super_'].call(this); if(storageAccount && storageAccessKey) { // account and key this.storageAccount = storageAccount; this.storageAccessKey = storageAccessKey; this.storageCredentials = new SharedKey(this.storageAccount, this.storageAccessKey, usePathStyleUri); } else if (sasToken) { // sas this.sasToken = sasToken; this.storageCredentials = new SharedAccessSignature(sasToken); } else { // anonymous this.anonymous = true; this.storageCredentials = { signRequest: function(webResource, callback){ // no op, anonymous access callback(null); } }; } Eif(host){ this.setHost(host); } this.apiVersion = HeaderConstants.TARGET_STORAGE_VERSION; this.usePathStyleUri = usePathStyleUri; this._initDefaultFilter(); this.logger = new Logger(Logger.LogLevels.INFO); this._setDefaultProxy(); this.xml2jsSettings = StorageServiceClient._getDefaultXml2jsSettings(); this.defaultLocationMode = StorageUtilities.LocationMode.PRIMARY_ONLY; } util.inherits(StorageServiceClient, events.EventEmitter); /** * Gets the default xml2js settings. * @ignore * @return {object} The default settings */ StorageServiceClient._getDefaultXml2jsSettings = function() { var xml2jsSettings = _.clone(xml2js.defaults['0.2']); // these determine what happens if the xml contains attributes xml2jsSettings.attrkey = Constants.TableConstants.XML_METADATA_MARKER; xml2jsSettings.charkey = Constants.TableConstants.XML_VALUE_MARKER; // from xml2js guide: always put child nodes in an array if true; otherwise an array is created only if there is more than one. xml2jsSettings.explicitArray = false; return xml2jsSettings; }; /** * Sets a host for the service. * @ignore * @param {string} host The host for the service. */ StorageServiceClient.prototype.setHost = function (host) { var parseHost = function(hostUri){ var parsedHost; if(!azureutil.objectIsNull(hostUri)) { if(hostUri.indexOf('http') === -1 && hostUri.indexOf('//') !== 0){ hostUri = '//' + hostUri; } parsedHost = url.parse(hostUri, false, true); if(!parsedHost.protocol){ parsedHost.protocol = ServiceSettings.DEFAULT_PROTOCOL; } if (!parsedHost.port) { if (parsedHost.protocol === Constants.HTTPS) { parsedHost.port = Constants.DEFAULT_HTTPS_PORT; } else { parsedHost.port = Constants.DEFAULT_HTTP_PORT; } } parsedHost = url.format({ protocol: parsedHost.protocol, port: parsedHost.port, hostname: parsedHost.hostname, pathname: parsedHost.pathname }); } return parsedHost; }; validate.isValidHost(host); this.host = { primaryHost: parseHost(host.primaryHost), secondaryHost: parseHost(host.secondaryHost) }; }; /** * Performs a REST service request through HTTP expecting an input stream. * @ignore * * @param {WebResource} webResource The webresource on which to perform the request. * @param {string} outputData The outgoing request data as a raw string. * @param {object} [options] The request options. * @param {int} [options.timeoutIntervalInMs] The timeout interval, in milliseconds, to use for the request. * @param {function} callback The response callback function. */ StorageServiceClient.prototype.performRequest = function (webResource, outputData, options, callback) { this._performRequest(webResource, { outputData: outputData }, options, callback); }; /** * Performs a REST service request through HTTP expecting an input stream. * @ignore * * @param {WebResource} webResource The webresource on which to perform the request. * @param {Stream} outputStream The outgoing request data as a stream. * @param {object} [options] The request options. * @param {int} [options.timeoutIntervalInMs] The timeout interval, in milliseconds, to use for the request. * @param {function} callback The response callback function. */ StorageServiceClient.prototype.performRequestOutputStream = function (webResource, outputStream, options, callback) { this._performRequest(webResource, { outputStream: outputStream }, options, callback); }; /** * Performs a REST service request through HTTP expecting an input stream. * @ignore * * @param {WebResource} webResource The webresource on which to perform the request. * @param {string} outputData The outgoing request data as a raw string. * @param {Stream} inputStream The ingoing response data as a stream. * @param {object} [options] The request options. * @param {int} [options.timeoutIntervalInMs] The timeout interval, in milliseconds, to use for the request. * @param {function} callback The response callback function. */ StorageServiceClient.prototype.performRequestInputStream = function (webResource, outputData, inputStream, options, callback) { this._performRequest(webResource, { outputData: outputData, inputStream: inputStream }, options, callback); }; /** * Performs a REST service request through HTTP. * @ignore * * @param {WebResource} webResource The webresource on which to perform the request. * @param {object} body The request body. * @param {string} [body.outputData] The outgoing request data as a raw string. * @param {Stream} [body.outputStream] The outgoing request data as a stream. * @param {Stream} [body.inputStream] The ingoing response data as a stream. * @param {object} [options] The request options. * @param {string} [options.clientRequestId] A string that represents the client request ID with a 1KB character limit. * @param {int} [options.timeoutIntervalInMs] The timeout interval, in milliseconds, to use for the request. * @param {function} callback The response callback function. */ StorageServiceClient.prototype._performRequest = function (webResource, body, options, callback) { var self = this; // Sets a requestId on the webResource if(!options.clientRequestId) { options.clientRequestId = guid.v1(); } webResource.withHeader(HeaderConstants.CLIENT_REQUEST_ID, options.clientRequestId); // Sets the user-agent string var userAgentComment = util.format('(NODE-VERSION %s; %s %s)', process.version, os.type(), os.release()); webResource.withHeader(HeaderConstants.USER_AGENT, Constants.USER_AGENT_PRODUCT_NAME + '/' + Constants.USER_AGENT_PRODUCT_VERSION + ' ' + userAgentComment); // Initialize the location that the request is going to be sent to. if(azureutil.objectIsNull(options.locationMode)) { options.locationMode = this.defaultLocationMode; } // Initialize the location that the request can be sent to. if(azureutil.objectIsNull(options.requestLocationMode)) { options.requestLocationMode = defaultRequestLocationMode; } // Initialize whether nagling is used or not. if(azureutil.objectIsNull(options.useNagleAlgorithm)) { options.useNagleAlgorithm = this.useNagleAlgorithm; } this._initializeLocation(options); // Initialize the operationExpiryTime this._setOperationExpiryTime(options); var operation = function (options, next) { self._validateLocation(options); var currentLocation = options.currentLocation; self._buildRequestOptions(webResource, body, options, function (err, finalRequestOptions) { Iif (err) { callback({ error: err, response: null }, function (finalRequestOptions, finalCallback) { finalCallback(finalRequestOptions); }); } else { self.logger.log(Logger.LogLevels.DEBUG, 'FINAL REQUEST OPTIONS:\n' + util.inspect(finalRequestOptions)); Iif(self._maximumExecutionTimeExceeded(Date.now(), options.operationExpiryTime)) { callback({ error: new Error(SR.MAXIMUM_EXECUTION_TIMEOUT_EXCEPTION), response: null }, function (finalRequestOptions, finalCallback) { finalCallback(finalRequestOptions); }); } else { var processResponseCallback = function (error, response) { var responseObject; Iif (error) { responseObject = { error: error, response: null }; } else { responseObject = self._processResponse(webResource, response); responseObject.contentMD5 = response.contentMD5; responseObject.length = response.length; } responseObject.operationEndTime = new Date(); // Required for listing operations to make sure successive operations go to the same location. responseObject.targetLocation = currentLocation; callback(responseObject, next); }; var endResponse; var buildRequest = function (headersOnly) { // Build request (if body was set before, request will process immediately, if not it'll wait for the piping to happen var requestStream; var requestWithDefaults; Iif(self.proxy) { if(requestWithDefaults === undefined) { requestWithDefaults = request.defaults({'proxy':self.proxy}); } } else { requestWithDefaults = request; } if (headersOnly) { requestStream = requestWithDefaults(finalRequestOptions); requestStream.on('error', processResponseCallback); requestStream.on('response', function (response) { var responseLength = 0; var internalHash = crypto.createHash('md5'); requestStream.on('data', function(data) { responseLength += data.length; internalHash.update(data); }); response.on('end', function () { // Calculate and set MD5 here Eif(azureutil.objectIsNull(options.disableContentMD5Validation) || options.disableContentMD5Validation === false) { response.contentMD5 = internalHash.digest('base64'); } response.length = responseLength; endResponse = response; }); }); } else { requestStream = requestWithDefaults(finalRequestOptions, processResponseCallback); } //If useNagleAlgorithm is not set or the value is set and is false, setNoDelay is set to true. if (azureutil.objectIsNull(options.useNagleAlgorithm) || options.useNagleAlgorithm === false) { requestStream.on('request', function(httpRequest) { httpRequest.setNoDelay(true); }); } // Workaround to avoid request from potentially setting unwanted (rejected) headers by the service var oldEnd = requestStream.end; requestStream.end = function () { if (finalRequestOptions.headers['content-length']) { requestStream.headers['content-length'] = finalRequestOptions.headers['content-length']; } else Iif (requestStream.headers['content-length']) { delete requestStream.headers['content-length']; } oldEnd.call(requestStream); }; // Bubble events up -- This is when the request is going to be made. requestStream.on('response', function (response) { self.emit('receivedResponseEvent', response); }); return requestStream; }; if (body && body.outputData) { finalRequestOptions.body = body.outputData; } // Pipe any input / output streams if (body && body.inputStream) { body.inputStream.on('close', function () { Iif (endResponse) { processResponseCallback(null, endResponse); endResponse = null; } }); body.inputStream.on('end', function () { if (endResponse) { processResponseCallback(null, endResponse); endResponse = null; } }); body.inputStream.on('finish', function () { Eif (endResponse) { processResponseCallback(null, endResponse); endResponse = null; } }); buildRequest(true).pipe(body.inputStream); } else if (body && body.outputStream) { var sendUnchunked = function () { var size = finalRequestOptions.headers['content-length'] ? finalRequestOptions.headers['content-length'] : Constants.BlobConstants.MAX_SINGLE_UPLOAD_BLOB_SIZE_IN_BYTES; var concatBuf = new Buffer(size); var index = 0; body.outputStream.on('data', function (d) { if(self._maximumExecutionTimeExceeded(Date.now(), options.operationExpiryTime)) { processResponseCallback(new Error(SR.MAXIMUM_EXECUTION_TIMEOUT_EXCEPTION)); } else { d.copy(concatBuf, index); index += d.length; } }).on('end', function () { var requestStream = buildRequest(); requestStream.write(concatBuf); requestStream.end(); }); if (azureutil.isStreamPaused(body.outputStream)) { body.outputStream.resume(); } }; var sendStream = function () { // NOTE: workaround for an unexpected EPIPE exception when piping streams larger than 29 MB Eif (!azureutil.objectIsNull(finalRequestOptions.headers['content-length']) && finalRequestOptions.headers['content-length'] < 29 * 1024 * 1024) { body.outputStream.pipe(buildRequest()); if (azureutil.isStreamPaused(body.outputStream)) { body.outputStream.resume(); } } else { sendUnchunked(); } }; Iif (!body.outputStream.readable) { // if the content length is zero, build the request and don't send a body if (finalRequestOptions.headers['content-length'] === 0) { buildRequest(); } else { // otherwise, wait until we know the readable stream is actually valid before piping body.outputStream.on('open', function () { sendStream(); }); } } else { sendStream(); } // This catches any errors that happen while creating the readable stream (usually invalid names) body.outputStream.on('error', function (error) { processResponseCallback(error); }); } else { buildRequest(); } } } }); }; // The filter will do what it needs to the requestOptions and will provide a // function to be handled after the reply self.filter(options, function (postFiltersRequestOptions, nextPostCallback) { Iif(self._maximumExecutionTimeExceeded(Date.now() + postFiltersRequestOptions.retryInterval, postFiltersRequestOptions.operationExpiryTime)) { callback({ error: new Error(SR.MAXIMUM_EXECUTION_TIMEOUT_EXCEPTION), response: null}, function (postFiltersRequestOptions, finalCallback) { finalCallback(postFiltersRequestOptions); }); } else { // If there is a filter, flow is: // filter -> operation -> process response if(postFiltersRequestOptions.retryContext) { var func = function() { operation(postFiltersRequestOptions, nextPostCallback); }; // Sleep for retryInterval before making the request setTimeout(func, postFiltersRequestOptions.retryInterval); } else { // No retry policy filter specified operation(postFiltersRequestOptions, nextPostCallback); } } }); }; /** * Builds the request options to be passed to the http.request method. * @ignore * @param {WebResource} webResource The webresource where to build the options from. * @param {object} options The request options. * @param {function(error, requestOptions)} callback The callback function. * @return {undefined} */ StorageServiceClient.prototype._buildRequestOptions = function (webResource, body, options, callback) { webResource.withHeader(HeaderConstants.STORAGE_VERSION, this.apiVersion); webResource.withHeader(HeaderConstants.MS_DATE, new Date().toUTCString()); if (!webResource.headers[HeaderConstants.ACCEPT]) { webResource.withHeader(HeaderConstants.ACCEPT, 'application/atom+xml,application/xml'); } webResource.withHeader(HeaderConstants.ACCEPT_CHARSET, 'UTF-8'); if(azureutil.objectIsNull(options.timeoutIntervalInMs)) { options.timeoutIntervalInMs = this.defaultTimeoutIntervalInMs; } if(!azureutil.objectIsNull(options.timeoutIntervalInMs) && options.timeoutIntervalInMs > 0) { webResource.withQueryOption(QueryStringConstants.TIMEOUT, options.timeoutIntervalInMs); } webResource.withHeaders(options.accessConditions, HeaderConstants.IF_MATCH, HeaderConstants.IF_MODIFIED_SINCE, HeaderConstants.IF_NONE_MATCH, HeaderConstants.IF_UNMODIFIED_SINCE, HeaderConstants.SEQUENCE_NUMBER_EQUAL, HeaderConstants.SEQUENCE_NUMBER_LESS_THAN, HeaderConstants.SEQUENCE_NUMBER_LESS_THAN_OR_EQUAL); webResource.withHeaders(options.sourceAccessConditions, HeaderConstants.SOURCE_IF_MATCH, HeaderConstants.SOURCE_IF_MODIFIED_SINCE, HeaderConstants.SOURCE_IF_NONE_MATCH, HeaderConstants.SOURCE_IF_UNMODIFIED_SINCE); if (!webResource.headers || webResource.headers[HeaderConstants.CONTENT_TYPE] === undefined) { // work around to add an empty content type header to prevent the request module from magically adding a content type. webResource.headers[HeaderConstants.CONTENT_TYPE] = ''; } else Iif (webResource.headers && webResource.headers[HeaderConstants.CONTENT_TYPE] === null) { delete webResource.headers[HeaderConstants.CONTENT_TYPE]; } if (!webResource.headers || webResource.headers[HeaderConstants.CONTENT_LENGTH] === undefined) { Iif (body && body.outputData) { webResource.withHeader(HeaderConstants.CONTENT_LENGTH, Buffer.byteLength(body.outputData, 'UTF8')); } else Eif (webResource.headers[HeaderConstants.CONTENT_LENGTH] === undefined) { webResource.withHeader(HeaderConstants.CONTENT_LENGTH, 0); } } else Iif (webResource.headers && webResource.headers[HeaderConstants.CONTENT_LENGTH] === null) { delete webResource.headers[HeaderConstants.CONTENT_LENGTH]; } // Sets the request url in the web resource. this._setRequestUrl(webResource, options); this.emit('sendingRequestEvent', webResource); // Now that the web request is finalized, sign it this.storageCredentials.signRequest(webResource, function (error) { var requestOptions = null; Eif (!error) { var targetUrl = webResource.uri; requestOptions = { url: url.format(targetUrl), method: webResource.method, headers: webResource.headers, }; Eif(options) { //set encoding of response data. If set to null, the body is returned as a Buffer requestOptions.encoding = options.responseEncoding; } } callback(error, requestOptions); }); }; /** * Process the response. * @ignore * * @param {WebResource} webResource The web resource that made the request. * @param {Response} response The response object. * @return The normalized responseObject. */ StorageServiceClient.prototype._processResponse = function (webResource, response) { var self = this; var validResponse = WebResource.validResponse(response.statusCode); var rsp = StorageServiceClient._buildResponse(validResponse, response.body, response.headers, response.statusCode, response.md5); var responseObject; if (validResponse && webResource.rawResponse) { responseObject = { error: null, response: rsp }; } else { // attempt to parse the response body, errors will be returned in rsp.error without modifying the body rsp = StorageServiceClient._parseResponse(rsp, self.xml2jsSettings); if (validResponse && !rsp.error) { responseObject = { error: null, response: rsp }; } else { rsp.isSuccessful = false; Iif (response.statusCode < 400 || response.statusCode >= 500) { this.logger.log(Logger.LogLevels.DEBUG, 'ERROR code = ' + response.statusCode + ' :\n' + util.inspect(rsp.body)); } // responseObject.error should contain normalized parser errors if they occured in _parseResponse // responseObject.response.body should contain the raw response body in that case var errorBody = rsp.body; Iif(rsp.error) { errorBody = rsp.error; delete rsp.error; } if (!errorBody) { var code = Object.keys(HttpResponseCodes).filter(function (name) { if (HttpResponseCodes[name] === rsp.statusCode) { return name; } }); errorBody = { error: { code: code[0] } }; } var normalizedError = StorageServiceClient._normalizeError(errorBody, response); responseObject = { error: normalizedError, response: rsp }; } } this.logger.log(Logger.LogLevels.DEBUG, 'RESPONSE:\n' + util.inspect(responseObject)); return responseObject; }; /** * Associate a filtering operation with this StorageServiceClient. Filtering operations * can include logging, automatically retrying, etc. Filter operations are objects * that implement a method with the signature: * * "function handle (requestOptions, next)". * * After doing its preprocessing on the request options, the method needs to call * "next" passing a callback with the following signature: * signature: * * "function (returnObject, finalCallback, next)" * * In this callback, and after processing the returnObject (the response from the * request to the server), the callback needs to either invoke next if it exists to * continue processing other filters or simply invoke finalCallback otherwise to end * up the service invocation. * * @param {Object} filter The new filter object. * @return {StorageServiceClient} A new service client with the filter applied. */ StorageServiceClient.prototype.withFilter = function (newFilter) { // Create a new object with the same members as the current service var derived = _.clone(this); // If the current service has a filter, merge it with the new filter // (allowing us to effectively pipeline a series of filters) var parentFilter = this.filter; var mergedFilter = newFilter; Eif (parentFilter !== undefined) { // The parentFilterNext is either the operation or the nextPipe function generated on a previous merge // Ordering is [f3 pre] -> [f2 pre] -> [f1 pre] -> operation -> [f1 post] -> [f2 post] -> [f3 post] mergedFilter = function (originalRequestOptions, parentFilterNext) { newFilter.handle(originalRequestOptions, function (postRequestOptions, newFilterCallback) { // handle parent filter pre and get Parent filter post var next = function (postPostRequestOptions, parentFilterCallback) { // The parentFilterNext is the filter next to the merged filter. // For 2 filters, that'd be the actual operation. parentFilterNext(postPostRequestOptions, function (responseObject, responseCallback, finalCallback) { parentFilterCallback(responseObject, finalCallback, function (postResponseObject) { newFilterCallback(postResponseObject, responseCallback, finalCallback); }); }); }; parentFilter(postRequestOptions, next); }); }; } // Store the filter so it can be applied in performRequest derived.filter = mergedFilter; return derived; }; /* * Builds a response object with normalized key names. * @ignore * * @param {Bool} isSuccessful Boolean value indicating if the request was successful * @param {Object} body The response body. * @param {Object} headers The response headers. * @param {int} statusCode The response status code. * @param {string} md5 The response's content md5 hash. * @return {Object} A response object. */ StorageServiceClient._buildResponse = function (isSuccessful, body, headers, statusCode, md5) { return { isSuccessful: isSuccessful, statusCode: statusCode, body: body, headers: headers, md5: md5 }; }; /** * Parses a server response body from XML or JSON into a JS object. * This is done using the xml2js library. * @ignore * * @param {object} response The response object with a property "body" with a XML or JSON string content. * @return {object} The same response object with the body part as a JS object instead of a XML or JSON string. */ StorageServiceClient._parseResponse = function (response, xml2jsSettings) { function parseXml(body) { var parsed; var parser = new xml2js.Parser(xml2jsSettings); parser.parseString(azureutil.removeBOM(body.toString()), function (err, parsedBody) { Iif (err) { throw err; } else { parsed = parsedBody; } }); return parsed; } if (response.body && Buffer.byteLength(response.body.toString()) > 0) { var contentType = ''; Eif (response.headers && response.headers['content-type']) { contentType = response.headers['content-type'].toLowerCase(); } try { if (contentType.indexOf('application/json') !== -1) { response.body = JSON.parse(response.body); } else Eif (contentType.indexOf('application/xml') !== -1 || contentType.indexOf('application/atom+xml') !== -1) { response.body = parseXml(response.body); } else { throw new Error(SR.CONTENT_TYPE_MISSING); } } catch (e) { response.error = e; } } return response; }; /** * Gets the storage settings. * * @param {string} [storageAccountOrConnectionString] The storage account or the connection string. * @param {string} [storageAccessKey] The storage access key. * @param {string} [host] The host address. * @param {object} [sasToken] The sas token. * * @return {StorageServiceSettings} */ StorageServiceClient.getStorageSettings = function (storageAccountOrConnectionString, storageAccessKey, host, sasToken) { var storageServiceSettings; if (storageAccountOrConnectionString && !storageAccessKey && !sasToken) { // If storageAccountOrConnectionString was passed and no accessKey was passed, assume connection string storageServiceSettings = StorageServiceSettings.createFromConnectionString(storageAccountOrConnectionString); } else if ((storageAccountOrConnectionString && storageAccessKey) || sasToken || host) { // Account and key or credentials or anonymous storageServiceSettings = StorageServiceSettings.createExplicitly(storageAccountOrConnectionString, storageAccessKey, host, sasToken); } else { // Use environment variables storageServiceSettings = StorageServiceSettings.createFromEnvironment(); } return storageServiceSettings; }; /** * Sets the webResource's requestUrl based on the service client settings. * @ignore * * @param {WebResource} webResource The web resource where to set the request url. * @return {undefined} */ StorageServiceClient.prototype._setRequestUrl = function (webResource, options) { // Normalize the path webResource.path = this._getPath(webResource.path); Iif(!this.host){ throw new Error(SR.STORAGE_HOST_LOCATION_REQUIRED); } var host = this.host.primaryHost; if(!azureutil.objectIsNull(options) && options.currentLocation === Constants.StorageLocation.SECONDARY) { host = this.host.secondaryHost; } webResource.uri = url.resolve(host, url.format({pathname: webResource.path, query: webResource.queryString})); webResource.path = url.parse(webResource.uri).pathname; }; /** * Retrieves the normalized path to be used in a request. * This takes into consideration the usePathStyleUri object field * which specifies if the request is against the emulator or against * the production service. It also adds a leading "/" to the path in case * it's not there before. * @ignore * @param {string} path The path to be normalized. * @return {string} The normalized path. */ StorageServiceClient.prototype._getPath = function (path) { if (path === null || path === undefined) { path = '/'; } else if (path.indexOf('/') !== 0) { path = '/' + path; } Iif (this.usePathStyleUri) { path = '/' + this.storageAccount + path; } return path; }; /** * Initializes the default filter. * This filter is responsible for chaining the pre filters request into the operation and, after processing the response, * pass it to the post processing filters. This method should only be invoked by the StorageServiceClient constructor. * @ignore * * @return {undefined} */ StorageServiceClient.prototype._initDefaultFilter = function () { this.filter = function (requestOptions, nextPreCallback) { Eif (nextPreCallback) { // Handle the next pre callback and pass the function to be handled as post call back. nextPreCallback(requestOptions, function (returnObject, finalCallback, nextPostCallback) { Eif (nextPostCallback) { nextPostCallback(returnObject); } else if (finalCallback) { finalCallback(returnObject); } }); } }; }; /** * Retrieves the metadata headers from the response headers. * @ignore * * @param {object} headers The metadata headers. * @return {object} An object with the metadata headers (without the "x-ms-" prefix). */ StorageServiceClient.prototype.parseMetadataHeaders = function (headers) { var metadata = {}; Iif (!headers) { return metadata; } for (var header in headers) { if (header.indexOf(HeaderConstants.PREFIX_FOR_STORAGE_METADATA) === 0) { var key = header.substr(HeaderConstants.PREFIX_FOR_STORAGE_METADATA.length, header.length - HeaderConstants.PREFIX_FOR_STORAGE_METADATA.length); metadata[key] = headers[header]; } } return metadata; }; /** * Gets the properties of a storage account’s service, including Azure Storage Analytics. * @ignore * * @this {StorageServiceClient} * @param {object} [options] The request options. * @param {LocationMode} [options.locationMode] Specifies the location mode used to decide which location the request should be sent to. * Please see StorageUtilities.LocationMode for the possible values. * @param {int} [options.timeoutIntervalInMs] The server timeout interval, in milliseconds, to use for the request. * @param {int} [options.maximumExecutionTimeInMs] The maximum execution time, in milliseconds, across all potential retries, to use when making this request. * The maximum execution time interval begins at the time that the client begins building the request. The maximum * execution time is checked intermittently while performing requests, and before executing retries. * @param {bool} [options.useNagleAlgorithm] Determines whether the Nagle algorithm is used; true to use the Nagle algorithm; otherwise, false. * The default value is false. * @param {errorOrResult} callback `error` will contain information if an error occurs; otherwise, `result` will contain the properties * and `response` will contain information related to this operation. */ StorageServiceClient.prototype.getAccountServiceProperties = function (optionsOrCallback, callback) { var userOptions; azureutil.normalizeArgs(optionsOrCallback, callback, function (o, c) { userOptions = o; callback = c; }); validate.validateArgs('getServiceProperties', function (v) { v.callback(callback); }); var options = extend(true, {}, userOptions); var webResource = WebResource.get() .withQueryOption(QueryStringConstants.COMP, 'properties') .withQueryOption(QueryStringConstants.RESTYPE, 'service'); options.requestLocationMode = RequestLocationMode.PRIMARY_OR_SECONDARY; var processResponseCallback = function (responseObject, next) { responseObject.servicePropertiesResult = null; Eif (!responseObject.error) { responseObject.servicePropertiesResult = ServicePropertiesResult.parse(responseObject.response.body.StorageServiceProperties); } // function to be called after all filters var finalCallback = function (returnObject) { callback(returnObject.error, returnObject.servicePropertiesResult, returnObject.response); }; // call the first filter next(responseObject, finalCallback); }; this.performRequest(webResource, null, options, processResponseCallback); }; /** * Sets the properties of a storage account’s service, including Azure Storage Analytics. * You can also use this operation to set the default request version for all incoming requests that do not have a version specified. * * @this {StorageServiceClient} * @param {object} serviceProperties The service properties. * @param {object} [options] The request options. * @param {LocationMode} [options.locationMode] Specifies the location mode used to decide which location the request should be sent to. * Please see StorageUtilities.LocationMode for the possible values. * @param {int} [options.timeoutIntervalInMs] The server timeout interval, in milliseconds, to use for the request. * @param {int} [options.maximumExecutionTimeInMs] The maximum execution time, in milliseconds, across all potential retries, to use when making this request. * The maximum execution time interval begins at the time that the client begins building the request. The maximum * execution time is checked intermittently while performing requests, and before executing retries. * @param {bool} [options.useNagleAlgorithm] Determines whether the Nagle algorithm is used; true to use the Nagle algorithm; otherwise, false. * The default value is false. * @param {errorOrResponse} callback `error` will contain information * if an error occurs; otherwise, `response` * will contain information related to this operation. */ StorageServiceClient.prototype.setAccountServiceProperties = function (serviceProperties, optionsOrCallback, callback) { var userOptions; azureutil.normalizeArgs(optionsOrCallback, callback, function (o, c) { userOptions = o; callback = c; }); validate.validateArgs('setServiceProperties', function (v) { v.object(serviceProperties, 'serviceProperties'); v.callback(callback); }); var options = extend(true, {}, userOptions); var servicePropertiesXml = ServicePropertiesResult.serialize(serviceProperties); var webResource = WebResource.put() .withQueryOption(QueryStringConstants.COMP, 'properties') .withQueryOption(QueryStringConstants.RESTYPE, 'service') .withHeader(HeaderConstants.CONTENT_TYPE, 'application/xml;charset="utf-8"') .withHeader(HeaderConstants.CONTENT_LENGTH, Buffer.byteLength(servicePropertiesXml)) .withBody(servicePropertiesXml); var processResponseCallback = function (responseObject, next) { var finalCallback = function (returnObject) { callback(returnObject.error, returnObject.response); }; next(responseObject, finalCallback); }; this.performRequest(webResource, webResource.body, options, processResponseCallback); }; // Other functions /** * Processes the error body into a normalized error object with all the properties lowercased. * * Error information may be returned by a service call with additional debugging information: * http://msdn.microsoft.com/en-us/library/windowsazure/dd179382.aspx * * Table services returns these properties lowercased, example, "code" instead of "Code". So that the user * can always expect the same format, this method lower cases everything. * * @ignore * * @param {Object} error The error object as returned by the service and parsed to JSON by the xml2json. * @return {Object} The normalized error object with all properties lower cased. */ StorageServiceClient._normalizeError = function (error, response) { Iif (azureutil.objectIsString(error)) { return new Error(error); } else Eif (error) { var normalizedError = {}; // blob/queue errors should have error.Error, table errors should have error['odata.error'] var errorProperties = error.Error || error.error || error['odata.error'] || error; for (var property in errorProperties) { Eif (errorProperties.hasOwnProperty(property)) { var key = property.toLowerCase(); normalizedError[key] = errorProperties[property]; // if this is a table error, message is an object - flatten it to normalize with blob/queue errors // ex: "message":{"lang":"en-US","value":"The specified resource does not exist."} becomes message: "The specified resource does not exist." if (key === 'message' && _.isObject(errorProperties[property])) { Eif (errorProperties[property]['value']) { normalizedError[key] = errorProperties[property]['value']; } } } } // add status code and server request id if available if (response) { Eif (response.statusCode) { normalizedError.statusCode = response.statusCode; } if (response.headers && response.headers['x-ms-request-id']) { normalizedError.requestId = response.headers['x-ms-request-id']; } } var errorObject = new Error(normalizedError.code); _.extend(errorObject, normalizedError); return errorObject; } return null; }; /* * Sets proxy object specified by caller. * * @param {object} proxy proxy to use for tunneling * { * host: hostname * port: port number * proxyAuth: 'user:password' for basic auth * headers: {...} headers for proxy server * key: key for proxy server * ca: ca for proxy server * cert: cert for proxy server * } * if null or undefined, clears proxy */ StorageServiceClient.prototype.setProxy = function (proxy) { Iif (proxy) { this.proxy = proxy; } else { this.proxy = null; } }; /** * Sets the service host default proxy from the environment. * Can be overridden by calling _setProxyUrl or _setProxy * */ StorageServiceClient.prototype._setDefaultProxy = function () { var proxyUrl = StorageServiceClient._loadEnvironmentProxyValue(); Iif (proxyUrl) { var parsedUrl = url.parse(proxyUrl); if (!parsedUrl.port) { parsedUrl.port = 80; } this.setProxy(parsedUrl); } else { this.setProxy(null); } }; /* * Loads the fields "useProxy" and respective protocol, port and url * from the environment values HTTPS_PROXY and HTTP_PROXY * in case those are set. * @ignore * * @return {string} or null */ StorageServiceClient._loadEnvironmentProxyValue = function () { var proxyUrl = null; Iif (process.env[StorageServiceClientConstants.EnvironmentVariables.HTTPS_PROXY]) { proxyUrl = process.env[StorageServiceClientConstants.EnvironmentVariables.HTTPS_PROXY]; } else Iif (process.env[StorageServiceClientConstants.EnvironmentVariables.HTTPS_PROXY.toLowerCase()]) { proxyUrl = process.env[StorageServiceClientConstants.EnvironmentVariables.HTTPS_PROXY.toLowerCase()]; } else Iif (process.env[StorageServiceClientConstants.EnvironmentVariables.HTTP_PROXY]) { proxyUrl = process.env[StorageServiceClientConstants.EnvironmentVariables.HTTP_PROXY]; } else Iif (process.env[StorageServiceClientConstants.EnvironmentVariables.HTTP_PROXY.toLowerCase()]) { proxyUrl = process.env[StorageServiceClientConstants.EnvironmentVariables.HTTP_PROXY.toLowerCase()]; } return proxyUrl; }; /** * Initializes the location to which the operation is being sent to. */ StorageServiceClient.prototype._initializeLocation = function (options) { Eif(!azureutil.objectIsNull(options.locationMode)) { switch(options.locationMode) { case StorageUtilities.LocationMode.PRIMARY_ONLY: case StorageUtilities.LocationMode.PRIMARY_THEN_SECONDARY: options.currentLocation = Constants.StorageLocation.PRIMARY; break; case StorageUtilities.LocationMode.SECONDARY_ONLY: case StorageUtilities.LocationMode.SECONDARY_THEN_PRIMARY: options.currentLocation = Constants.StorageLocation.SECONDARY; break; default: throw new Error(util.format(SR.ARGUMENT_OUT_OF_RANGE_ERROR, 'locationMode', options.locationMode)); } } else { options.locationMode = StorageUtilities.LocationMode.PRIMARY_ONLY; options.currentLocation = Constants.StorageLocation.PRIMARY; } }; /** * Validates the location to which the operation is being sent to. */ StorageServiceClient.prototype._validateLocation = function (options) { if(this._invalidLocationMode(options.locationMode)) { throw new Error(SR.STORAGE_HOST_MISSING_LOCATION); } switch(options.requestLocationMode) { case Constants.RequestLocationMode.PRIMARY_ONLY: Iif(options.locationMode === StorageUtilities.LocationMode.SECONDARY_ONLY) { throw new Error(SR.PRIMARY_ONLY_COMMAND); } options.currentLocation = Constants.StorageLocation.PRIMARY; options.locationMode = StorageUtilities.LocationMode.PRIMARY_ONLY; break; case Constants.RequestLocationMode.SECONDARY_ONLY: if(options.locationMode === StorageUtilities.LocationMode.PRIMARY_ONLY) { throw new Error(SR.SECONDARY_ONLY_COMMAND); } options.currentLocation = Constants.StorageLocation.SECONDARY; options.locationMode = StorageUtilities.LocationMode.SECONDARY_ONLY; break; default: // no op } }; /** * Checks whether we have the relevant host information based on the locationMode. */ StorageServiceClient.prototype._invalidLocationMode = function (locationMode) { switch(locationMode) { case StorageUtilities.LocationMode.PRIMARY_ONLY: return azureutil.objectIsNull(this.host.primaryHost); case StorageUtilities.LocationMode.SECONDARY_ONLY: return azureutil.objectIsNull(this.host.secondaryHost); default: return (azureutil.objectIsNull(this.host.primaryHost) || azureutil.objectIsNull(this.host.secondaryHost)); } }; /** * Checks to see if the maximum execution timeout provided has been exceeded. */ StorageServiceClient.prototype._maximumExecutionTimeExceeded = function (currentTime, expiryTime) { Iif(!azureutil.objectIsNull(expiryTime) && currentTime > expiryTime) { return true; } else { return false; } }; /** * Sets the operation expiry time. */ StorageServiceClient.prototype._setOperationExpiryTime = function (options) { Eif(azureutil.objectIsNull(options.operationExpiryTime)) { Iif(!azureutil.objectIsNull(options.maximumExecutionTimeInMs)) { options.operationExpiryTime = Date.now() + options.maximumExecutionTimeInMs; } else Iif(this.defaultMaximumExecutionTimeInMs) { options.operationExpiryTime = Date.now() + this.defaultMaximumExecutionTimeInMs; } } }; module.exports = StorageServiceClient; |