class for manipulating an array of objects, typically from CSV data

memberof

  preprocessing

### Hierarchy

- DataSet

## Index

### Constructors

- constructor

### Properties

- columnArray
- columnMatrix
- config
- data
- encodeObject
- encoders
- getTransforms
- labels
- oneHotDecoder
- oneHotEncoder
- reverseColumnMatrix
- reverseColumnVector
- scalers
- selectColumns
- data
- encoders

### Methods

- columnDescale
- columnMerge
- columnReducer
- columnReplace
- columnScale
- exportFeatures
- filterColumn
- fitColumns
- fitInverseTransforms
- fitTransforms
- importFeatures
- inverseTransformObject
- labelDecode
- labelEncoder
- oneHotColumnArray
- transformObject
- columnArray
- columnMatrix
- encodeObject
- getBinaryValue
- getTransforms
- mapToObject
- oneHotColumnArray
- oneHotDecoder
- oneHotEncoder
- reverseColumnMatrix
- reverseColumnVector
- selectColumns

## Constructors

### constructor

- new DataSet(data?: Data, options?: {}): DataSet

- - Defined in src/DataSet.ts:465

  creates a new raw data instance for preprocessing data for machine learning

  example

    const dataset = new ms.DataSet(csvData);

  #### Parameters

  - ##### data: Data = ...

  - ##### options: {} = ...

  #### Returns DataSet

## Properties

### columnArray

columnArray: (...args: any[]) => any

- Defined in src/DataSet.ts:75

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### columnMatrix

columnMatrix: (...args: any[]) => any

- Defined in src/DataSet.ts:79

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### config

config: {}

- Defined in src/DataSet.ts:69

#### Type declaration

- ##### [index: string]: any

### data

data: Data

- Defined in src/DataSet.ts:70

### encodeObject

encodeObject: (...args: any[]) => any

- Defined in src/DataSet.ts:76

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### encoders

encoders: any

- Defined in src/DataSet.ts:72

### getTransforms

getTransforms: (...args: any[]) => any

- Defined in src/DataSet.ts:82

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### labels

labels: any

- Defined in src/DataSet.ts:71

### oneHotDecoder

oneHotDecoder: (...args: any[]) => any

- Defined in src/DataSet.ts:78

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### oneHotEncoder

oneHotEncoder: (...args: any[]) => any

- Defined in src/DataSet.ts:77

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### reverseColumnMatrix

reverseColumnMatrix: (...args: any[]) => any

- Defined in src/DataSet.ts:80

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### reverseColumnVector

reverseColumnVector: (...args: any[]) => any

- Defined in src/DataSet.ts:81

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### scalers

scalers: any

- Defined in src/DataSet.ts:73

### selectColumns

selectColumns: (...args: any[]) => any

- Defined in src/DataSet.ts:74

#### Type declaration

- - (...args: any[]): any

  - #### Parameters

    - ##### Rest ...args: any[]

    #### Returns any

### Static data

data: any

- Defined in src/DataSet.ts:84

### Static encoders

encoders: any

- Defined in src/DataSet.ts:83

## Methods

### columnDescale

- columnDescale(name: any, options: any): any

- - Defined in src/DataSet.ts:684

  Returns a new array of descaled values

  example

    //dataset.columnArray('Age') => [ '44','27','30','38','40','35',38.77777777777778,'48','50','37' ] const scaledData = [ 3.784189633918261, 3.295836866004329, 3.4011973816621555, 3.6375861597263857, 3.6888794541139363, 3.5553480614894135, 3.657847344866208, 3.8712010109078907, 3.912023005428146, 3.6109179126442243 ] dataset.columnDescale('Age') // => [ '44','27','30','38','40','35',38.77777777777778,'48','50','37' ]

  #### Parameters

  - ##### name: any

    name - csv column header, or JSON object property name

  - ##### options: any

  #### Returns any

  returns an array of scaled values

### columnMerge

- columnMerge(name: any, data?: never[]): {}

- - Defined in src/DataSet.ts:830

  it returns a new column that is merged onto the data set

  example

    CSVDataSet.columnMerge('DoubleAge', [ 88, 54, 60, 76, 80, 70, 0, 96, 100, 74 ]); //=> { DoubleAge: [ 88, 54, 60, 76, 80, 70, 0, 96, 100, 74 ] }

  #### Parameters

  - ##### name: any

    name of new Column

  - ##### data: never[] = ...

    new dataset data

  #### Returns {}

### columnReducer

- columnReducer(name: any, options: { columnName: any; columnOptions: any; reducer: any }): {}

- - Defined in src/DataSet.ts:816

  it returns a new column that reduces a column into a new column object, this is used in data prep to create new calculated columns for aggregrate statistics

  example

    const reducer = (result, value, index, arr) => { result.push(value * 2); return result; }; CSVDataSet.columnReducer('DoubleAge', { columnName: 'Age', reducer, }); //=> { DoubleAge: [ 88, 54, 60, 76, 80, 70, 0, 96, 100, 74 ] }

  #### Parameters

  - ##### name: any

    name of new Column

  - ##### options: { columnName: any; columnOptions: any; reducer: any }

    - ##### columnName: any

      name property for columnArray selection

    - ##### columnOptions: any

      options property for columnArray

    - ##### reducer: any

      reducer function to reduce into new array, it should push values into the resulting array

  #### Returns {}

  a new object that has reduced array as the value

### columnReplace

- columnReplace(name: any, options?: any): any

- - Defined in src/DataSet.ts:992

  returns a new array of a selected column from an array of objects and replaces empty values, encodes values and scales values

  example

    //column Replace returns new Array with replaced missing data //[ '44','27','30','38','40','35',38.77777777777778,'48','50','37' ] const ReplacedAgeMeanColumn = dataset.columnReplace('Age',{strategy:'mean'});

  #### Parameters

  - ##### name: any

    csv column header, or JSON object property name

  - ##### options: any = ...

  #### Returns any

### columnScale

- columnScale(name: any, options?: {}): number[]

- - Defined in src/DataSet.ts:605

  Returns a new array of scaled values which can be reverse (descaled). The scaling transformations are stored on the DataSet

  example

    //dataset.columnArray('Age') => [ '44','27','30','38','40','35',38.77777777777778,'48','50','37' ] dataset.columnScale('Age',{strategy:'log'}) // => [ 3.784189633918261, 3.295836866004329, 3.4011973816621555, 3.6375861597263857, 3.6888794541139363, 3.5553480614894135, 3.657847344866208, 3.8712010109078907, 3.912023005428146, 3.6109179126442243 ] dataset.scalers.get('Age').scale(45) // => 3.8066624897703196 dataset.scalers.get('Age').descale(3.8066624897703196) // => 45 //this supports, log/exponent, minmax/normalization and standardscaling

  #### Parameters

  - ##### name: any

    name - csv column header, or JSON object property name

  - ##### options: {} = ...

  #### Returns number[]

  returns an array of scaled values

### exportFeatures

- exportFeatures(options?: {}): { encoders: any; labels: any; scalers: any }

- - Defined in src/DataSet.ts:508

  returns Object of all encoders and scalers

  example

    const csvObj = new DataSet([{col1:1,col2:5},{col1:false,col2:6}]); DataSet.fitColumns({col1:['label',{binary:true}]}); Dataset.data // => [{col1:true,col2:5},{col1:false,col2:6}] Dataset.exportFeatures() //=> { labels: { col1: { "0": false, "1": true, "N": 0, "Yes": 1, "No": 0, "f": 0, "false": 1, } } }

  #### Parameters

  - ##### options: {} = ...

  #### Returns { encoders: any; labels: any; scalers: any }

  JavaScript Object of transforms encoders and scalers(labels, encoders, scalers)

  - ##### encoders: any

  - ##### labels: any

  - ##### scalers: any

### filterColumn

- filterColumn(filter?: () => boolean): Datum[]

- - Defined in src/DataSet.ts:589

  returns filtered rows of data

  example

    const csvObj = new DataSet([{col1:1,col2:5},{col1:2,col2:6}]); csvObj.filterColumn((row)=>row.col1>=2); // => //[ // [2,6], //]

  #### Parameters

  - ##### filter: () => boolean = ...

    - - (): boolean

      - #### Returns boolean

  #### Returns Datum[]

  filtered array of data

### fitColumns

- fitColumns(options?: any, mockDataOptions?: {}): Data | DataSet

- - Defined in src/DataSet.ts:1093

  mutates data property of DataSet by replacing multiple columns in a single command

  example

    //fit Columns, mutates dataset dataset.fitColumns({ columns:[{name:'Age',options:{ strategy:'mean'} }] }); // dataset // class DataSet // data:[ // { // 'Country': 'Brazil', // 'Age': '38.77777777777778', // 'Salary': '72000', // 'Purchased': 'N', // } // ... // ]

  #### Parameters

  - ##### options: any = ...

  - ##### mockDataOptions: {} = ...

  #### Returns Data | DataSet

### fitInverseTransforms

- fitInverseTransforms(options?: any): Data | DataSet

- - Defined in src/DataSet.ts:1161

  Mutate dataset data by inversing all transforms

  example

    DataSet.data; // [{ // Country: 'Brazil', // Age: 3.784189633918261, // Salary: '72000', // Purchased: 'N', // Country_Brazil: 1, // Country_Mexico: 0, // Country_Ghana: 0 // }, // ... // ] DataSet.fitInverseTransforms(); // => // [{ // 'Country': 'Brazil', // 'Age': '44', // 'Salary': '72000', // 'Purchased': 'N', // }, // ... // ]

  #### Parameters

  - ##### options: any = ...

  #### Returns Data | DataSet

### fitTransforms

- fitTransforms(options?: any): Data | DataSet

- - Defined in src/DataSet.ts:1198

  Mutate dataset data with all transforms

  example

    DataSet.data; // [{ // 'Country': 'Brazil', // 'Age': '44', // 'Salary': '72000', // 'Purchased': 'N', // }, // ... // ] DataSet.fitTransforms(); // => // [{ // Country: 'Brazil', // Age: 3.784189633918261, // Salary: '72000', // Purchased: 'N', // Country_Brazil: 1, // Country_Mexico: 0, // Country_Ghana: 0 // }, // ... // ]

  #### Parameters

  - ##### options: any = ...

  #### Returns Data | DataSet

### importFeatures

- importFeatures(features?: any): void

- - Defined in src/DataSet.ts:525

  set encoders, labels and scalers

  example

    const csvObj = new DataSet([{col1:1,col2:5},{col1:false,col2:6}]); DataSet.fitColumns({col1:['label',{binary:true}]}); Dataset.data // => [{col1:true,col2:5},{col1:false,col2:6}] Dataset.exportFeatures() //=> { labels: { col1: { "0": false, "1": true, "N": 0, "Yes": 1, "No": 0, "f": 0, "false": 1, } } }

  #### Parameters

  - ##### features: any = ...

  #### Returns void

### inverseTransformObject

- inverseTransformObject(data: {}, options: {}): {}

- - Defined in src/DataSet.ts:857

  Inverses transform on an object

  example

    DataSet.data; //[{ // Age: 0.6387122698222066, // Salary: 72000, // Purchased: 0, // Country_Brazil: 1, // Country_Mexico: 0, // Country_Ghana: 0, // }, ...] DataSet.inverseTransformObject(DataSet.data[0]); // => { // Country: 'Brazil', // Age: 44, // Salary: 72000, // Purchased: 'N', // };

  #### Parameters

  - ##### data: {}

    - ##### [x: string]: any

  - ##### options: {}

  #### Returns {}

  returns object with inverse transformed data

  - ##### [x: string]: any

### labelDecode

- labelDecode(name: any, options?: any): any

- - Defined in src/DataSet.ts:749

  returns a new array and decodes an encoded column back to the original array values

  #### Parameters

  - ##### name: any

    csv column header, or JSON object property name

  - ##### options: any = ...

  #### Returns any

### labelEncoder

- labelEncoder(name: any, options: {}): any

- - Defined in src/DataSet.ts:708

  returns a new array and label encodes a selected column

  example

    const oneHotCountryColumn = dataset.oneHotEncoder('Country');

    // [ 'N', 'Yes', 'No', 'f', 'Yes', 'Yes', 'false', 'Yes', 'No', 'Yes' ] const originalPurchasedColumn = dataset.labelEncoder('Purchased'); // [ 0, 1, 0, 0, 1, 1, 1, 1, 0, 1 ] const encodedBinaryPurchasedColumn = dataset.labelEncoder('Purchased',{ binary:true }); // [ 0, 1, 2, 3, 1, 1, 4, 1, 2, 1 ] const encodedPurchasedColumn = dataset.labelEncoder('Purchased');

  see

    http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.LabelEncoder.html

  #### Parameters

  - ##### name: any

    csv column header, or JSON object property name

  - ##### options: {}

  #### Returns any

### oneHotColumnArray

- oneHotColumnArray(name: any, options: any): any

- - Defined in src/DataSet.ts:790

  Return one hot encoded data

  example

    const csvData = [{ 'Country': 'Brazil', 'Age': '44', 'Salary': '72000', 'Purchased': 'N', }, { 'Country': 'Mexico', 'Age': '27', 'Salary': '48000', 'Purchased': 'Yes', }, ... ]; const EncodedCSVDataSet = new ms.preprocessing.DataSet(csvData); EncodedCSVDataSet.fitColumns({ columns: [ { name: 'Country', options: { strategy: 'onehot', }, }, ], });

    EncodedCSVDataSet.oneHotColumnArray('Country);// => // [ { Country_Brazil: 1, Country_Mexico: 0, Country_Ghana: 0 }, // { Country_Brazil: 0, Country_Mexico: 1, Country_Ghana: 0 }, // { Country_Brazil: 0, Country_Mexico: 0, Country_Ghana: 1 }, // ...]

  #### Parameters

  - ##### name: any

    column name

  - ##### options: any

  #### Returns any

  returns an array of objects from an one hot encoded column

### transformObject

- transformObject(data: {}, options: {}): {}

- - Defined in src/DataSet.ts:922

  transforms an object and replaces values that have been scaled or encoded

  example

    DataSet.transformObject({ 'Country': 'Brazil', 'Age': '44', 'Salary': '72000', 'Purchased': 'N', }); // => // { // Country: 'Brazil', // Age: 3.784189633918261, // Salary: '72000', // Purchased: 'N', // Country_Brazil: 1, // Country_Mexico: 0, // Country_Ghana: 0 // }

  #### Parameters

  - ##### data: {}

    - ##### [x: string]: any

  - ##### options: {}

  #### Returns {}

  - ##### [x: string]: any

### Static columnArray

- columnArray(name: string | number, options?: any): any

- - Defined in src/DataSet.ts:341

  returns a new array of a selected column from an array of objects, can filter, scale and replace values

  example

    //column Array returns column of data by name // [ '44','27','30','38','40','35','','48','50', '37' ] const OringalAgeColumn = dataset.columnArray('Age');

  #### Parameters

  - ##### name: string | number

    csv column header, or JSON object property name

  - ##### options: any = ...

  #### Returns any

### Static columnMatrix

- columnMatrix(vectors?: never[], data?: never[]): Matrix

- - Defined in src/DataSet.ts:404

  returns a matrix of values by combining column arrays into a matrix

  example

    const csvObj = new DataSet([{col1:1,col2:5},{col1:2,col2:6}]); csvObj.columnMatrix([['col1',{parseInt:true}],['col2']]); // => //[ // [1,5], // [2,6], //]

  #### Parameters

  - ##### vectors: never[] = ...

  - ##### data: never[] = ...

  #### Returns Matrix

  a matrix of column values

### Static encodeObject

- encodeObject(data: Datum, options: { labels: string[]; name: string; prefix: string }): Datum

- - Defined in src/DataSet.ts:179

  Returns an object into an one hot encoded object

  example

    const labels = ['apple', 'orange', 'banana',]; const prefix = 'fruit_'; const name = 'fruit'; const options = { labels, prefix, name, }; const data = { fruit: 'apple', }; EncodedCSVDataSet.encodeObject(data, options); // => { fruit_apple: 1, fruit_orange: 0, fruit_banana: 0, }

  #### Parameters

  - ##### data: Datum

    object to encode

  - ##### options: { labels: string[]; name: string; prefix: string }

    encoded object options

    - ##### labels: string[]

    - ##### name: string

    - ##### prefix: string

  #### Returns Datum

  one hot encoded object

### Static getBinaryValue

- getBinaryValue(value?: string | boolean): 1 | 0

- - Defined in src/DataSet.ts:449

  returns 0 or 1 depending on the input value

  example

    DataSet.getBinaryValue('true') // => 1 DataSet.getBinaryValue('false') // => 0 DataSet.getBinaryValue('No') // => 0 DataSet.getBinaryValue(false) // => 0

  #### Parameters

  - ##### value: string | boolean = ''

  #### Returns 1 | 0

  0 or 1 depending on truthiness of value

### Static getTransforms

- getTransforms(transforms?: DataSetTransform): FitColumnsOptions

- - Defined in src/DataSet.ts:102

  Allows for fit transform short hand notation

  example

    DataSet.getTransforms({ Age: ['scale',], Rating: ['label',], }); //=> [ // { // name: 'Age', options: { strategy: 'scale', }, }, // }, // { // name: 'Rating', options: { strategy: 'label', }, // }, // ];

  #### Parameters

  - ##### transforms: DataSetTransform = ...

  #### Returns FitColumnsOptions

  returns fit columns, columns property

### Static mapToObject

- mapToObject(mapObj?: Map<any, any>): any

- - Defined in src/DataSet.ts:426

  returns a JavaScript Object from a Map (supports nested Map Objects)

  example

    const csvObj = new DataSet([{col1:1,col2:5},{col1:2,col2:6}]); csvObj.columnMatrix([['col1',{parseInt:true}],['col2']]); // => //[ // [1,5], // [2,6], //]

  #### Parameters

  - ##### mapObj: Map<any, any> = ...

    Map to convert into JavaScript Object

  #### Returns any

  JavaScript Object converted from a Map

### Static oneHotColumnArray

- oneHotColumnArray(name: any, oneHotColumnArrayOptions: any): any

- - Defined in src/DataSet.ts:294

  #### Parameters

  - ##### name: any

  - ##### oneHotColumnArrayOptions: any

  #### Returns any

### Static oneHotDecoder

- oneHotDecoder(name: any, options: any): any

- - Defined in src/DataSet.ts:274

  Return one hot encoded data

  example

    const csvData = [{ 'Country': 'Brazil', 'Age': '44', 'Salary': '72000', 'Purchased': 'N', }, { 'Country': 'Mexico', 'Age': '27', 'Salary': '48000', 'Purchased': 'Yes', }, ... ]; const EncodedCSVDataSet = new ms.preprocessing.DataSet(csvData); EncodedCSVDataSet.fitColumns({ columns: [ { name: 'Country', options: { strategy: 'onehot', }, }, ], });

    EncodedCSVDataSet.oneHotDecoder('Country);// => // [ { Country: 'Brazil' }, // { Country: 'Mexico' }, // { Country: 'Ghana' }, // { Country: 'Mexico' }, // ...]

  #### Parameters

  - ##### name: any

    column name

  - ##### options: any

  #### Returns any

  returns an array of objects from an one hot encoded column

### Static oneHotEncoder

- oneHotEncoder(name: string, options: OneHotEncoderOptions): OneHotEncodedData

- - Defined in src/DataSet.ts:205

  returns a new object of one hot encoded values

  example

    // [ 'Brazil','Mexico','Ghana','Mexico','Ghana','Brazil','Mexico','Brazil','Ghana', 'Brazil' ] const originalCountry = dataset.columnArray('Country');

    // { originalCountry: // { Country_Brazil: [ 1, 0, 0, 0, 0, 1, 0, 1, 0, 1 ], // Country_Mexico: [ 0, 1, 0, 1, 0, 0, 1, 0, 0, 0 ], // Country_Ghana: [ 0, 0, 1, 0, 1, 0, 0, 0, 1, 0 ] }, // } const oneHotCountryColumn = dataset.oneHotEncoder('Country');

  see

    http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html

  #### Parameters

  - ##### name: string

    csv column header, or JSON object property name

  - ##### options: OneHotEncoderOptions

  #### Returns OneHotEncodedData

### Static reverseColumnMatrix

- reverseColumnMatrix(options?: ReverseColumnMatrixOptions): Data

- - Defined in src/DataSet.ts:139

  returns an array of objects by applying labels to matrix of columns

  example

    const data = [{ Age: '44', Salary: '44' }, { Age: '27', Salary: '27' }] const AgeDataSet = new MS.DataSet(data); const dependentVariables = [ [ 'Age', ], [ 'Salary', ], ]; const AgeSalMatrix = AgeDataSet.columnMatrix(dependentVariables); // => // [ [ '44', '72000' ], // [ '27', '48000' ] ]; MS.DataSet.reverseColumnMatrix({vectors:AgeSalMatrix,labels:dependentVariables}); // => [{ Age: '44', Salary: '44' }, { Age: '27', Salary: '27' }]

  #### Parameters

  - ##### options: ReverseColumnMatrixOptions = ...

  #### Returns Data

  an array of objects with properties derived from options.labels

### Static reverseColumnVector

- reverseColumnVector(options?: ReverseColumnVectorOptions): Data

- - Defined in src/DataSet.ts:152

  #### Parameters

  - ##### options: ReverseColumnVectorOptions = ...

  #### Returns Data

### Static selectColumns

- selectColumns(names: any[], options?: any): any

- - Defined in src/DataSet.ts:311

  returns a list of objects with only selected columns as properties

  example

    const data = [{ Age: '44', Salary: '44' , Height: '34' }, { Age: '27', Salary: '44' , Height: '50' }] const AgeDataSet = new MS.DataSet(data); const cols = [ 'Age', 'Salary' ]; const selectedCols = CSVDataSet.selectColumns(cols); // => [{ Age: '44', Salary: '44' }, { Age: '27', Salary: '27' }]

  #### Parameters

  - ##### names: any[]

    array of selected columns

  - ##### options: any = ...

  #### Returns any

  an array of objects with properties derived from names