Skip to content

Data migration job handler to migrate dynamodb tables to mongodb db collections

License

Notifications You must be signed in to change notification settings

igroene/dynamodb-mongodb-migrate

 
 

Repository files navigation

DynamoDB MongoDB Migrate

npm version Build Status

Data migration module for migrating dynamodb tables to mongodb db collections

Installing

npm install dynamodb-mongodb-migrate

Quick Usage

const MigrationJob = require('dynamodb-mongodb-migrate');

let sourceConnectionOptions = {
    region: <AWS_REGION>,
    accessKeyId: <AWS_ACCESS_KEY_ID>,
    secretAccessKey: <AWS_SECRET_ACCESS_KEY>
};
let targetConnectionOptions = {
    host: <MONGODB_ENDPOINT>,
    user: <MONGODB_USERNAME>,
    password: <MONGODB_PASSWORD>
};

const migrationJob = new MigrationJob('DYNAMODB_TABLE_NAME', 'MONGODB_COLLECTION_NAME', 'MONGODB_DATABASE_NAME', sourceConnectionOptions, targetConnectionOptions, DYNAMODB_SCAN_LIMIT, DYNAMODB_READ_THROUGHPUT);

migrationJob.run()

Adavance Usage

Initialize

const MigrationJob = require('dynamodb-mongodb-migrate');

let sourceConnectionOptions = {
    region: <AWS_REGION>,
    accessKeyId: <AWS_ACCESS_KEY_ID>,
    secretAccessKey: <AWS_SECRET_ACCESS_KEY>
};
let targetConnectionOptions = {
    host: <MONGODB_ENDPOINT>,
    user: <MONGODB_USERNAME>,
    password: <MONGODB_PASSWORD>
};

const migrationJob = new MigrationJob('DYNAMODB_TABLE_NAME', 'MONGODB_COLLECTION_NAME', 'MONGODB_DATABASE_NAME', sourceConnectionOptions, targetConnectionOptions, DYNAMODB_SCAN_LIMIT, DYNAMODB_READ_THROUGHPUT);

Set dynamodb filter expression - filter when scanning dynamodb

const filterExpression = '#attr1 = :val1';
const expressionAttributeNames = {
    '#attr1':'attribute1'
};
const expressionAttributeValues = {
    ':val1':'value1'
}
migrationJob.setSourcefilterExpression(filterExpression, expressionAttributeNames, expressionAttributeValues);

Set data filter function - filter after scan result - similar to lodash filter

const filterFunction = (item) =>{
    return item.attr1 !== null;
}

migrationJob.setFilterFunction(metadata.filterFunction);

Set data mapper function - similar to lodash map

const mapperFunction = (item) =>{
    return {
        mappedAttr1 : item.attr1,
        mappedAttr2 : item.attr2
    }
}

migrationJob.setMapperFunction(mapperFunction);

Run

migrationJob.run()

Testing

npm test

About

Data migration job handler to migrate dynamodb tables to mongodb db collections

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • JavaScript 97.6%
  • Shell 1.3%
  • Dockerfile 1.1%