@push.rocks/smartdata: A TypeScript-First MongoDB Wrapper for Modern Applications

Philipp Kunz

Philipp Kunz

— 4 min read

In the world of TypeScript development, finding the right database tools that combine type safety with powerful features can be challenging. As applications grow in complexity, particularly in distributed environments, the need for robust data management becomes ever more critical. This is where @push.rocks/smartdata enters the picture - a TypeScript-first MongoDB wrapper designed to simplify complex data operations while providing powerful features for modern application development.

What is @push.rocks/smartdata?

@push.rocks/smartdata is a comprehensive MongoDB wrapper that provides strong TypeScript integration, offering type-safe operations, automated schema handling, and advanced features for distributed systems. Created by Philipp Kunz, this library aims to simplify database interactions while adding powerful features that extend MongoDB's capabilities.

Key Features That Set It Apart

What makes @push.rocks/smartdata stand out from other MongoDB wrappers? Let's explore its most distinctive features:

1. True TypeScript-First Approach

Unlike many libraries that add TypeScript as an afterthought, @push.rocks/smartdata is built from the ground up with TypeScript in mind. This provides several advantages:

  • Decorator-Based Schema Definition: Define your MongoDB schemas using TypeScript decorators
  • Type-Safe CRUD Operations: Eliminate runtime errors with compile-time type checking
  • Deep Query Type Safety: Fully type-safe queries for nested object properties via DeepQuery<T>

2. Advanced Search Capabilities

The library implements a powerful and intuitive search system that goes beyond basic queries:

// Define a model with searchable fields
@Collection(() => db)
class Product extends SmartDataDbDoc<Product, Product> {
  @unI() public id: string = 'product-id';
  @svDb() @searchable() public name: string;
  @svDb() @searchable() public description: string;
  @svDb() @searchable() public category: string;
  @svDb() public price: number;
}

// Advanced search options
await Product.search('"Kindle Paperwhite"'); // Exact phrase across all fields
await Product.search('Air*'); // Wildcard search
await Product.search('name:Air*'); // Field-scoped wildcard
await Product.search('category:Electronics AND name:iPhone'); // Boolean operators
await Product.search('(Furniture OR Electronics) AND Chair'); // Grouping

This search functionality supports Lucene-style queries with exact matches, wildcards, field-scoping, boolean operators, and more - all while maintaining type safety.

3. EasyStore for Simple Key-Value Storage

For simpler data needs, @push.rocks/smartdata offers EasyStore - a type-safe key-value storage system with automatic persistence:

interface ConfigStore {
  apiKey: string;
  settings: {
    theme: string;
    notifications: boolean;
  };
}

// Create a type-safe EasyStore
const store = await db.createEasyStore<ConfigStore>('app-config');

// Write and read with full type safety
await store.writeKey('apiKey', 'secret-api-key-123');
await store.writeKey('settings', { theme: 'dark', notifications: true });

const apiKey = await store.readKey('apiKey'); // Type: string
const settings = await store.readKey('settings'); // Type: { theme: string, notifications: boolean }

4. Built-in Distributed Coordination

One of the most powerful features is the built-in support for distributed systems:

// Create a distributed coordinator
const coordinator = new SmartdataDistributedCoordinator(db);

// Start coordination
await coordinator.start();

// Handle leadership changes
coordinator.on('leadershipChange', (isLeader) => {
  if (isLeader) {
    // This instance is now the leader
    startPeriodicJobs();
  } else {
    // This instance is no longer the leader
    stopPeriodicJobs();
  }
});

// Execute tasks only on the leader
await coordinator.executeIfLeader(async () => {
  // This code only runs on the leader instance
  await runImportantTask();
});

This makes it significantly easier to build reliable distributed applications with leader election, task coordination, and more.

5. Real-Time Data Synchronization

For applications requiring real-time updates, @push.rocks/smartdata provides watchers with RxJS integration:

// Create a watcher for active users
const watcher = await User.watch(
  { active: true },
  { fullDocument: true, bufferTimeMs: 100 }
);

// Subscribe to changes using RxJS
watcher.changeSubject.subscribe((change) => {
  console.log('Change operation:', change.operationType); 
  console.log('Document changed:', change.docInstance);
  
  // Handle different operations
  if (change.operationType === 'insert') {
    notifyNewUser(change.docInstance);
  }
});

This simplifies building reactive applications that respond to data changes in real-time.

Practical Implementation

Let's walk through a practical implementation to see how @push.rocks/smartdata can be used in a real-world scenario.

Setting Up Your Database Connection

First, establish a connection to your MongoDB database:

import { SmartdataDb } from '@push.rocks/smartdata';

// Create a database instance
const db = new SmartdataDb({
  mongoDbUrl: 'mongodb://localhost:27017/myapp',
  mongoDbName: 'myapp',
  mongoDbUser: 'username',
  mongoDbPass: 'password',
});

// Initialize and connect
await db.init();

Defining Your Data Models

Next, define your data models using TypeScript classes with decorators:

import {
  SmartDataDbDoc,
  Collection,
  unI,
  svDb,
  index,
  searchable,
} from '@push.rocks/smartdata';
import { ObjectId } from 'mongodb';

@Collection(() => db)
class User extends SmartDataDbDoc<User, User> {
  @unI()
  public id: string = 'user-' + Math.random().toString(36).substring(2, 9);

  @svDb()
  @searchable()
  public username: string;

  @svDb()
  @searchable()
  @index()
  public email: string;

  @svDb()
  public organizationId: ObjectId; // Stored as BSON ObjectId

  @svDb()
  public profilePicture: Buffer; // Stored as BSON Binary

  @svDb({
    serialize: (data) => JSON.stringify(data),
    deserialize: (data) => JSON.parse(data),
  })
  public preferences: Record<string, any>;

  constructor(username: string, email: string) {
    super();
    this.username = username;
    this.email = email;
  }
}

Performing CRUD Operations

With your models defined, you can now perform fully type-safe CRUD operations:

// Create a user
const user = new User('johndoe', '[email protected]');
user.preferences = { theme: 'dark', notifications: true };
await user.save();

// Retrieve users
const singleUser = await User.getInstance({ username: 'johndoe' });
const users = await User.getInstances({ email: /example\.com$/ });

// Update a user
singleUser.email = '[email protected]';
await singleUser.save();

// Upsert operation
const upsertedUser = await User.upsert(
  { username: 'janedoe' },
  { email: '[email protected]', preferences: { theme: 'light' } }
);

// Delete a user
await singleUser.delete();

Using Advanced Features

For more complex scenarios, leverage the advanced features:

// Using transactions for atomic operations
const session = db.startSession();
try {
  await session.withTransaction(async () => {
    const sender = await User.getInstance({ id: 'user1' }, session);
    const recipient = await User.getInstance({ id: 'user2' }, session);
    
    // Transfer credits atomically
    sender.credits -= 100;
    recipient.credits += 100;
    
    await sender.save({ session });
    await recipient.save({ session });
  });
} finally {
  await session.endSession();
}

// Implementing document lifecycle hooks
@Collection(() => db)
class Order extends SmartDataDbDoc<Order, Order> {
  @unI()
  public id: string = 'order-' + Date.now();

  @svDb()
  public items: string[];

  @svDb()
  public total: number = 0;

  // Hook called before saving
  async beforeSave() {
    // Calculate total from items
    this.total = await calculateItemsTotal(this.items);
  }

  // Hook called after saving
  async afterSave() {
    // Notify inventory system
    await notifyInventorySystem(this);
  }
}

Best Practices and Performance Optimization

When working with @push.rocks/smartdata, consider these best practices:

  1. Proper Indexing: Use the @unI(), @index(), and collection-level indexing to ensure optimal query performance.

  2. Cursor Management: For large datasets, use cursors with manual iteration:

    const cursor = await User.getCursor({ active: true });
try {
  await cursor.forEach(user => {
    // Process each user without loading all into memory
  });
} finally {
  await cursor.close(); // Always close cursors
}

  3. Search Optimization: Mark only necessary fields with @searchable() and use field-specific searches when possible.

  4. Transaction Usage: Prefer transactions for operations that must succeed or fail together.

  5. Connection Management: Always properly initialize and close database connections.

The Gist

@push.rocks/smartdata offers a compelling solution for TypeScript developers working with MongoDB. By combining strong type safety with powerful features like distributed coordination, advanced search capabilities, and real-time data synchronization, it addresses many of the common challenges in modern application development.

Whether you're building a simple application or a complex distributed system, this library provides the tools needed to efficiently manage your data while maintaining type safety throughout your codebase.

For more information, visit the Git repository or install the package via npm:

npm install @push.rocks/smartdata --save

Read more