Pools

Lightweight TypeScript library for managing data collections with filters, sorting, and composition

What is Pools?

Pools is a modern TypeScript library that replaces arrays, objects, and Maps with a powerful abstraction for working with collections. Think of it as a smart wrapper around your data that gives you filtering, sorting, metadata tracking, and pool composition out of the box.

Not a replacement for:

ORMs or databases
Processing millions of records
Enterprise frameworks

Perfect for:

Daily work with collections in memory
Managing pools of resources (proxies, accounts, sessions)
Quick prototyping with typed data
Building tools that need smart data selection

Installation

bun install

Quick Start

import { Pool, Selectors } from './src';

interface Proxy {
	ip: string;
	country: string;
	speed: number;
}

const proxies = new Pool<Proxy>();

// Add data with metadata
proxies.add({ ip: '1.1.1.1', country: 'US', speed: 100 }, { usedCount: 0, active: true });

// Query with filters and sorting
const bestProxy = proxies
	.query()
	.where((e) => e.data.country === 'US')
	.where((e) => e.meta.active === true)
	.orderBy('speed', 'desc')
	.select(Selectors.first);

console.log(bestProxy); // { ip: '1.1.1.1', country: 'US', speed: 100 }

// Pool of pools - store pools as data
const usProxies = new Pool<Proxy>();
const ukProxies = new Pool<Proxy>();
const poolOfPools = new Pool<Pool<Proxy>>();
poolOfPools.add(usProxies);
poolOfPools.add(ukProxies);

Core Concepts

PoolEntry

Every item in a pool is wrapped in a PoolEntry:

type PoolEntry<T> = {
	data: T; // Your data
	meta: Record<string, any>; // Metadata (usage stats, flags, etc.)
};

Filter

A function that decides whether to include an entry:

type Filter<T> = (entry: PoolEntry<T>) => boolean;

// Example
const usFilter = (e) => e.data.country === 'US';

Selector

A function that picks one entry from filtered results:

type Selector<T> = (entries: PoolEntry<T>[]) => PoolEntry<T> | null;

// Built-in selectors
Selectors.first; // First entry
Selectors.last; // Last entry
Selectors.random; // Random entry
Selectors.minBy('field'); // Entry with minimum value
Selectors.weighted(fn); // Weighted random

API Reference

Pool

Creating Pools

// Empty pool
const pool = new Pool<Proxy>();

// Pool of pools - pools as data
const usProxies = new Pool<Proxy>();
const ukProxies = new Pool<Proxy>();
const poolOfPools = new Pool<Pool<Proxy>>();
poolOfPools.add(usProxies);
poolOfPools.add(ukProxies);

// Query pools from pool of pools
const bestPool = poolOfPools
	.query()
	.where((e) => e.data.size > 10)
	.select(Selectors.first);

CRUD Operations

// Add single entry
pool.add({ ip: '1.1.1.1', country: 'US', speed: 100 }, { usedCount: 0 });

// Add batch
pool.addBatch([
	{ data: { ip: '2.2.2.2', country: 'UK', speed: 200 }, meta: {} },
	{ data: { ip: '3.3.3.3', country: 'DE', speed: 150 }, meta: {} },
]);

// Remove entries
pool.remove((data) => data.country === 'UK');

// Remove batch
pool.removeBatch([(data) => data.country === 'UK', (data) => data.speed < 100]);

Map-like Operations

// Get by field (like Map.get)
const proxy = pool.get('ip', '1.1.1.1');

// Get by predicate
const fastProxy = pool.get((e) => e.data.speed > 200);

// Check existence (like Map.has)
const exists = pool.has('ip', '1.1.1.1');
const hasFast = pool.has((e) => e.data.speed > 200);

// Set (update or add)
pool.set('ip', '1.1.1.1', { ip: '1.1.1.1', country: 'US', speed: 150 });

// Delete (like Map.delete)
const deleted = pool.delete('ip', '1.1.1.1'); // returns boolean

Query API

// Build queries with chaining
const result = pool
	.query()
	.where((e) => e.data.country === 'US')
	.where((e) => e.meta.active === true)
	.orderBy('speed', 'desc')
	.orderByMeta('usedCount', 'asc')
	.limit(10)
	.toArray();

// Select single entry
const proxy = pool
	.query()
	.where((e) => e.data.country === 'US')
	.select(Selectors.random);

// Convert query to pool
const usProxies = pool
	.query()
	.where((e) => e.data.country === 'US')
	.toPool();

Events

// Auto-update metadata on use
pool.on('get', (entry) => {
	entry.meta.usedCount++;
	entry.meta.lastUsed = new Date();
});

// Logging
pool.on('add', (entry) => {
	console.log(`Added: ${entry.data.ip}`);
});

// Available events:
// 'add', 'remove', 'get', 'set'
// 'batchAdd', 'batchRemove'
// 'beforeSelect', 'afterSelect'

Combining Pools

const pool1 = new Pool<Proxy>();
const pool2 = new Pool<Proxy>();

// Merge all
pool1.merge(pool2);

// Merge unique by field
pool1.mergeUnique(pool2, 'ip');

// Merge unique by function
pool1.mergeUnique(pool2, (p) => `${p.ip}:${p.provider}`);

// Union (no duplicates)
pool1.union(pool2, (a, b) => a.ip === b.ip);

// Intersect (only common)
pool1.intersect(pool2, (a, b) => a.ip === b.ip);

// Difference (remove items from other)
pool1.difference(pool2, (a, b) => a.ip === b.ip);

// Remove duplicates
pool.deduplicate('ip');

Transformations

// Clone
const backup = pool.clone();

// Partition into two pools
const [active, inactive] = pool.partition((e) => e.meta.active === true);

// Random sample
const sample = pool.sample(10);

// Shuffle in place
pool.shuffle();

// Group by field
const byCountry = pool.groupBy('country');
// Map<string, Pool<Proxy>>

// Group by function
const bySpeed = pool.groupBy((p) => (p.speed > 200 ? 'fast' : 'slow'));

Static Methods

// Merge multiple pools
const merged = Pool.merge(pool1, pool2, pool3);

// Merge unique
const unique = Pool.mergeUnique([pool1, pool2], 'ip');

// Merge with conflict resolution
const best = Pool.mergeUniqueWith([pool1, pool2], 'ip', (existing, duplicate) => (existing.data.speed > duplicate.data.speed ? existing : duplicate));

// Intersect two pools
const common = Pool.intersect(pool1, pool2, (a, b) => a.provider === b.provider);

// Group multiple sources
const groups = Pool.groupBy([pool1, pool2, pool3], 'country');

Properties

pool.size; // Number of entries
pool.all; // Array of data objects
pool.allEntries; // Array of PoolEntry objects

Method Wrapping

// Wrap methods to add custom behavior
pool.wrap('add', (original, data, meta) => {
	console.log('Before add');
	const result = original(data, meta);
	console.log('After add');
	return result;
});

// Example: Database sync
pool.wrap('add', async (original, data, meta) => {
	const entry = original(data, meta);
	await db.insert('proxies', { data, meta });
	return entry;
});

Query

const query = pool.query();

// Filtering
query.where((e) => e.data.country === 'US');
query.whereOr([(e) => e.data.provider === 'A', (e) => e.data.provider === 'B']);

// Sorting (chainable)
query.orderBy('speed', 'desc');
query.orderBy((a, b) => a.data.speed - b.data.speed);
query.orderByMeta('usedCount', 'asc');

// Pagination
query.offset(20).limit(10);

// Materialization
query.select(Selectors.first); // Single entry
query.toArray(); // Array of data
query.toPool(); // Convert query to Pool
query.count; // Number of entries

Binder

Bind multiple pools together for complex selections:

const combo = new Binder()
	.bind('proxy', proxies)
	.bind('account', accounts)
	.bind('service', services)
	.where('proxy', (e) => e.data.country === 'US')
	.where('account', (e) => e.data.service === 'twitter')
	.selectWith('proxy', Selectors.minBy('usedCount'))
	.selectWith('account', Selectors.random)
	.execute();

// Result:
// {
//   proxy: { ip: '1.1.1.1', ... },
//   account: { username: 'user1', ... },
//   service: { name: 'API', ... }
// }

// Returns null if any pool has no matching entry

Built-in Selectors

import { Selectors } from './src';

// Random entry
pool.query().select(Selectors.random);

// First entry
pool.query().select(Selectors.first);

// Last entry
pool.query().select(Selectors.last);

// Minimum by field (checks both data and meta)
pool.query().select(Selectors.minBy('usedCount'));

// Weighted random (higher weight = higher probability)
pool.query().select(Selectors.weighted((entry) => 1 / (entry.meta.usedCount + 1)));

Examples

Basic Usage

See examples/basic.ts:

bun run examples/basic.ts

Proxy Pool Management

See examples/proxy-pool.ts:

bun run examples/proxy-pool.ts

This example demonstrates:

CRUD operations
Event handling
Complex queries with multiple filters
Pool combination
Transformations
Binder usage
Weighted selectors

Map-like Usage

See examples/map-like.ts:

bun run examples/map-like.ts

This example demonstrates:

Using Pool as a Map replacement
get/has/set/delete operations
Pool of pools with identifiers
Cache implementation
Config management

Game Service

See examples/game-service.ts:

bun run examples/game-service.ts

Comprehensive example demonstrating ALL library features:

Multiple interconnected pools (games, accounts, servers, sessions)
Complex matchmaking logic
Event-driven architecture
Pool of pools for regional organization
Weighted server selection
Statistics and analytics

Use Cases

Managing Proxies

const proxies = new Pool<Proxy>();

// Auto-track usage
proxies.on('get', (entry) => {
	entry.meta.usedCount++;
	entry.meta.lastUsed = new Date();
});

// Get least-used US proxy
const proxy = proxies
	.query()
	.where((e) => e.data.country === 'US')
	.where((e) => e.meta.active)
	.orderByMeta('usedCount', 'asc')
	.select(Selectors.first);

Session Management

const sessions = new Pool<Session>();

// Auto-expire old sessions
sessions.on('get', (entry) => {
	if (Date.now() - entry.meta.lastUsed > 3600000) {
		entry.meta.expired = true;
	}
});

// Get valid session
const session = sessions
	.query()
	.where((e) => !e.meta.expired)
	.select(Selectors.random);

Resource Allocation

// Bind proxy + account + service
const resources = new Binder()
	.bind('proxy', proxies)
	.bind('account', accounts)
	.bind('service', services)
	.where('proxy', (e) => e.meta.usedCount < 10)
	.where('account', (e) => !e.meta.banned)
	.selectWith('proxy', Selectors.minBy('usedCount'))
	.selectWith('account', Selectors.random)
	.execute();

if (resources) {
	await doTask(resources.proxy, resources.account, resources.service);
}

TypeScript Support

Full TypeScript support with generic types:

interface MyData {
	id: number;
	name: string;
}

const pool = new Pool<MyData>();

// TypeScript knows the type
const result = pool.query().select(Selectors.first);
// result: MyData | null

// Autocomplete works
result?.name; // ✓
result?.invalid; // ✗ TypeScript error

Performance

Pools is designed for collections of hundreds to thousands of items. For very large datasets (100k+ items), consider:

Using pagination with offset() and limit()
Filtering early to reduce the working set
Using query.toPool() to cache filtered results

Project Structure

pools/
├── src/
│   ├── types.ts       # Core types
│   ├── Pool.ts        # Main Pool class
│   ├── Query.ts   # Query builder
│   ├── Binder.ts  # Multi-pool binding
│   ├── Selectors.ts   # Built-in selectors
│   └── index.ts       # Public API
├── examples/
│   ├── basic.ts       # Basic usage
│   └── proxy-pool.ts  # Advanced example
└── README.md

License

MIT

Contributing

Contributions welcome! This is a learning project but pull requests and issues are appreciated.

Name		Name	Last commit message	Last commit date
Latest commit History 26 Commits
.claude		.claude
.github/workflows		.github/workflows
docs		docs
examples		examples
src		src
test		test
.gitignore		.gitignore
CLAUDE_PLAN.MD		CLAUDE_PLAN.MD
DEPLOY_DOCS.md		DEPLOY_DOCS.md
README.md		README.md
bun.lock		bun.lock
package.json		package.json
tsconfig.json		tsconfig.json

phederal/pools

Folders and files

Latest commit

History

Repository files navigation