SmartPipe.Core v1.0.4 — 22 New Features, 12% Faster

SmartPipe.Core v1.0.4 — 22 New Features, 12% Faster

Performance

• ValueTask_Transform 12% faster (69.12 ns vs 78.81 ns)
• PredictNextLatency: 0.16 ns (practically free)
• 0 regressions across all benchmarks

New in Core

• P-Controller Parallelism — smooth thread scaling with dead zone and anti-windup
• Double EMA + Prediction — velocity tracking + one-step latency forecast
• Hybrid CircuitBreaker — EWMA early warning + Sliding window decisions
• P-Controller Backpressure — continuous throttling, no oscillation
• PipelineState + Cancel() — lifecycle management with events
• Progress reporting — OnProgress with ETA
• Auto DeadLetter routing — exhausted retries → DeadLetterSink
• Metrics.Export() — JSON + Prometheus format
• 4 new OWASP patterns — JWT, AWS Key, GitHub Token, OAuth Token

New in Extensions (13 components)

• File sources: CsvFileSource, JsonFileSource, DeadLetterSource
• File sinks: CsvFileSink, JsonFileSink
• Transforms: FilterTransform (And/Or/Not), ValidationTransform (DataAnnotations), ConditionalTransform, CompositeTransform
• Sinks: DbSink, HttpSink

Quality

• 243 tests (+28 from v1.0.3)
• 96.4% line coverage
• Algorithm benchmarks for all new controllers

Links

• Complete Feature Reference
• NuGet: SmartPipe.Core
• NuGet: SmartPipe.Extensions

SmartPipe.Core v1.0.3 — 13 New features

SmartPipe.Core v1.0.3 — 13 New Features

New Features

• Middleware Transformer — Func<T,T> as lightweight ITransformer
• Rendezvous Channel — strict Producer-Consumer sync (BoundedCapacity=0)
• HyperLogLogEstimator — Count-Distinct with O(1) memory
• Dual-threshold Watermark — Pause/Resume prevents oscillation
• Liveness/Readiness Health Checks — Kubernetes-native
• DeadLetterSink — failed items persistence
• Data Lineage — provenance tracking in Metadata
• ChannelMerge — merge two ChannelReader streams
• RunInBackground() — non-blocking pipeline execution
• Hybrid Queue — FullMode option (Wait/DropOldest/DropNewest)
• AsChannelReader() — SignalR/gRPC integration

Quality

• 215 tests (+29 from v1.0.2)
• 96.4% line coverage
• 0 regressions in all benchmarks

Links

• Complete Feature Reference
• NuGet: SmartPipe.Core
• NuGet: SmartPipe.Extensions

SmartPipe.Core v1.0.2 — Lock-Free Performance

SmartPipe.Core v1.0.2 — Lock-Free Performance

Performance Improvements

• Lock-free CircuitBreaker — lock() → Interlocked.CompareExchange + ConcurrentQueue
  • AllowRequest(): 49.30ns → 27.76ns
• Lock-free RetryQueue — Task.Delay(50) polling → WaitToReadAsync + timeout
  • EnqueueAsync(): 86.58ns → 69.16ns

New Features

• SmartPipeEventSource — monitor via dotnet-counters (EventCounters)
• SmartPipeHostedService — native BackgroundService for ASP.NET Core
• SmartPipeHealthCheck — pipeline health check for Kubernetes/YARP
• AddSmartPipeResilience() — DI extension for IServiceCollection

Mathematical Improvements

• Adaptive EMA — dynamic α (0.2 stable, 0.8 spike)
• Dynamic Watermark — throughput-based backpressure thresholds
• TryRead in DrainAsync — instant drain without delays

Code Quality

• 186 tests (+47 from v1.0.1)
• 96.3% line coverage (+9.8%)
• 81.2% branch coverage (+11.7%)
• Refactored ConsumeAsync — 6 smaller methods
• 0 regressions in all benchmarks

Breaking Changes

• ProcessingContext<T> — record class → mutable class with Reset() method

SmartPipe.Core v1.0.1 — First public pre-release

🚀 SmartPipe.Core v1.0.1 — First Public pre-release

Universal streaming pipeline engine for .NET 10 with zero dependencies and 0 allocations in hot path.

✨ What's Included

Core Engine (31 files, 21 resilience features):

• SmartPipeChannel with System.Threading.Channels
• AdaptiveParallelism (Little's Law), AdaptiveMetrics (EMA), BackpressureStrategy
• DeduplicationFilter (Bloom, O(1)), CuckooFilter (with deletion)
• CircuitBreaker (sliding window, HalfOpen limits, manual Isolate/Reset)
• RetryQueue with jitter (thundering herd protection)
• RetryPolicy (Fixed, Linear, Exponential backoff)
• OpenTelemetry Tracing + Metrics
• Graceful Shutdown (DrainAsync, Pause/Resume)
• SecretScanner (OWASP), ObjectPool, JumpHash, ReservoirSampler

Extensions (9 selectors/transforms/sinks):

• HttpSelector, EfCoreSelector, DapperSelector
• JsonTransform, CsvTransform, MapsterTransform, CompressionTransform
• PollyResilienceTransform, LoggerSink

Testing:

• 147 tests (137 unit + 8 property-based + 7 chaos engineering)
• 5 BenchmarkDotNet benchmarks (0 allocations verified)

📦 NuGet Packages

Package	Version	Downloads
SmartPipe.Core	1.0.1	Zero dependencies
SmartPipe.Extensions	1.0.0	Polly, EF Core, Dapper, Mapster

🎯 Quick Start

dotnet add package SmartPipe.Core
dotnet add package SmartPipe.Extensions

⚡ First Pipeline (5 lines)

using SmartPipe.Core;
using SmartPipe.Extensions.Selectors;
using SmartPipe.Extensions.Transforms;
using SmartPipe.Extensions.Sinks;

var pipeline = PipelineBuilder
    .From(new HttpSelector<MyDto>("https://api.example.com/data"))
    .Transform(new JsonTransform<MyDto, MyEntity>())
    .WithOptions(o => o.MaxDegreeOfParallelism = 4);
await pipeline.To(new LoggerSink<MyEntity>(logger));

🔗 Links

• GitHub Repository
• NuGet: SmartPipe.Core
• NuGet: SmartPipe.Extensions

🤝 Contributing

SmartPipe is an open-source project. Contributions are welcome!

• Report bugs via GitHub Issues
• Read CONTRIBUTING.md for guidelines
• Security vulnerabilities: use Private Vulnerability Reporting

⭐ Star the repo if you find it useful!