Greg Rundlett

Are you ready to say "Hasta la vista" to outdated search methods? Elastic is releasing its first live coding session: "How to Build Hybrid Search with Elasticsearch." What’s in it for you? ✅ Learn how to combine full-text search and semantic search into a powerful hybrid search solution ✅ Discover how retrievers like Reciprocal Rank Fusion (RRF) and Linear retrievers can fine-tune your search results ✅ Watch a live demo featuring a movie search app inspired by sci-fi classics Whether you're a developer, data scientist, or just passionate about AI-driven search technology, this session will equip you with the tools to build smarter, faster, and more relevant search engines. 📅 Date: July 21 📍 Location: YouTube 👉 Register here: https://gag.gl/LqiFi0

39

New blog post: A Fork in the Road: Deciding Kafka's Diskless Future. Kafka is getting serious about S3 and finds itself at an architectural crossroads that will shape its next decade. Several new KIPs (1150, 1176, 1183) aim to reduce replication costs across cloud availability zones, but the implications go far beyond networking cost. It’s a mistake to think of S3 as simply a cheaper disk or a networking cheat. Building on object storage opens the door to operational benefits such as elastic, stateless compute — something many modern analytics systems already exploit. In my latest post, I outline two competing future paths: 🔹 Evolutionary path: Reuse large parts of existing Kafka components to reduce code changes and long-term maintenance. 🔹 Revolutionary path: Separate stateless and stateful layers to realize the full operational benefits of disaggregated storage. The post examines the trade-offs of each path, how the current KIPs map to these two paths and poses a broader question: what should Kafka become? And what will keep it relevant in the decade ahead? https://lnkd.in/d4E2BAe4

149

12 Comments

Thoughts on DuckDB's DuckLake so far: 1. As someone's who is familiar with the backend implementations of Delta Lake and Iceberg, the simplicity of DuckLake is admirable. 2. Iceberg table import/export is coming. Compaction support seems to be in the extension. 3. I would love to see integrations for DuckLake with AWS Athena (Trino/Presto) and Spark. Implementors, just vendor the DuckDB implementation and call into it via JNI. 4. The use of table level statistics is fantastic, join planning will perform well. 5. Adding support for data sketches (bloom filters) on columns seems possible too! #duckdb #deltalake #iceberg #apacheiceberg #ducklake #lakehouse #formatwars #lakehouseformatwars #hudi #parquet

118

9 Comments

Pro tip for open source: There are a few OSS projects I've leveraged in the enterprise that were maintained by a single person. Lodash is one. Mountebank another. They always made me nervous but the way they communicated with users and the care they took about docs (popularity isn't enough for me) gave me enough confidence to use them. I was recently starting to evaluate another open source project that's been getting traction recently. I asked Claude to give some feedback on it. - Alpha release with an unstable API. Fair enough, something to test out. It's very popular, so that will improve - Single maintainer Not awesome, but not a deal killer - Docs and implementation do not match Hmmmm - 300+ open issues Hmmm Then I talked to someone who had ask the maintainer to do some consulting - Self aggrandizing - Hard to tell if he's really smart or a confident BSer (Elon Musk Syndrome) - Makes wild claims about accomplishments and people stealing ideas Yeah, I'm out. Open source is a mixed bag. You need to understand the risk profile of your supply chain. I'm looking for a core capability for an enterprise, not a toy, and this project is a pass. No, I won't say which one in public for obvious legal reasons.

25

11 Comments

I think the Kubernetes community we need to move away from label selector-based component discovery. It's not working and it's highly error-prone plus hard to debug--no controller, nothing is telling you there's a label mismatch, not even from lifecycle events emitted. I have been in hours-long debugging sessions on this before. Our Spring Boot based microservices on an on-prem cluster with traffic routed through a Cilium-based Gateway API. We discovered some services work fine and others don't. Why? A label selector mismatch had crept in somewhere. We tried all sorts of things before I suggested we do service port-forwarding (not so many people know about this because they do pod port-forwarding all the time). Essentially, port-forward the service to figure out if it can reach the pod. Gateway was able to reach the service but service wasn't able to reach the pod. And when you port-forward the pod, service is running fine. I had been in a similar boat for a few days now, parked the work because too many layers were involved, until I spent some time to dissect the layers and isolate the problem to realise one of the issues—components weren't getting discovered—was a capitalised initial name where I used lowercase. Shame. We really need to look forward to better solutions around autodiscovery of components, not relying on a static, error-prone label approach. #HPC #GPU #NVIDIA #InfiniBand #RDMA #Kubernetes #distributedcomputing #MLOps #AIInfrastructure #deeplearning #llm 

40

10 Comments

Why do so many mission-critical workloads still land in AWS us-east-1—even after years of well-documented outages? Because data has gravity. Architectures have inertia. And moving bits? That’s expensive—in cost, complexity, and culture. We love to talk about resilience like it’s just a checkbox or a config tweak. But true resilience? That has real architectural and financial cost. Here’s the part that often gets overlooked: The architects behind these decisions aren’t clueless. They understood the risks. They likely flagged them. They probably ran a risk-cost analysis. The result? A few hours of Venmo downtime vs. the ongoing cost of chasing another ‘9’ of uptime across multiple regions. That’s not negligence. That’s business tradeoff. It’s the math of availability, margins, and user tolerance. So before we judge those still anchored to us-east-1, remember: Sometimes “bad” architecture is just deliberate architecture under constraint.

41

9 Comments