Announcing bring your own bucket for bitdrift Capture

At bitdrift, we have a very different take on observability: on-device intelligence. Instead of sending loads of expensive telemetry data only to later sift through it for a few precious insights, we couple a sophisticated device SDK, local storage, and real-time control via our control plane SaaS, in order to dynamically fetch only the data that is needed in order to understand customer behavior and solve problems quickly. We give you 1000x the observability at 0.01x the cost. bitdrift’s architecture means that we are uniquely positioned to grow with you to unlimited scale: from startup all the way to the Fortune 50. Modern enterprises are increasingly having to navigate a complex set of compliance challenges around data ownership and control, especially when utilizing both public cloud and additional SaaS vendors. While modern SaaS services undoubtedly provide tremendous enterprise value, storing sensitive data outside the corporate perimeter can be a tough sell for security teams. At the same time, control planes like bitdrift are sufficiently sophisticated that asking large enterprises to run it themselves “on-prem” is also a tough sell. What to do?

Rise of the blob store architecture

In recent years, almost every modern database is being built with what might be called a “blob store first architecture.” In this architecture, the only cold storage location for data is a blob storage system such as AWS S3. Database nodes may use local storage and RAM for caching, but no data is permanently kept there. This architecture is appealing as it allows for effectively unlimited storage size, relying on the data replication, redundancies, and encryption at rest provided by the blob storage system, greatly simplifying many aspects of database administration and operation. Not surprisingly, bitdrift’s underlying storage architecture is also built on top of blob storage. The above diagram is a simplified version of the bitdrift SaaS.

1. Clients maintain a real-time connection to our SaaS that is used to both send telemetry and receive command and control instructions.
2. Various metadata databases are used for indexes and storing summary statistics. These indexes contain no logs, session replays, or anything else that might possibly include PII.
3. Logging data is written directly to cold storage in a highly efficient packed and compressed format.
4. When a session is viewed in our UI, it is “hydrated” from cold storage, which decompresses and unpacks the relevant logging data for the session, and repacks it in a session specific format that is easier to search and paginate, even if the session contains hundreds of thousands of logs. This data is directly consumed via our SaaS for display in our UI.

The key takeaway is that the only storage at rest of data potentially containing PII is in blob storage.

BYOB

In the BYOB architecture, the enterprise customer provides the blob storage while bitdrift continues to independently run the rest of the control plane as a SaaS. This has the following benefits:

1. The customer controls IAM access to the log data directly.
2. The customer controls versioning and backup policies for the log data.
3. The customer controls any other relevant compliance policies for the log data such as scanning, data locality, etc.
4. The customer does not have to run the rest of the SaaS at large operational cost.

One potential downside of the BYOB architecture, especially when spanning multiple public cloud vendors, can be an increase in data transfer fees. Bitdrift’s “store nothing by default” architecture is uniquely unaffected by this, as the actual data written and read to cold storage is typically a tiny fraction of the overall data that might be produced by an app, even for deployments with hundreds of millions of active users. Try that with any other observability provider! In our view this is the best of both worlds: enterprise customers obtain strict control over their sensitive data while bitdrift runs the rest of the system as a full SaaS.

Join us for the future of observability

BYOB is available today for all enterprise customers across all three public clouds. New and existing customers can contact us to learn more. Capture is changing the mobile observability game by adding a control plane and local storage on every mobile device, providing extremely detailed telemetry when you need it, and none when you don’t. If lack of strict data ownership was keeping you away, now is the time to give us a try! Interested in learning more? Check out the sandbox to get a hands-on feel for what working with Capture is like and then get in touch with us for a demo. Please join us in Slack as well to ask questions and give feedback!