bd tail: magical real-time streaming of device telemetry

Consider a situation where you start to receive reports of end-users not being able to use your application, but the observability that remains after a series of cost-cutting efforts shows nothing wrong. Imagine now being able to attach directly to the device of an end-user reporting an issue and get a live feed of all the logs emitted by that device, including metadata about all HTTP calls made by the client! Consider another situation in which a production issue can only be reproduced in a specific network environment, and you need to attach to a device in that environment to understand what is going on. This is the kind of power that we are bringing to mobile developers with the release of bd tail. Traditionally, mobile logs have been confined to outputting to the device console, limiting their utility to local debugging through an IDE. If you’re lucky, some of them could be uploaded via logging systems to existing observability pipelines server side, where logs are stored in a database and subsequently queried at a (much) later stage. To provide a cheaper alternative to storing all the logs, our initial release of the Capture SDK provided a way for developers to access the mobile device logs by defining workflow explorations that instruct the device to upload logs under certain conditions, capturing the logs for later viewing via the Timeline. We are now announcing bd tail — an innovation that provides developers with the ability to initiate a temporary streaming session with a select number of devices. While developers can access mobile logs today via workflow explorations, bd tail allows attaching directly to mobile devices and seeing their logs (or a subset) in real-time. This will allow developers to:

• Stream logs from a physical device without the need to attach a debugger, be it for local debugging, remote debugging, or diagnosing a customer issue
• Inspect the raw logs being emitted by clients, making it easier to construct the desired workflow explorations that rely on matching against those logs

All of this is done via ephemeral streaming sessions that are never persisted to disk, which ensures that even if sensitive information is streamed via this tool nothing is ever recorded.

CLI Access

For the initial release, we are making it possible to access bd tail via a CLI tool. This authenticates the user with the bitdrift SaaS, ensuring that only authenticated users are able to stream data from the mobile devices. The CLI allows the user to specify an expression that influences both what kind of devices are connected to, as well as what logs are of interest. A few examples of the kinds of queries that can be made: Stream logs from ios clients, only include debug logs:

bash
> bd tail ios loglevel debug

Stream logs from android clients, only include failed network response logs:

bash
> bd tail android ltype network field _success false

See the full documentation here. The bd tail CLI then attaches to one or more running live client sessions which have their logs streamed based on the specified expression. For cases where the targeting criteria is very broad, the SaaS will coordinate selecting only a small number of clients to initiate streaming. This ensures that no matter the query, the overall impact on the potentially massive mobile fleet remains minimal.

Direct Device Debugging

In addition to supporting these kinds of broad queries against the fleet of connected devices, we also provide a way to generate short-lived device codes that can be used to connect to a specific device. This is meant to make it easy to attach to a running device, be this for initiating a live debugging session with an end-user or for in-house testing where a developer wants to debug a physical device not connected to a traditional debugger. The SDK provides a createTemporaryDeviceCode function that can be called to generate the short code, which can be used to power debug UIs that expose the code to make it easier to attach to the specific device being debugged. This enables the developer to attach directly to a running device, be this for debugging or for triaging issues observed by testers, customers, etc. Once a code has been created, a bd tail session can be initiated via

bash
bd tail code $code

Backing Architecture

While we are only releasing a CLI tool at this moment, the underlying tailing architecture is generic enough to power any number of real-time use cases. For example, we are excited to explore the idea of using log tailing to improve the developer experience when writing workflow explorations by providing live feedback on matched logs. To understand how bd tail works, recall the SDK overview blog post where we talk about a persistent API stream between all the clients and the bitdrift SaaS. As part of establishing this connection, clients identify themself with enough information that the SaaS is able to fan out both long- and short-lived configuration to clients that match certain criteria. The bd tail flow looks like this:

1. SaaS receives a request for a bd tail session with an accompanying query expression
2. The targeting criteria from the query expression is extracted and used to determine a subset of active clients to consider
3. Using the log matching criteria extracted from the remaining query expression, a small configuration payload specifying what logs should be streamed is sent to the subset of clients
4. For the duration of the bd tail session, logs that are received from the targeted clients are forwarded directly to the attached bd tail client
5. Upon stream completion, the mobile devices are detached from the bd tail session and the configuration is reverted

The logs that are streamed via bd tail are the same logs that would be made available via Record Session within the Capture product, which makes this tool suitable for understanding the shape of the data without having to go through the process of creating a workflow and capturing data from multiple clients. The JSON output format that can be accessed via the --json option also provides the raw data, which can be used to understand what the data being matched on by workflows looks like.

If this sounds interesting to you, please get a demo and/or come and chat with us in our community Slack. We would love to hear from you! If you ’re excited about what we’re building, and would like to build it with us, check out our open positions. Let’s create something amazing together!