#515 Kafka Data Delivery to File Storage

Companies:

Ozon

Themes:

System DesignEvent-Driven Architecture

Describe how you would implement a fault-tolerant system for delivering data from Kafka topics to distributed file storage.

Conditions:

Delivery guarantee — Exactly Once;
The output format must preserve message data: partition, offset, headers, key, and message body;
The delay between a message appearing in the topic and being written to storage must not exceed 20 minutes;
100+ topics and dynamic subscription to new topics;
20 TB of topic data per day;
The recommended minimum storage-file size is 512 MB.

Question review

Mentor

Answer as you would in a real interview. I’ll ask follow-up questions, then compare your answer with the reference answer and show your accuracy as a %.

Sign in to start the dialogue The mentor keeps the context of the current question and your previous messages.