Confluent CCDAK Dumps

A 409 Conflict response in Kafka Connect REST API usually means:

The connector name provided already exists (i.e., a connector with that name is already deployed).

The Kafka Connect cluster is temporarily unavailable to accept new tasks due to a rebalance in progress.

From the Confluent Developer documentation and Kafka Connect REST API spec:

“409 – Conflict. The connector name is already in use or the worker group is in the process of rebalancing and cannot complete the request at this time.”

Option A is incorrect because invalid configurations return 400 Bad Request, not 409.

Option B is invalid because Kafka Connect does not have a strict concept of "capacity" that prevents connector creation with a 409.

According to the official Apache Kafka consumer documentation, the auto.offset.reset configuration is only applied when a consumer has no valid committed offset for a partition. In this scenario, the consumer last committed offset 217, but the topic’s earliest available offset is now 318. This means the previously committed offset is out of range, most likely due to topic retention cleanup.

When a committed offset is out of range, Kafka treats it as invalid and applies the auto.offset.reset policy. Since the consumer is configured with auto.offset.reset=latest, it will start reading from the end of the log, not from the earliest available offset.

Kafka defines the “latest” offset as the log end offset, which is the offset of the next message to be written. Given that the topic’s offsets end at 588, the next offset is 589.

Therefore, the consumer will start reading from offset 589, skipping all existing records in the topic, which aligns with the documented Kafka consumer behavior.

=========================

The Apache Kafka documentation explains that Kafka’s parallelism model is based on topic partitions and consumer groups. Within a single consumer group, each partition can be assigned to at most one consumer at a time. As a result, the maximum level of parallelism for a consumer group is equal to the number of partitions in the topic.

In this scenario, the topic has five partitions, which means that up to five consumers in the same consumer group can actively consume data in parallel. If more than five consumers are added, the extra consumers will remain idle because there are no additional partitions to assign.

Option B is incorrect because Kafka does not enforce equal message consumption across partitions; message distribution depends on producers and partitioning strategy. Option C is false because partitions may be hosted on different brokers, and consumers fetch data from multiple brokers. Option D is incorrect because a single partition cannot be consumed by multiple consumers within the same group simultaneously.

Therefore, the only valid assumption is that a maximum of five consumers can actively consume messages in the same consumer group.

=========================

To send failed records to a dead letter queue (DLQ), you must configure:

errors.tolerance=all: Tells the connector to not fail on errors but handle them (e.g., send to DLQ).

errors.deadletterqueue.topic.name=DLQ-Topic: Specifies the DLQ topic.

errors.deadletterqueue.context.headers.enable=true: Includes error context in message headers.

From Kafka Connect Error Handling Docs:

“Kafka Connect supports directing problematic records to a separate topic (DLQ) using errors.* configs. Headers can include failure metadata.”

Options D, E, F are related to logging, not DLQ behavior.

According to the official Apache Kafka documentation, log compaction is the mechanism used to retain only the latest record for each key in a topic. By setting cleanup.policy=compact, Kafka ensures that older records with the same key are eventually removed, leaving only the most recent value for each key.

This behavior is exactly suited for use cases such as tracking the current state of an entity, including GPS tracker locations, user profiles, or configuration data. In this scenario, the key (trackerId) uniquely identifies a tracker, and the value represents its most recent latitude and longitude.

Option B (retention.ms=infinite) retains all historical data, which contradicts the requirement. Option C (min.cleanable.dirty.ratio) controls when compaction runs, not what data is retained. Option D (retention.ms=0) would immediately delete all data and is unsafe.

Therefore, setting cleanup.policy=compact is the correct and officially documented solution to always retain the latest location per tracker.

=========================

If a key is not specified, the default Kafka partitioner uses round-robin logic to assign partitions. This leads to even distribution of messages across all partitions.

From Kafka Producer Design:

“If the key is null, the producer will assign the partition in a round-robin fashion to achieve load balancing.”

There is no error, and partition 0 is not always chosen in this case.

For real-time applications, the key metric to monitor is consumer lag — the difference between the latest offset on the broker and the last committed offset by the consumer.

From Kafka Monitoring Guide:

“Consumer lag is the most important metric for real-time applications. It tells you how far behind the consumer is from the latest data in Kafka.”

B relates to replication, not client responsiveness.

C tracks group membership stability, not data delay.

D tracks input throughput, not processing latency.

According to the Apache Kafka producer documentation, serialization is configured independently for message keys and values. The key and value can use different serializers depending on their data types.

In this scenario, the message key is a string, so the producer must be configured with org.apache.kafka.common.serialization.StringSerializer for the key. The value contains JSON, so a suitable JSON serializer must be configured for the value, such as a custom JSON serializer, a library-based serializer, or a byte-array serializer after converting the JSON to bytes.

Option B correctly reflects this separation by specifying a string serializer for the key and a JSON serializer for the value. Kafka does not automatically infer serializers, and setting a serializer to null is invalid. Additionally, there is no built-in “JSON” serializer for keys, making Option D incorrect.

Therefore, the correct approach—fully aligned with Kafka’s producer configuration model—is to explicitly configure a StringSerializer for the key and a JSON-capable serializer for the value.

=========================

Low I/O thread activity with blocked throughput often indicates that producer callbacks are consuming too much time, causing the sender thread to block while waiting for onCompletion() to finish.

From Kafka Producer Performance Guide:

“Expensive logic in callbacks (e.g., I/O or complex computation) can block the sender thread, reducing throughput.”

Compression (A) may slightly impact CPU but not I/O thread usage.

Large batches (B) improve throughput if managed correctly.

A Layer 2 network issue (C) would lead to packet loss, not specifically low callback metrics.

The official Apache Kafka Streams developer documentation clearly states that TopologyTestDriver is the recommended and supported tool for unit testing Kafka Streams applications without requiring a running Kafka cluster. It is specifically designed to test complex topologies consisting of multiple sources, processors, state stores, sinks, and even sub-topologies.

TopologyTestDriver allows developers to instantiate the Kafka Streams topology directly in memory and feed test records into input topics while reading results from output topics. This makes it ideal for fast, deterministic, and isolated unit tests. As documented, it supports testing stateful and stateless operations, windowed aggregations, joins, and exactly-once behavior at the topology level.

Option A is incorrect because TestProducer and TestConsumer are not official Kafka Streams testing utilities. Option B is invalid because KafkaUnitTestDriver does not exist in Apache Kafka. Option D (MockProducer and MockConsumer) belongs to the Kafka Clients API and is suitable for producer/consumer testing, but it does not support Kafka Streams topologies or state stores.

Therefore, TopologyTestDriver is the only fully supported and officially documented solution for unit testing Kafka Streams applications without a real Kafka cluster.

Using assign() gives manual control over partition assignment. In this mode:

B is true: The application chooses partitions directly without involving the group coordinator.

C is true: Since no group coordination occurs, no rebalancing happens if a consumer dies.

From Kafka Consumer API Docs:

“assign() manually assigns a list of partitions to this consumer. No coordination takes place and no rebalancing is triggered.”

A is false: subscribe() and assign() are mutually exclusive.

D is incorrect: Partitions don't need to be equal across topics.

If a message key is not provided, Kafka’s default partitioner uses round-robin distribution across available partitions.

From Kafka Producer Design:

“If no key is provided, the default partitioner distributes messages round-robin to available partitions.”

A is incorrect — not all go to partition 0.

C is invalid — Kafka doesn’t group messages into one segment without a key.

D is false — ordering is not preserved across partitions without a key.

This syntax is a clear match to Protocol Buffers (Protobuf). It defines a schema with fields, types, and tags, which is a format supported by Kafka when using Protobuf-based schema registry serialization.

From Confluent Schema Registry Docs:

“Kafka supports Protobuf serialization, where schemas are written in .proto files and include fields with tags.”

int32, double, and string are standard Protobuf types.

Avro uses JSON-style schema.

JSON Schema uses JSON object structure, not .proto.

YAML is unrelated.

The default maximum message size that a Kafka broker accepts is 1MB (1,048,576 bytes), controlled by the config property message.max.bytes.

From Kafka Broker Configuration Docs:

“The default maximum message size is 1MB. To accept larger messages, configure message.max.bytes and the producer’s max.request.size.”

Producers also have a matching limit via max.request.size, and consumers via fetch.message.max.bytes.

The Apache Kafka Streams documentation clearly distinguishes between the Kafka Streams DSL and the Processor API. While the DSL is designed for common stream processing operations such as filtering, mapping, joining, and aggregations, it does not support fully custom processing logic.

For use cases that require custom transformations, fine-grained control over record processing, access to headers, timestamps, state stores, and punctuation, Kafka Streams provides the Processor API. This API allows developers to implement custom Processor, Transformer, or ValueTransformer classes and explicitly define the processing topology.

Option A (TopologyTestDriver) is a testing utility, not a development API. Option C (DSL) is higher-level and not suitable for advanced custom logic. Option D does not exist in Kafka.

Therefore, the correct and officially supported approach for defining custom processors in Kafka Streams is to use the Processor API.

=========================

The official Apache Kafka producer documentation states that setting enable.idempotence=true guarantees that messages are written to a partition exactly once, even in the presence of retries caused by network failures or broker timeouts. This feature prevents duplicate records by assigning producer sequence numbers and validating them on the broker side.

For idempotent producers to work correctly, retries must be enabled, which is why Kafka recommends a very large value such as retries=2147483647. Additionally, limiting max.in.flight.requests.per.connection to 1 ensures strict ordering during retries, preventing message reordering in older Kafka versions and providing the safest configuration.

Option A is irrelevant to producers. Option B explicitly disables idempotence, which causes duplicates. Option D disables retries, which increases the risk of message loss.

Therefore, the correct and fully documented solution to eliminate duplicate messages is enabling idempotence with retries and a safe in-flight request limit, as shown in Option C.

Frequent rebalances in a consumer group occur when:

Consumers are too slow and miss heartbeats (A), or

Instances crash or restart, triggering group membership changes (D)

From Kafka Consumer Group Coordination Docs:

“If a consumer fails to send a heartbeat in time (due to slowness or crash), it is removed from the group, causing a rebalance.”

Option B (partition mismatch) affects load balancing, not rebalance frequency.

Option C (broker storage) doesn't trigger consumer rebalances.

The bootstrap.servers setting is used by clients (producers and consumers) to initially connect to the Kafka cluster. It does not need to list all brokers—just a few that are up and reachable. The client will then use the cluster metadata to discover the full broker list.

From Kafka Java Client Configuration Docs:

“This is a list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping.”

Hence, listing multiple brokers provides redundancy and fault tolerance in the event one is down during client startup.

The ConsumerRebalanceListener lets you handle partition assignments and revocations during rebalances. This is critical for managing offsets, stateful processing, or external transactions.

From Kafka Consumer Rebalance Docs:

“Implementing ConsumerRebalanceListener allows your application to take action before and after partitions are reassigned.”

A is true: It lets your app react when partitions assigned to the consumer change.

B, C, and D are unrelated to consumer rebalancing directly.

For SASL/PLAIN authentication over SSL, the correct combination is:

security.protocol=SASL_SSL

sasl.jaas.config=...PlainLoginModule required username=... password=...

From Kafka Security Documentation:

“For SASL/PLAIN over SSL, configure security.protocol=SASL_SSL and provide login module configuration using sasl.jaas.config.”

Options B and D are incorrect because they do not pair the correct protocol with the authentication method.

The Apache Kafka consumer documentation explains that when a RecordDeserializationException occurs, the consumer cannot continue polling until the problematic offset is skipped. Simply logging the error is insufficient, because the consumer will repeatedly fail on the same record.

The recommended pattern is to log the malformed record, extract its topic, partition, and offset from the exception, and then call consumer.seek() to move the consumer position to the next offset (offset + 1). This allows the application to skip the poison pill and resume processing subsequent records.

Option B is invalid because Kafka does not provide a consumer.skip() API. Option C is unnecessary if the application is designed to tolerate malformed records. Option D results in an infinite failure loop.

Therefore, seeking past the bad record after logging it is the correct and officially documented way to handle poison pill records while maintaining consumer liveness and resilience.

According to the official Apache Kafka consumer semantics documentation, at-most-once delivery means that records may be lost but must never be processed more than once. To achieve this guarantee, a consumer must commit offsets before processing records. If the application crashes after committing but before processing, messages are skipped rather than reprocessed, which satisfies at-most-once semantics.

Option B correctly implements this pattern. It disables automatic offset commits (enable.auto.commit=false) and explicitly calls commitAsync() immediately after polling and before processing records. This ensures that offsets are advanced regardless of whether processing succeeds, eliminating the possibility of duplicate processing.

Option C represents at-least-once semantics, because offsets are committed after processing, which can lead to duplicates on failure. Options A and D rely on automatic commits or incorrect ordering and do not provide strict at-most-once guarantees.

The Kafka documentation explicitly states that manual offset management with commits before processing is required for at-most-once delivery.

Therefore, Option B is the best and officially documented configuration and code skeleton to avoid duplicate message processing.

Kafka producers can group messages destined for different partitions into separate batches, and send multiple batches in one request to optimize network usage.

From Kafka Producer Internals:

“The producer maintains buffers (batches) for each partition and can send multiple of these batches together in a single request to the broker.”

Message size is not a constraint for batching (A is false).

Broker failures don’t disable batching (B is false).

There is no separate thread per batch (C is false).

According to the official Apache Kafka documentation for Kafka Streams, exactly-once semantics (EOS) are controlled using the processing.guarantee configuration. To ensure that each input record is processed once and only once, and that the results are written atomically to the output topic, Kafka Streams provides built-in EOS support.

The correct and recommended configuration is processing.guarantee=exactly_once_v2, which is the default EOS implementation starting from Kafka 3.0. This mode uses Kafka’s transactional APIs to atomically commit consumer offsets and produce output records within a single transaction. As documented, this prevents duplicates even in the presence of failures, retries, or application restarts.

Option A is incorrect because log compaction does not guarantee exactly-once processing; it only retains the latest record per key. Option B is insufficient because enable.idempotence alone does not handle consumer offset commits or state store consistency in Kafka Streams. Option C is invalid because no such configuration exists in Kafka Streams.

Therefore, setting processing.guarantee=exactly_once_v2 is the only fully correct and officially supported solution for achieving EOS in Kafka Streams applications.

Kafka Connect sink connectors use the standard Kafka consumer mechanism to track offsets, which means offsets are stored in the __consumer_offsets internal topic.

From Kafka Connect Documentation:

“Sink connectors are regular Kafka consumers and store their offsets in the __consumer_offsets topic like any other consumer.”

The other options refer to source connectors or worker configuration.

offsets.storage.topic is relevant for source connectors.

According to the official Apache Kafka producer metrics documentation, the I/O thread ratio and I/O thread wait ratio indicate how busy the producer’s network I/O thread is and how much time it spends waiting. When both metrics are low, it means the I/O thread is underutilized and not blocked on network operations.

This strongly suggests that the bottleneck is not network-related. If there were network latency or a bad data link (Option B), the I/O thread wait ratio would be high. Similarly, large batches (Option A) or compression (Option D) generally increase throughput and would result in higher I/O utilization.

An expensive or blocking callback function (for example, heavy logic in onCompletion()) executes in the producer’s sender thread context. Kafka documentation explicitly warns that slow callbacks can throttle producer throughput because they delay the processing of acknowledgements and subsequent sends.

Therefore, the most likely and documented cause of low producer throughput with low I/O utilization is expensive callback logic in the producer code.

=========================

A GlobalKTable is a replicated, read-only table available in full on all instances. It's best for reference or lookup datasets such as:

Product catalog

Customer profiles

Warehouse inventory

From Kafka Streams Developer Guide:

“Use GlobalKTable when you need to perform joins using non-partition-aligned reference data that's small enough to replicate.”

Purchases and transactions are high-throughput, time-sensitive streams, not static reference data.