Apache Kafka does not guarantee record order for topics. Kafka preserves the record order only in a topic partition. A Kafka consumer polls from all partitions at the same time. Depending on a number of factors, such as network connectivity or broker utilization, a Kafka consumer receives responses with records from multiple partitions in a non-deterministic order.

Recall that when a Kafka topic contains one partition only, a consumer receives records in order. This is because with one partition, a consumer polls only one Kafka broker. Therefore, a consumer receives the records in the order preserved within a partition.

For more information about one-partition Kafka topics, see the section called “Creating Topics”.

Event order tends to be crucial for event flows. An event flow is a logical group of events that model a business process. For example, the following series of events creates a mortgage pre-approval event flow:

In the preceding flow, each event might depend on the result of previous events. For example, determining if a customer is eligible for a mortgage can depend on calculated monthly payments. Pre-approval can then depend on information from all of the preceding events. If a consumer receives the CalculatedEligibility event but no other events from the flow, then it cannot determine the next action.

Event flows can span multiple services. Each event in the flow might be produced by a different microservice in your application. If you can identify event flows within your application, then consider creating topics per event flow. This enables the partial ordering of events emitted by multiple producers within a topic.

You can enforce event order in the following ways depending on the requirements for your use case:

If your application requires ordering, then consider if you need total order. You can achieve partial order by assigning events related keys.

For example, consider a stock market ticker application. Each event in the application contains a unique stock ticker symbol and current value, such as ACME, 15. If you use the stock ticker symbol as an event key, then events related to the same stock arrive to the same partition. Consequently, when consuming the events, one consumer receives ticks from the same stock.

The preceding example, however, does not maintain total order due to the polling behavior. A consumer receives responses with records from multiple partitions in a non-deterministic order. Consequently, a Kafka application might process the ACME ticks before the AACME ticks. However, every event that contains the ACME key is sent to the same partition. Consequently, because a partition that contains events with the ACME key is ordered, a consumer node receives all ACME records in order.

Kafka enables you to group records based on the record key. Windowing enables you to further split the key-based groups based on time intervals.

Kafka Streams enables you to define the following windows:

Use the Kafka Streams DSL to implement event windowing. The following is an example code snippet that uses a window:

import org.apache.kafka.streams.kstream.Suppressed;
import org.apache.kafka.streams.kstream.TimeWindows;

// class and imports omitted

new StreamsBuilder().stream("input-topic",
  Consumed.with(Serdes.String(), Serdes.String())
  .groupByKey()
  .windowedBy( 
    TimeWindows.of(Duration.ofSeconds(10)) 
    .grace(Duration.ofSeconds(5))) 
  .count() 
  .suppress(
    Suppressed.untilWindowCloses(
      Suppressed.BufferConfig.unbounded())) 
  .toStream() 
  .print(Printed.toSysOut());

Out-of-order (OOO) or late events are events that arrive later than expected. For example, an event with timestamp 00:00 can arrive at 00:30 due to various issues, such as a network outage or a consumer issue.

In stateless applications, late or out of order events typically do not impact processing logic. In applications that use stateful operations, like joins or other aggregations, a late event can cause incorrect results.

You can implement several strategies for dealing with late events, such as: