Key Default Type Description
pulsar.source.allowKeySharedOutOfOrderDelivery
false Boolean If enabled, it will relax the ordering requirement, allowing the broker to send out-of-order messages in case of failures. This will make it faster for new consumers to join without being stalled by an existing slow consumer.
In this case, a single consumer will still receive all the keys, but they may be coming in different orders.
pulsar.source.autoCommitCursorInterval
5000 Long This option is used only when the user disables the checkpoint and uses Exclusive or Failover subscription. We would automatically commit the cursor using the given period (in ms).
pulsar.source.enableAutoAcknowledgeMessage
false Boolean Flink commits the consuming position with pulsar transactions on checkpoint. However, if you have disabled the Flink checkpoint or disabled transaction for your Pulsar cluster, ensure that you have set this option to true.
The source would use pulsar client's internal mechanism and commit cursor in a given interval.
pulsar.source.enableMetrics
true Boolean The metrics from Pulsar Consumer are only exposed if you enable this option.You should set the pulsar.client.statsIntervalSeconds to a positive value if you enable this option.
pulsar.source.enableSchemaEvolution
false Boolean If you enable this option and use PulsarSourceBuilder.setDeserializationSchema(Schema), we would consume and deserialize the message by using Pulsar's Schema interface with extra schema evolution check.
pulsar.source.fetchOneMessageTime
(none) Integer The time (in ms) for fetching one message from Pulsar. If time exceed and no message returned from Pulsar. We would consider there is no record at the current topic partition and stop fetching until next switch.
It's not configured by default. We will use the remaining time in pulsar.source.maxFetchTime by default, which may cause a long wait in small message rates. Add this option in source builder avoiding waiting too long.
pulsar.source.maxFetchRecords
100 Integer The maximum number of records to fetch to wait when polling. A longer time increases throughput but also latency. A fetch batch might be finished earlier because of pulsar.source.maxFetchTime.
pulsar.source.maxFetchTime
10000 Long The maximum time (in ms) to wait when fetching records. A longer time increases throughput but also latency. A fetch batch might be finished earlier because of pulsar.source.maxFetchRecords.
pulsar.source.partitionDiscoveryIntervalMs
300000 Long The interval (in ms) for the Pulsar source to discover the new partitions. A non-positive value disables the partition discovery.
pulsar.source.resetSubscriptionCursor
false Boolean The StartCursor in connector is used to create the initial subscription. Enable this option will reset the start cursor in subscription by using StartCursor everytime you start the application without the checkpoint.
pulsar.source.verifyInitialOffsets
WARN_ON_MISMATCH

Enum

Upon (re)starting the source, check whether the expected message can be read. If failure is enabled, the application fails. Otherwise, it logs a warning. A possible solution is to adjust the retention settings in Pulsar or ignoring the check result.

Possible values:
  • "FAIL_ON_MISMATCH": Fail the consuming from Pulsar when we don't find the related cursor.
  • "WARN_ON_MISMATCH": Print a warn message and start consuming from the valid offset.