Table of Contents

Class KNetSourceConnector<TSourceConnector, TTask>

Namespace
MASES.KNet.Connect
Assembly
MASES.KNet.dll

An implementation of KNetConnector<TConnector> for source connectors

public abstract class KNetSourceConnector<TSourceConnector, TTask> : KNetConnector<TSourceConnector>, IKNetSourceConnector, IKNetConnector, IKNetCommon, IKNetConnectLogging, IConnector, IVersion where TSourceConnector : KNetSourceConnector<TSourceConnector, TTask> where TTask : KNetSourceTask<TTask>

Type Parameters

TSourceConnector

The connector class inherited from KNetSourceConnector<TSourceConnector, TTask>

TTask

The task class inherited from KNetSourceTask<TTask>

Inheritance
KNetConnector<TSourceConnector>
KNetSourceConnector<TSourceConnector, TTask>
Implements
Derived
Inherited Members

Properties

CanDefineTransactionBoundaries

Implement the method to return the ConnectorTransactionBoundaries value

public virtual ConnectorTransactionBoundaries CanDefineTransactionBoundaries { get; }

Property Value

ConnectorTransactionBoundaries

Context

The SourceConnectorContext

public SourceConnectorContext Context { get; }

Property Value

SourceConnectorContext

ExactlyOnceSupport

Implement the method to return the ExactlyOnceSupport value

public virtual ExactlyOnceSupport ExactlyOnceSupport { get; }

Property Value

ExactlyOnceSupport

ReflectedRemoteObjectClassName

Set the ReflectedRemoteObjectClassName of the connector to a fixed value

protected override sealed string ReflectedRemoteObjectClassName { get; }

Property Value

string

TaskClassType

Set the TaskClassType of the connector to the value defined from TTask

public override sealed Type TaskClassType { get; }

Property Value

Type

Methods

AlterOffsets(Map<String, String>, Map<Map<String, object>, Map<String, object>>)

Invoked when users request to manually alter/reset the offsets for this connector via the Connect worker's REST API. Connectors that manage offsets externally can propagate offset changes to their external system in this method. Connectors may also validate these offsets to ensure that the source partitions and source offsets are in a format that is recognizable to them. Connectors that neither manage offsets externally nor require custom offset validation need not implement this method beyond simply returning true.

public virtual bool AlterOffsets(Map<String, String> connectorConfig, Map<Map<String, object>, Map<String, object>> offsets)

Parameters

connectorConfig Map<String, String>

The configuration of the connector

offsets Map<Map<String, object>, Map<String, object>>

A Map<K, V>> from source partition to source offset, containing the offsets that the user has requested to alter/reset. For any source partitions whose offsets are being reset instead of altered, their corresponding source offset value in the map will be null. This map may be empty, but never null>. An empty offsets Map<K, V>> could indicate that the offsets were reset previously or that no offsets have been committed yet.

Returns

bool

whether this method has been overridden by the connector; the default implementation returns false, and all other implementations (that do not unconditionally throw exceptions) should return true

Remarks

User requests to alter/reset offsets will be handled by the Connect runtime and will be reflected in the offsets returned by any OffsetStorageReader instances provided to this connector and its tasks. Note that altering/resetting offsets is expected to be an idempotent operation and this method should be able to handle being called more than once with the same arguments (which could occur if a user retries the request due to a failure in writing the new offsets to the offsets store, for example). Similar to validate, this method may be called by the runtime before the Start(IReadOnlyDictionary<string, string>)> method is invoked.

Edit this page