ETLJobNode

This guide describes information about the source, transform, and target types of nodes in a data flow ETL job.

• ETLJobNode: Define ETL job nodes.
• JobNodeCategory: Define node categories.
• JobNodeType: Define node category types.
• Field: Define data fields.

Syntax

The syntax is as follows:

class ETLJobNode {
    private id: Long;
    private name: String;
    private JobNodeType type;
    private List<Long> parentNodeIds;

    enum Operator { EQ, NE, GT, LT, GTE, LTE }
    enum ConditionOperator { HEAD, AND, OR }
}

enum JobNodeCategory {
    SOURCE, TRANSFORM, TARGET
}

enum JobNodeType(JobNodeCategory category) {
    SOURCE_CATALOG(JobNodeCategory.SOURCE),
    SOURCE_CDB_MYSQL(JobNodeCategory.SOURCE),
    SOURCE_CDB_POSTGRESQL(JobNodeCategory.SOURCE),
    SOURCE_OBS(JobNodeCategory.SOURCE),
    TRANSFORM_DEFINE(JobNodeCategory.TRANSFORM),
    TRANSFORM_SELECT(JobNodeCategory.TRANSFORM),
    TRANSFORM_JOIN(JobNodeCategory.TRANSFORM),
    TRANSFORM_FILTER(JobNodeCategory.TRANSFORM),
    TRANSFORM_UNION(JobNodeCategory.TRANSFORM),
    TRANSFORM_AGGREGATE(JobNodeCategory.TRANSFORM),
    TRANSFORM_RENAME(JobNodeCategory.TRANSFORM),
    TRANSFORM_DROP_DUPLICATION(JobNodeCategory.TRANSFORM),
    TRANSFORM_FILL_MISSING_VALUES(JobNodeCategory.TRANSFORM),
    TRANSFORM_SQL(JobNodeCategory.TRANSFORM),
    TARGET_CDB_MYSQL(JobNodeCategory.TARGET),
    TARGET_CDB_POSTGRESQL(JobNodeCategory.TARGET),
    TARGET_OBS(JobNodeCategory.TARGET),
    TARGET_CATALOG(JobNodeCategory.TARGET)
}

class Field {
    private String name;
    private Type type;
    private List<Field> properties;

    enum Type {
       Void, Boolean, Tinyint, Smallint, Int, Bigint, Float, Double, String, Char, Varchar, Date, Datetime, Timestamp, Decimal,
       Binary, TimestampWithTimeZone, Array, Map, Struct, Uniontype;
    }
}

Field

The following describes the fields: