k8s nifi-1.17.0

zookeeper 3.6.3과 nifi 1.17.0을 k8s에 statefulset으로 올리고, nifi web ui는 NodePort를 사용한다.

zookeeper yaml link

https://kyeongseo.tistory.com/entry/k8s-nifi-zookeeper

k8s nifi-1.13.2 zookeeper-3.6.3

 

nifi.yaml

nifi 1.14부터는 NIFI_SENSITIVE_PROPS_KEY가 필수임

NIFI_WEB_HTTP_PORT를 지정하면 http, 지정하지 않으면 https

TZ env를 사용해 timezone을 kst로 잡아준다.

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: nifi
spec:
  selector:
    matchLabels:
      app: nifi
  serviceName: nifi-hs
  replicas: 3
  updateStrategy:
    type: RollingUpdate
  podManagementPolicy: OrderedReady
  template:
    metadata:
      labels:
        app: nifi
    spec:
      affinity:
        podAntiAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            - labelSelector:
                matchExpressions:
                  - key: "app"
                    operator: In
                    values:
                    - nifi
              topologyKey: "kubernetes.io/hostname"
      containers:
      - name: nifi
        image: "oks529/nifi:1.17.0"
        env:
        - name: NIFI_CLUSTER_IS_NODE
          value: "true"
        - name: HOSTNAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: NIFI_CLUSTER_ADDRESS
          value: $(HOSTNAME).nifi-hs.default.svc.cluster.local
        - name: NIFI_CLUSTER_NODE_PROTOCOL_PORT
          value: "1025"
        - name: NIFI_WEB_HTTP_HOST
          value: $(HOSTNAME).nifi-hs.default.svc.cluster.local
        - name: NIFI_WEB_HTTP_PORT
          value: "8080"
        - name: NIFI_CLUSTER_NODE_PROTOCOL_MAX_THREADS
          value: "100"
        - name: NIFI_ZK_CONNECT_STRING
          value: "zk-0.zk-hs.default.svc.cluster.local:2181,zk-1.zk-hs.default.svc.cluster.local:2181,zk-2.zk-hs.default.svc.cluster.local:2181"
        - name: NIFI_ELECTION_MAX_CANDIDATES
          value: "3"
        - name: NIFI_FLOW_CONFIGURATION_FILE
          value: "/data/flow.xml.gz"
        - name: NIFI_DATABASE_DIRECTORY
          value: "/data/database_repository"
        - name: NIFI_FLOWFILE_REPOSITORY_DIRECTORY
          value: "/data/flowfile_repository"
        - name: NIFI_CONTENT_REPOSITORY_DIRECTORY_DEFAULT
          value: "/data/content_repository"
        - name: NIFI_PROVENANCE_REPOSITORY_DIRECTORY_DEFAULT
          value: "/data/provenance_repository"
        - name: NIFI_SENSITIVE_PROPS_KEY
          value: "test123123123"
        - name: TZ
          value: Asia/Seoul
        ports:
        - containerPort: 8080
          name: client
        - containerPort: 1025
          name: nodeport
        volumeMounts:
        - name: nifi-data
          mountPath: /data
        - name: nifi-script
          mountPath: /script
      securityContext:
        runAsUser: 0
        runAsGroup: 1000
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: nifi-data
    spec:
      accessModes: ["ReadWriteMany"]
      resources:
        requests:
          storage: 5Gi
      storageClassName: "nfs-client"
  - metadata:
      name: nifi-script
    spec:
      accessModes: ["ReadWriteMany"]
      resources:
        requests:
          storage: 1Gi
      storageClassName: "nfs-client"
---
kind: Service
apiVersion: v1
metadata:
  name: nifi
  namespace: default
  labels:
    app: nifi
spec:
  type: NodePort
  selector:
    app: nifi
  ports:
    - protocol: TCP
      port: 8080
      name: nifi
---
kind: Service
apiVersion: v1
metadata:
  name: nifi-hs
spec:
  clusterIP: None
  selector:
    app: nifi
  ports:
    - port: 8081
      targetPort: 8081
      name: nifi-listen-http
    - port: 2881
      targetPort: 2881
      name: nifi-site-protocol
    - port: 2882
      targetPort: 2882
      name: nifi-node-protocol

 

ip:NodePort로 접속한다.