Release Note
V3.8
Korean Version(한국어 버전)
Library Search
Options
Case Sensitive
Type
All
Heading
Code
Table
Figure
Text
Preface
Statutory notice
Scope of users
Contents of this manual
Notation rules
Additional explanations and guidelines
Revision history
1.
Overview
1.1
Introduction to system
1.2
Composition of Brightics DP
Component details
Hardware/Software logic diagram
Block diagram of DP component interfaces
2.
Preparing installation
2.1
Installation environment
O/S
System requirements
Server checklist
Service details
2.2
Information about the ports of DP components
Available ports of DP components and load balancing policy
Major web information
2.3
Preparing for installation
Check that OS and Brightics Hadoop are installed and the authority of the executing user is available.
Preparing installation files
2.4
Pre-settings
Checking environment variables before installation
3.
Installing
3.1
Single-node installation
Running installation script
Selecting installation mode
Modifying User Config
Starting installation
Initiating services
Checking web access
Installing HAProxy (optional)
HAProxy Logging
Making the initial setting of haproxy.cfg
Modifying Config for SW Components
3.2
Installing Brightics DP of high-availability configuration
Installing DP
Checking installation files
Running installation script
Selecting installation mode
Modifying User Config
Modifying PostgreSQL Config
Starting installation
Installing an embedded DB
Starting and stopping services
Checking web access
Configuring high-availability environments for Software Components
API Gateway
Installing Pacemaker
Cluster Configuration
Installing HAProxy (mandatory)
Using the HAProxy Logging function
Making the initial setting of haproxy.cfg
Interfacing Clusterware (Pacemaker) with HAProxy
Stopping and starting Clusterware and HAProxy
PostgreSQL
Modifying path
Modifying PostgreSQL Config
Creating a Replication user and database
Registering repmgr
Adding PostgreSQL to HAProxy
Modifying DP Service Metadata Config
Zookeeper (mandatory)
ActiveMQ
Modifying the ActiveMQ setting
Modifying DP Service and Nifi ActiveMQ Config
MongoDB
Setting MongoDB Replication
Modifying DP Service Config
Adding MongoDB to HAProxy
ElasticSearch
Configuring Elasticsearch Replication
Adding Elasticsearch to HAProxy
Modifying DP Service Elasticsearch Config
Nifi (mandatory installation)
Configuring Nifi Clustering
Registering Nifi to HAProxy
Modifying DP Service Nifi Config
DP (mandatory installation)
Synchronizing DP jwt key
DP UI
DP SparkShell
DP Service
3.3
Installing DP in a container (Kubernetes)
Required environment
Information about Brightics-dp images
Installing Brightics-dp
Preparing Helm Chart
Installing Helm Chart
Checking pods
Accessing Brightics-dp (Ingress)
Checking Brightics-dp log
Deleting Brightics-dp
Configuring Brightics-dp
Parameters (./brightics-dp/values.yaml)
Global parameters
Component parameters
Ldk Parameters
Introduction to the methods for utilizing PVC
User-Upload
Applying LDK
Adding external jars
3.4
Deleting DP
Running uninstallation script
4.
Checking normal operation
4.1
Working with base templates and registering samples
Querying template list
Registering base templates
Carrying out sample jobs
Batch sample job
Creating batch_category
Creating sample_ingest feed (Data Ingest Template)
Creating sample_transform feed (Data Transformation Template)
Stream sample job
Creating a Stream Category
Creating a Stream Data Source
Creating a Stream Feed
Running a Stream Feed
Running Sample Kafka and Message Sender
Checking Stream Dashboard
Wrapping up the job
4.2
Restarting SW Components
Checking DP start, stop, and status
Checking Nifi start, stop, and status
Checking ActiveMQ start, stop, and status
Checking MongoDB start, stop, and status
Checking Elasticsearch start, stop, and status
Checking PostgreSQL start, stop, and status
Checking Auto Tagging Core start, stop, and status
Checking ASB start, stop, and status
Pacemaker Cluster
Stopping or starting all
Stopping or starting a cluster
Checking status
Checking HAProxy start, stop, and status
Checking Zookeeper start, stop, and status
Checking SSSD start, stop, and status
Checking LDAP start, stop, and status
5.
Installing additional functions
5.1
Auto Tagging Service
Installing Auto Tagging Service
Modifying Hive configuration
Configuring HA
Registering Auto Tagging in HAProxy
Modifying Auto Tagging HA setting for DP Service
Stopping and starting
5.2
Installing ASB
Required environment
Installation files
Installing ASB
Running ASB
Adding configuration to DP Service
Adding the Semi-Structured Data Ingest Template to DP
Checking installation
Applying high availability
Registering HAProxy
Modifying ASB Service configuration
Modifying DP Service configuration
5.3
Using the data access control (Ranger/ LDAP) function
Installing Ranger and Solr
Installing Ranger Plugins
Installing LDAP and SSSD
Installing LDAP
Installing SSSD
Configuring LDAP and Ranger
Adding configuration to LDAP
Adding configuration to Ranger
Configuring HDFS
Enabling Hive Authentication
Configuring Nifi
Setting DP Service
Configuring LDAP in HA environment
Configuring syncprov
Registering LDAP to HAProxy
Modifying SSSD Config
Modifying DP Service Config
6.
Interfacing to Brightics Workspace (based on ML v3.8.1)
6.1
Configuring Brightics DP
Enabling the Brightics Workspace login module in DP Service
Enabling the Brightics Workspace login module in DP UI
Restarting DP
6.2
Adding configuration to Brightics Workspace
Adding DP users
Enabling DP Palette
Assigning CPU and memory to projects
6.3
Creating default project in DP (based on 3.7.X version)
1.
Overview
No Caption
No Caption
No Caption
No Caption
3.
Installing
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
4.
Checking normal operation
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
5.
Installing additional functions
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
6.
Interfacing to Brightics Workspace (based on ML v3.8.1)
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
No Caption
Brightics DP v3.8 Installation guide
