flink mac本地调试至流云

🧙 Questions

在mac中调试至流云

☄️ Ideas

前提

• 安装java
• 安装gradle
• 安装pnpm
• 安装isx
• 开启mac远程登录

下载编译代码

至轻云项目: flink-yun

# 下载代码
isx clone
# 安装依赖
isx install
# 构建代码
isx package

启动项目

或者使用idea运行项目

# 清理项目缓存文件
isx clean
# 启动项目
isx backend

• http://localhost:8080
• 用户: admin
• 密码: admin123

创建项目基础信息

• 添加用户
• 添加租户
• 下载上传许可证

standalone调试

• 添加集群
  • 集群: flinkcluster
  • 类型: flinkcluster
  • 节点: localhost
  • Host: localhost
  • 用户名: ispong
  • 密码:
• 节点检测
• 节点安装
• 查看本地启动的flink: http://localhost:8081

偷梁换柱

bash /Users/ispong/zhiliuyun-agent/bin/stop.sh
# 使用idea启动至流云代理

创建作业

短时间flinksql

CREATE TABLE from_table(
  username STRING,
  age INT
) WITH (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='users',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

CREATE TABLE to_table(
  username  STRING,
  age INT
) WITH  (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='users2',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

insert  into to_table select  * from from_table;

长时间flinksql

CREATE TABLE from_table(
  s_name STRING,
  s_suppkey INT
) WITH (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='supplier',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

CREATE TABLE to_table(
  username  STRING,
  age INT
) WITH  (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='users2',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

insert  into to_table select  s_name, s_suppkey from from_table;

yarn调试

启动hadoop服务

start-all.sh

• 查看yarn服务: http://ispong-mac.local:8088

• 添加集群

  • 集群: yarn
  • 类型: yarn
  • 节点: localhost
  • Host: localhost
  • 用户名: ispong
  • 密码:

• 节点检测

• 节点安装

偷梁换柱

bash /Users/ispong/zhiliuyun-agent/bin/stop.sh
# idea启动至流云代理

k8s调试

启动k8s服务

kubectl cluster-info

# 拉取镜像
docker pull flink:1.18.1-scala_2.12
# 创建zhiliuyun-space空间
kubectl create namespace zhiliuyun-space
# 创建账号zhiliuyun
kubectl create serviceaccount zhiliuyun -n zhiliuyun-space
# 构建权限flink-role
kubectl create clusterrolebinding flink-role --clusterrole=edit --serviceaccount=zhiliuyun-space:zhiliuyun --namespace=zhiliuyun-space

• 添加集群
  • 集群: k8s
  • 类型: kubernetes
  • 节点: localhost
  • Host: localhost
  • 用户名: ispong
  • 密码:
• 节点检测
• 节点安装

偷梁换柱

bash /Users/ispong/zhiliuyun-agent/bin/stop.sh
# idea启动至流云代理

🔗 Links

• flink docs
• flink docs history