flink mac本地调试至流云

Last updated on July 19, 2024 pm

🧙 Questions

在mac中调试至流云

☄️ Ideas

前提

  • 安装java
  • 安装gradle
  • 安装pnpm
  • 安装isx
  • 开启mac远程登录
下载编译代码

至轻云项目: flink-yun

# 下载代码
isx clone
# 安装依赖
isx install
# 构建代码
isx package
启动项目

或者使用idea运行项目

# 清理项目缓存文件
isx clean
# 启动项目
isx backend
创建项目基础信息
standalone调试
  • 添加集群
    • 集群: flinkcluster
    • 类型: flinkcluster
    • 节点: localhost
    • Host: localhost
    • 用户名: ispong
    • 密码:
  • 节点检测
  • 节点安装
  • 查看本地启动的flink: http://localhost:8081
偷梁换柱
bash /Users/ispong/zhiliuyun-agent/bin/stop.sh
# 使用idea启动至流云代理
创建作业

短时间flinksql

CREATE TABLE from_table(
  username STRING,
  age INT
) WITH (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='users',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

CREATE TABLE to_table(
  username  STRING,
  age INT
) WITH  (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='users2',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

insert  into to_table select  * from from_table;

长时间flinksql

CREATE TABLE from_table(
  s_name STRING,
  s_suppkey INT
) WITH (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='supplier',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

CREATE TABLE to_table(
  username  STRING,
  age INT
) WITH  (
  'connector'='jdbc',
  'url'='jdbc:mysql://47.92.152.18:40004/isxcode_db',
  'table-name'='users2',
  'driver'='com.mysql.cj.jdbc.Driver',
  'username'='root',
  'password'='Zhiqingyun@isxcode');

insert  into to_table select  s_name, s_suppkey from from_table;
yarn调试

启动hadoop服务

start-all.sh
  • 查看yarn服务: http://ispong-mac.local:8088

  • 添加集群

    • 集群: yarn
    • 类型: yarn
    • 节点: localhost
    • Host: localhost
    • 用户名: ispong
    • 密码:
  • 节点检测

  • 节点安装

偷梁换柱
bash /Users/ispong/zhiliuyun-agent/bin/stop.sh
# idea启动至流云代理
k8s调试

启动k8s服务

kubectl cluster-info

# 拉取镜像
docker pull flink:1.18.1-scala_2.12
# 创建zhiliuyun-space空间
kubectl create namespace zhiliuyun-space
# 创建账号zhiliuyun
kubectl create serviceaccount zhiliuyun -n zhiliuyun-space
# 构建权限flink-role
kubectl create clusterrolebinding flink-role --clusterrole=edit --serviceaccount=zhiliuyun-space:zhiliuyun --namespace=zhiliuyun-space
  • 添加集群
    • 集群: k8s
    • 类型: kubernetes
    • 节点: localhost
    • Host: localhost
    • 用户名: ispong
    • 密码:
  • 节点检测
  • 节点安装
偷梁换柱
bash /Users/ispong/zhiliuyun-agent/bin/stop.sh
# idea启动至流云代理

flink mac本地调试至流云
https://ispong.isxcode.com/hadoop/flink/flink mac本地调试至流云/
Author
ispong
Posted on
April 22, 2024
Licensed under