使用/Portkey-AI/gateway反向代理大模型接口

wylc123 17小时前 ⋅ 11 阅读

背景:

公司禁用智谱大模型接入

先再服务器安装nodejs

sudo yum install -y nodejs
[root@iZ2zeboalrxym420100ygvZ ~]# npm -v
10.8.2
[root@iZ2zeboalrxym420100ygvZ ~]# node -v
v20.20.2

安装portkey-ai/gateway

[root@iZ2zeboalrxym420100ygvZ ~]# npx @portkey-ai/gateway
Need to install the following packages:
@portkey-ai/gateway@1.15.2
Ok to proceed? (y)
🚀 Your AI Gateway is running at:
   http://localhost:8787

📱 UI: http://localhost:8787/public/

✨ Ready for connections!

安装pm2使其能五窗口启动

[root@iZ2zeboalrxym420100ygvZ ~]# sudo npm install -g pm2

changed 45 packages in 1s

6 packages are looking for funding
  run `npm fund` for details
[root@iZ2zeboalrxym420100ygvZ ~]# pm2 start "npx @portkey-ai/gateway" --name portkey-gateway

                        -------------

__/\\\\\\\\\\\\\____/\\\\____________/\\\\____/\\\\\\\\\_____
 _\/\\\/////////\\\_\/\\\\\\________/\\\\\\__/\\\///////\\\___
  _\/\\\_______\/\\\_\/\\\//\\\____/\\\//\\\_\///______\//\\\__
   _\/\\\\\\\\\\\\\/__\/\\\\///\\\/\\\/_\/\\\___________/\\\/___
    _\/\\\/////////____\/\\\__\///\\\/___\/\\\________/\\\//_____
     _\/\\\_____________\/\\\____\///_____\/\\\_____/\\\//________
      _\/\\\_____________\/\\\_____________\/\\\___/\\\/___________
       _\/\\\_____________\/\\\_____________\/\\\__/\\\\\\\\\\\\\\\_
        _\///______________\///______________\///__\///////////////__


                          Runtime Edition

        PM2 is a Production Process Manager for Node.js applications
                     with a built-in Load Balancer.

                Start and Daemonize any application:
                $ pm2 start app.js

                Load Balance 4 instances of api.js:
                $ pm2 start api.js -i 4

                Monitor in production:
                $ pm2 monitor

                Make pm2 auto-boot at server restart:
                $ pm2 startup

                To go further checkout:
                http://pm2.io/


                        -------------

[PM2] Spawning PM2 daemon with pm2_home=/root/.pm2
[PM2] PM2 Successfully daemonized
[PM2] Starting /usr/bin/bash in fork_mode (1 instance)
[PM2] Done.
┌────┬────────────────────┬─────────────┬─────────┬─────────┬──────────┬────────┬──────┬───────────┬──────────┬──────────┬──────────┬──────────┐
│ id │ name               │ namespace   │ version │ mode    │ pid      │ uptime │ ↺    │ status    │ cpu      │ mem      │ user     │ watching │
├────┼────────────────────┼─────────────┼─────────┼─────────┼──────────┼────────┼──────┼───────────┼──────────┼──────────┼──────────┼──────────┤
│ 0  │ portkey-gateway    │ default     │ N/A     │ fork    │ 3203213  │ 0s     │ 0    │ online    │ 0%       │ 27.5mb   │ root     │ disabled │
└────┴────────────────────┴─────────────┴─────────┴─────────┴──────────┴────────┴──────┴───────────┴──────────┴──────────┴──────────┴──────────┘
[root@iZ2zeboalrxym420100ygvZ ~]# pm2 save && pm2 startup
[PM2] Saving current process list...
[PM2] Successfully saved in /root/.pm2/dump.pm2
[PM2] Init System found: systemd
Platform systemd
Template
[Unit]
Description=PM2 process manager
Documentation=https://pm2.keymetrics.io/
After=network.target

[Service]
Type=forking
User=root
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/softs/jdk1.8.1/bin:/usr/local/php/bin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
Environment=PM2_HOME=/root/.pm2
PIDFile=/root/.pm2/pm2.pid
Restart=on-failure

ExecStart=/usr/local/lib/node_modules/pm2/bin/pm2 resurrect
ExecReload=/usr/local/lib/node_modules/pm2/bin/pm2 reload all
ExecStop=/usr/local/lib/node_modules/pm2/bin/pm2 kill

[Install]
WantedBy=multi-user.target

Target path
/etc/systemd/system/pm2-root.service
Command list
[ 'systemctl enable pm2-root' ]
[PM2] Writing init configuration in /etc/systemd/system/pm2-root.service
[PM2] Making script booting at startup...
[PM2] [-] Executing: systemctl enable pm2-root...
Created symlink /etc/systemd/system/multi-user.target.wants/pm2-root.service → /etc/systemd/system/pm2-root.service.
[PM2] [v] Command successfully executed.
+---------------------------------------+
[PM2] Freeze a process list on reboot via:
$ pm2 save

[PM2] Remove init script via:
$ pm2 unstartup systemd
[root@iZ2zeboalrxym420100ygvZ ~]# pm2 logs portkey-gateway
[TAILING] Tailing last 15 lines for [portkey-gateway] process (change the value with --lines option)
/root/.pm2/logs/portkey-gateway-error.log last 15 lines:
/root/.pm2/logs/portkey-gateway-out.log last 15 lines:
🚀 Your AI Gateway is running at:
0|portkey- |    http://localhost:8787
0|portkey- |
0|portkey- | 📱 UI: http://localhost:8787/public/
0|portkey- |
0|portkey- | ✨ Ready for connections!

配置nginx使其能外网访问

[root@iZ2zeboalrxym420100ygvZ nginx]# sudo htpasswd -c /usr/softs/nginx/.htpasswd admin
New password:
Re-type new password:
Adding password for user admin

nginx配置

#portkey
   	 server {
		listen       80;
		server_name  portkey.songbin.top;
		#rewrite ^(.*)$ https://$host$1 permanent;   #将所有http请求通过rewrite重定向到https。
		location / {
			proxy_pass http://127.0.0.1:8787;
			proxy_set_header Host $host;
			proxy_set_header X-Forwarded-For $remote_addr;
			proxy_http_version 1.1;
			proxy_buffering off;          # 流式 SSE 必须关,否则打字机效果失效
			proxy_read_timeout 300s;

			#auth_basic "portkey gateway";
			#auth_basic_user_file /usr/softs/nginx/.htpasswd;
		}
   	 }

重启nginx

./nginx -s reload

 智谱大模型举例

智谱免费模型:
API 定价 - 智谱AI开放文档

1.文本模型

模型名称 上下文 输入单价(元/百万 Tokens) 输出单价(元/百万 Tokens) 缓存存储(元/百万 Tokens/小时) 缓存命中(元/百万 Tokens) 输入模态
GLM-4-Flash-250414 128K 免费 免费 限时免费 不支持
GLM-Z1-Flash 128K 免费 免费 限时免费 不支持
GLM-4.7-Flash 200K 免费 免费 限时免费 免费
GLM-4.5-Flash 200K 免费 免费 限时免费 免费
curl http://xxxxx.songbin.top/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "x-portkey-provider: openai" \
  -H "x-portkey-custom-host: https://open.bigmodel.cn/api/paas/v4/" \
  -H "Authorization: Bearer 你的apiKey" \
  -H "x-portkey-forward-headers: Authorization" \
  -d '{"model":"glm-4.5-flash","messages":[{"role":"user","content":"你好"}]}'

 

2.多模态生

模型名称 简介 规格 单价 Batch API 定价
CogView-3-Flash 图像生成 多分辨率 免费 不支持
CogVideoX-Flash 视频生成 多分辨率 免费 不支持

3.视觉理解

模型名称 简介 规格 单价 Batch API 定价
CogView-3-Flash 图像生成 多分辨率 免费 不支持
CogVideoX-Flash 视频生成 多分辨率 免费 不支持

4.知识库服务

功能名称 具体能力 支持范围 计费类型 价格
图片理解 图像解析 图片 单价 免费
上下文增强 上下文检索补全增强 文本 单价 免费

 

#portkey
   	 server {
		listen       80;
		server_name  xxxx.songbin.top;
		#rewrite ^(.*)$ https://$host$1 permanent;   #将所有http请求通过rewrite重定向到https。
		location / {
			proxy_pass http://127.0.0.1:8787;
			proxy_set_header Host $host;
			proxy_set_header X-Forwarded-For $remote_addr;
			proxy_http_version 1.1;
			proxy_buffering off;          # 流式 SSE 必须关,否则打字机效果失效
			proxy_read_timeout 300s;

			#auth_basic "portkey gateway";
			#auth_basic_user_file /usr/softs/nginx/.htpasswd;
		}
   	 }

相关文章推荐
  • 该目录下还没有内容!

全部评论: 0 条

    我有话说: