【占坑】如何将已有的discourse实例迁移到1panel(btpanel同理) + Amazon s3改用cloudflare r2

先占个坑,有空完善

注意

:warning:在以下操作前,请全盘备份!:warning:

:warning:在以下操作前,请全盘备份!:warning:

背景概述

:warning:在以下操作前,请全盘备份!:warning:

之前在一台设备上单独运行了discourse,性能过剩非常浪费,所以想在设备上同时运行很多服务,并且用1panel(btpanel等同理)反向代理+ssl,实现各个网站都可以https域名访问

此外,之前的discourse使用了Amazon s3进行文件上传、备份,虽然不贵,但每个月都要给我扣钱,很没必要,所以想改用Cloudflare R2,其免费权限支持10G/月,足够满足我的需求

目的

:warning:在以下操作前,请全盘备份!:warning:

为此,我们需要运行起来1panel(btpanel等同理),由面板接管80和443端口,然后在面板上设置反向代理,当访问不同的域名时,面板可以转发到对应的端口

比如discourse默认是80、443端口,但这两个端口已经被面板占用了,我们把它改成8080、80443端口,当用户访问域名的时候,面板让用户访问正确的8080、80443端口

接下来要将新的文件新的备份都传到Cloudflare R2,并且既往已上传Amazon s3的文件还能正常访问,当然,最好是老数据全部迁移到r2,但我没折腾明白,现在这样也不是不能用(

具体过程(还没写完)

1panel接管discourse端口

关闭discourse自带ssl,改用1panel(btpanel等同理)ssl
app.yml里设置templates相关内容

templates:
# 把Lets Encrypt相关templates全部注释禁用,由1panel(btpanel等同理)完成ssl
  ## Uncomment these two lines if you wish to add Lets Encrypt (https)
  # - "templates/web.ssl.template.yml"
  # - "templates/web.letsencrypt.ssl.template.yml"
    # - "templates/web.socketed.template.yml"
    # - "templates/cloudflare.template.yml"

改用Cloudflare R2

app.yml里设置s3相关内容

# amazon s3 storage
  DISCOURSE_USE_S3: true
  DISCOURSE_S3_REGION: "auto" #alias to auto
  DISCOURSE_S3_INSTALL_CORS_RULE: false #it should be supported
  DISCOURSE_S3_ENDPOINT: https://xxxxxxx.r2.cloudflarestorage.com
  DISCOURSE_S3_ACCESS_KEY_ID: xxxxxx
  DISCOURSE_S3_SECRET_ACCESS_KEY: xxxxx
  # CDN在discourse网站里面设置
  # DISCOURSE_S3_CDN_URL: xxxx
  DISCOURSE_S3_BUCKET: xxxx
  # backup相关功能在discourse网站里面设置
  # DISCOURSE_S3_BACKUP_BUCKET: xxxxx #optional
  # DISCOURSE_BACKUP_LOCATION: xxxxx #optional

将已经存到s3的数据转移到r2

sippy (吸管,嘬嘬嘬小口吸)当访问s3中的内容时才复制过来:Sippy · Cloudflare R2 docs
Super Slurper (超级吸溜侠,吸溜吸溜大口吸) 直接把s3全盘复制过来:Super Slurper · Cloudflare R2 docs

首先需要找到目前使用的s3链接

# 先进入rails控制台
cd /var/discourse
./launcher enter app
rails c

# 随机查看uploads表里的10个链接
Upload.order(Arel.sql('RANDOM()')).limit(10).pluck(:id, :url)

比如我运行以上命令得到结果

(discourse):5:in `<main>': undefined local variable or method `c' for main (NameError)
=> 
[[1344,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.jpeg"],
 [1588,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.jpeg"],
 [2297,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.png"],
 [709,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.jpeg"],
 [1665,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.png"],
 [1447,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.jpeg"],
 [1950,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.png"],
 [2462,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/original/xxxx.png"],
 [1100,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/xxxx.png"],
 [121,
  "//whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/xxxx.png"]]
~

那么我的aws s3链接是whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com
接下来将所有aws s3链接替换成Cloudflare r2链接(我的是discoursecdn.xxx.com

# 先进入discourse app
cd /var/discourse
./launcher enter app

官方建议是通过rake posts:remap完成转换

rake 'posts:remap[whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com,discoursecdn.beginner.center]'

但是不管什么命令,总是提示如下,说明rake posts:remap 失败,我在`meta论坛看到很多类似情况](How to run rake posts:remap with domain? - Support - Discourse Meta)

Remapping
0 posts remapped!

此时可以尝试在rails控制台里操作

# 先进入rails 控制台
cd /var/discourse
./launcher enter app
rails -c
# 用DbHelper.remap命令替换
DbHelper.remap("[whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com](https://whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com/)","discoursecdn.beginner.center")

检验成果

# 在rails控制台里检查 Upload 表中是否有匹配的 URL
Upload.where("url LIKE ?", "%whitewatercn-discourse.s3.dualstack.ap-northeast-1.amazonaws.com%").count

# 再随机找找链接,看看现在用的是什么链接
Upload.order(Arel.sql('RANDOM()')).limit(10).pluck(:id, :url)

参考

2 Likes

这个每月10g是怎么算?每个月10g?还是累计10g?

总共10g

1 Like

应该是每个月 10G。

1 Like

更新了如何将已经存到s3的数据全部转移到r2