今天网站无故被人攻击,我也没在意,想着攻击结束就可以正常访问了,结果,等到晚上的时候,我发现攻击结束了,但是网站却502了。
然后,我在论坛上找解决方案,有人说用“./launcher rebuild app”这个命令,我试了下;结果出现下面的问题,我一看这不是说内存不足吗?
root@kehan:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 88G 84G 0 100% /
Would you like to attempt to recover space by cleaning docker images and containers in the system? (y/N)N
root@kehan:/var/discourse#
root@kehan:/var/discourse#
root@kehan:/var/discourse#
root@kehan:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 88G 84G 0 100% /
Would you like to attempt to recover space by cleaning docker images and containers in the system? (y/N)y
If the cleanup was successful, you may try again now
root@kehan:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed
Please be patient
2.0.20240825-0027: Pulling from discourse/base
Digest: sha256:6de68cb49198b5281f79ed9401b3fe818c854d220dcf0238549fe2f2adb19146
Status: Downloaded newer image for discourse/base:2.0.20240825-0027
docker.io/discourse/base:2.0.20240825-0027
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 88G 84G 0 100% /
我的网站刚建好,内存应该不到300M,怎么会有这么大,刚开始我不是这么想的,因为下午的时候,系统给我发了一封邮件内容如下:
由于达到了 download_remote_images_threshold 的磁盘空间限制,download_remote_images_to_local 设置被禁用。
我当时想的是,谁入侵了我的网站,植入了下载图片的木马,然后,我想的是,可能网站会有一个很大的图片,把那个图片删除就好。
但是我是个菜鸟,不知道用啥命令,然后就在官方论坛一顿搜,看着有用的命令就执行。
后面想到了chatgpt,在chatgpt的帮助下,我终于找到了问题的元凶,就是70G的日志文件,下面是我再chatgpt的指引下执行的命令,还有反馈的结果:
Would you like to attempt to recover space by cleaning docker images and containers in the system? (y/N)y
If the cleanup was successful, you may try again now
root@kehan:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed
Please be patient
2.0.20240825-0027: Pulling from discourse/base
Digest: sha256:6de68cb49198b5281f79ed9401b3fe818c854d220dcf0238549fe2f2adb19146
Status: Downloaded newer image for discourse/base:2.0.20240825-0027
docker.io/discourse/base:2.0.20240825-0027
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 88G 84G 0 100% /
Would you like to attempt to recover space by cleaning docker images and containers in the system? (y/N)y
If the cleanup was successful, you may try again now
root@kehan:/var/discourse#
root@kehan:/var/discourse# ./launcher enter app
x86_64 arch detected.
WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed
Please be patient
2.0.20240825-0027: Pulling from discourse/base
Digest: sha256:6de68cb49198b5281f79ed9401b3fe818c854d220dcf0238549fe2f2adb19146
Status: Downloaded newer image for discourse/base:2.0.20240825-0027
docker.io/discourse/base:2.0.20240825-0027
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 88G 84G 0 100% /
Would you like to attempt to recover space by cleaning docker images and containers in the system? (y/N)y
If the cleanup was successful, you may try again now
root@kehan:/var/discourse#
root@kehan:/var/discourse# ^C
root@kehan:/var/discourse# ^C
root@kehan:/var/discourse# sudo apt install byobu
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
gawk libevent-core-2.1-7 libmpfr6 libsigsegv2 libutempter0 pastebinit python3-distro python3-newt tmux
Suggested packages:
apport ccze gnome-terminal | xterm po-debconf screen speedometer ttf-ubuntu-font-family update-notifier-common vim wireless-tools gawk-doc
Recommended packages:
run-one
The following NEW packages will be installed:
byobu gawk libevent-core-2.1-7 libmpfr6 libsigsegv2 libutempter0 pastebinit python3-distro python3-newt tmux
0 upgraded, 10 newly installed, 0 to remove and 93 not upgraded.
Need to get 2,270 kB of archives.
After this operation, 6,915 kB of additional disk space will be used.
E: You don't have enough free space in /var/cache/apt/archives/.
root@kehan:/var/discourse# sudo purge-old-kernel
sudo: purge-old-kernel: command not found
root@kehan:/var/discourse# ./launcher rebuild app
x86_64 arch detected.
WARNING: We are about to start downloading the Discourse base image
This process may take anywhere between a few minutes to an hour, depending on your network speed
Please be patient
2.0.20240825-0027: Pulling from discourse/base
Digest: sha256:6de68cb49198b5281f79ed9401b3fe818c854d220dcf0238549fe2f2adb19146
Status: Downloaded newer image for discourse/base:2.0.20240825-0027
docker.io/discourse/base:2.0.20240825-0027
You have less than 5GB of free space on the disk where /var/lib/docker is located. You will need more space to continue
Filesystem Size Used Avail Use% Mounted on
/dev/vda1 88G 84G 0 100% /
Would you like to attempt to recover space by cleaning docker images and containers in the system? (y/N)y
If the cleanup was successful, you may try again now
root@kehan:/var/discourse# docker image prune -a
WARNING! This will remove all images without at least one container associated to them.
Are you sure you want to continue? [y/N] N
root@kehan:/var/discourse# du -sh /* | sort -h
du: cannot access '/proc/8758/task/8758/fd/4': No such file or directory
du: cannot access '/proc/8758/task/8758/fdinfo/4': No such file or directory
du: cannot access '/proc/8758/fd/3': No such file or directory
du: cannot access '/proc/8758/fdinfo/3': No such file or directory
0 /bin
0 /dev
0 /initrd.img
0 /initrd.img.old
0 /lib
0 /lib32
0 /lib64
0 /libx32
0 /proc
0 /sbin
0 /sys
0 /vmlinuz
0 /vmlinuz.old
4.0K /home
4.0K /mnt
4.0K /srv
8.0K /media
16K /lost+found
16K /opt
36K /tmp
48K /root
568K /run
3.8M /etc
56M /boot
1.4G /usr
2.1G /swapfile
87G /var
root@kehan:/var/discourse# du -sh /var/* | sort -h
0 /var/lock
0 /var/run
4.0K /var/local
4.0K /var/mail
4.0K /var/opt
12K /var/spool
20K /var/tmp
408K /var/backups
114M /var/log
290M /var/cache
17G /var/lib
70G /var/discourse
root@kehan:/var/discourse# docker system prune -a
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all images without at least one container associated to them
- all build cache
Are you sure you want to continue? [y/N] N
root@kehan:/var/discourse# ^C
root@kehan:/var/discourse# sudo apt-get clean
root@kehan:/var/discourse# du -sh /var/www/discourse/public/uploads/*
du: cannot access '/var/www/discourse/public/uploads/*': No such file or directory
root@kehan:/var/discourse# find /var/www/discourse/public/uploads -type f -exec du -sh {} + | sort -rh
find: ‘/var/www/discourse/public/uploads’: No such file or directory
root@kehan:/var/discourse# ^C
root@kehan:/var/discourse# find /var -type d -name "uploads"
/var/discourse/shared/standalone/uploads
root@kehan:/var/discourse# find /path/to/uploads -type f -exec du -h {} + | sort -rh | head -n 10
find: ‘/path/to/uploads’: No such file or directory
root@kehan:/var/discourse# df -h
Filesystem Size Used Avail Use% Mounted on
udev 962M 0 962M 0% /dev
tmpfs 197M 568K 197M 1% /run
/dev/vda1 88G 83G 279M 100% /
tmpfs 984M 0 984M 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
overlay 88G 83G 279M 100% /var/lib/docker/overlay2/c9d06a2d4a1af6ca0389f6a8098e2ba418373eeeefa997ecab2d4002fd60c703/merged
tmpfs 197M 0 197M 0% /run/user/0
root@kehan:/var/discourse# /var/discourse# du -sh /* | sort -h
-bash: /var/discourse#: No such file or directory
root@kehan:/var/discourse# du -sh /var/* | sort -h
0 /var/lock
0 /var/run
4.0K /var/local
4.0K /var/mail
4.0K /var/opt
12K /var/spool
20K /var/tmp
408K /var/backups
6.6M /var/cache
114M /var/log
17G /var/lib
70G /var/discourse
root@kehan:/var/discourse#
root@kehan:/var/discourse#
root@kehan:/var/discourse#
root@kehan:/var/discourse# du -sh /var/discourse/* | sort -h
4.0K /var/discourse/bin
4.0K /var/discourse/cids
4.0K /var/discourse/LICENSE
8.0K /var/discourse/scripts
12K /var/discourse/discourse-doctor
12K /var/discourse/README.md
20K /var/discourse/containers
24K /var/discourse/launcher
24K /var/discourse/tests
28K /var/discourse/discourse-setup
32K /var/discourse/samples
180K /var/discourse/launcher_go
180K /var/discourse/templates
232K /var/discourse/image
70G /var/discourse/shared
root@kehan:/var/discourse# ^C
root@kehan:/var/discourse# du -sh /var/discourse/shared/* | sort -h
70G /var/discourse/shared/standalone
root@kehan:/var/discourse# du -sh /var/discourse/shared/standalone
70G /var/discourse/shared/standalone
root@kehan:/var/discourse# cd /var/discourse/shared/standalone
du -sh *
5.8M backups
1.8M letsencrypt
70G log
4.0K postgres_backup
110M postgres_data
180K postgres_run
11M redis_data
20K ssl
28K state
12K tmp
11M uploads
上面这一顿乱操作下来,我一刷新,网站竟然可以访问了。
但是问题来了,70G的日志文件可以删吗?
为啥会生成这么大的日志文件,如何禁止被攻击时生成这么大的日志文件?