本文共 6165 字,大约阅读时间需要 20 分钟。
安装与优化Linux系统(基于个人实践)
作为一名从事数据库运维的开发者,我在日常工作中经常需要对Linux系统进行优化。以下是我对系统优化的实践经验和步骤分享。
一、安装常用包在安装必要软件包之前,我会使用以下命令:
yum -y install coreutils glib2 lrzsz mpstat dstat sysstat e4fsprogs xfsprogs ntp readline-devel zlib-devel openssl-devel pam-devel libxml2-devel libxslt-devel python-devel tcl-devel gcc make smartmontools flex bison perl-devel perl-ExtUtils* openldap-devel jadetex openjade bzip2
二、配置OS内核参数内核参数的配置需要谨慎,每个参数都有其意义。以下是我常用的配置:
# /etc/sysctl.confkernel.core_pattern=/data01/corefiles/core_%e_%u_%t_%s.%pkernel.sem=4096 2147483647 2147483646 512000kernel.shmall=107374182kernel.shmmax=274877906944kernel.shmmni=819200net.core.netdev_max_backlog=10000net.core.rmem_default=262144net.core.rmem_max=4194304net.core.wmem_default=262144net.core.wmem_max=4194304net.core.somaxconn=4096net.ipv4.tcp_max_syn_backlog=4096net.ipv4.tcp_keepalive_intvl=20net.ipv4.tcp_keepalive_probes=3net.ipv4.tcp_keepalive_time=60net.ipv4.tcp_mem=8388608 12582912 16777216net.ipv4.tcp_fin_timeout=5net.ipv4.tcp_synack_retries=2net.ipv4.tcp_syncookies=1net.ipv4.tcp_timestamps=1net.ipv4.tcp_tw_recycle=0net.ipv4.tcp_tw_reuse=1net.ipv4.tcp_max_tw_buckets=262144net.ipv4.tcp_rmem=8192 87380 16777216net.ipv4.tcp_wmem=8192 65536 16777216net.nf_conntrack_max=1200000net.netfilter.nf_conntrack_max=1200000vm.dirty_background_bytes=409600000vm.dirty_expire_centisecs=3000vm.dirty_ratio=95vm.dirty_writeback_centisecs=100vm.mmap_min_addr=65536vm.overcommit_memory=0vm.overcommit_ratio=90vm.swappiness=0vm.zone_reclaim_mode=0net.ipv4.ip_local_port_range=40000 65535fs.nr_open=20480000
配置完成后,使用以下命令生效:
sysctl -p
三、配置OS资源限制资源限制对于性能至关重要。以下是我推荐的配置:
# /etc/security/limits.confnofile.* soft nofile 1024000nofile.* hard nofile 1024000nproc.* soft unlimitednproc.* hard unlimitedcore.* soft unlimitedcore.* hard unlimitedmemlock.* soft unlimitedmemlock.* hard unlimited
注意:请检查 /etc/security/limits.d 目录中的文件配置,确保没有冲突。
四、配置OS防火墙清除所有规则并开放管理网络:
iptables -Fiptables -A INPUT -s 192.168.0.0/16 -j ACCEPTiptables -A INPUT -s 10.0.0.0/8 -j ACCEPTiptables -A INPUT -s 172.16.0.0/16 -j ACCEPT
五、selinux配置如果不需要安全增强,建议禁用:
# /etc/sysconfig/selinuxSELINUX=disabledSELINUXTYPE=targeted
六、关闭不必要的OS服务使用以下命令关闭不必要的服务:
chkconfig --list | grep on# 例如:chkconfig iscsi off
七、部署文件系统建议使用ext4文件系统,并对SSD进行对齐:
parted -s /dev/sda mklabel gptparted -s /dev/sda mkpart primary 1MiB 100%
格式化文件系统:
mkfs.ext4 /dev/sda1 -m 0 -O extent,uninit_bg -E lazy_itable_init=1 -T largefile -L u01
建议使用以下挂载选项:
# /etc/fstabLABEL=u01 /u01 ext4 defaults,noatime,nodiratime,nodelalloc,barrier=0,data=writeback 0 0
为什么需要 data=writeback?这是因为 data=writeback 提高了写入性能,适合高IO负载的应用。
八、设置SSD盘的调度如果使用SSD,建议设置为deadline调度:
echo deadline > /sys/block/sda/queue/scheduler
永久设置可以通过修改grub.conf:
vi /boot/grub.confelevator=deadline
九、关闭透明大页和numa建议在grub.conf中设置:
vi /boot/grub.confelevator=deadline numa=off transparent_hugepage=never
十、编译器配置建议使用较新的编译器版本:
# 安装gcc 6.2.0cd ~tar -jxvf gcc6.2.0.tar.bz2tar -jxvf python2.7.12.tar.bz2vi /etc/ld.so.conf
十一、环境变量配置我通常会使用这样的脚本:
vi ~/env_pg.shexport PS1="$USER@`/bin/hostname -s`-"export PGPORT=$1export PGDATA=/$2/digoal/pg_root$PGPORTexport LANG=en_US.utf8export PGHOME=/home/digoal/pgsql9.6export LD_LIBRARY_PATH=/home/digoal/gcc6.2.0/lib:/home/digoal/gcc6.2.0/lib64:/home/digoal/python2.7.12/lib:$PGHOME/lib:/lib64:/usr/lib64:/lib:/usr/lib:/usr/local/lib64:/usr/local/libexport PATH=/home/digoal/gcc6.2.0/bin:/home/digoal/python2.7.12/bin:/home/digoal/cmake3.6.3/bin:$PGHOME/bin:$PATH:. export DATE=`date +"%Y%m%d%H%M"`export MANPATH=$PGHOME/share/man:$MANPATHexport PGHOST=$PGDATAexport PGUSER=postgresexport PGDATABASE=postgresalias rm='rm -i'alias ll='ls -lh'unalias vi
十二、编译PostgreSQL我通常会使用以下命令:
cd postgresql-9.6.1export USE_NAMED_POSIX_SEMAPHORES=1LIBS=-lpthread CC="/home/digoal/gcc6.2.0/bin/gcc" CFLAGS="-O3 -flto" ./configure --prefix=/home/digoal/pgsql9.6make world -j 64
如果是开发环境,可以考虑调试版本:
CD postgresql-9.6.1export USE_NAMED_POSIX_SEMAPHORES=1LIBS=-lpthread CC="/home/digoal/gcc6.2.0/bin/gcc" CFLAGS="-O0 -flto -g -ggdb -fno-omit-frame-pointer" ./configure --prefix=/home/digoal/pgsql9.6 --enable-cassertmake world -j 64
十三、初始化数据库集群我会将pg_xlog放在独立的IOPS设备上:
. ~/env_pg.sh 1921 u01initdb -D $PGDATA -E UTF8 --locale=C -U postgres -X /u02/digoal/pg_xlog$PGPORT
十四、配置postgresql.conf针对内存为512GB的主机:
vi postgresql.conflisten_addresses='0.0.0.0'port=1921max_connections=5000unix_socket_directories='.'tcp_keepalives_idle=60tcp_keepalives_interval=10tcp_keepalives_count=10shared_buffers=128GBmaintenance_work_mem=2GBdynamic_shared_memory_type=posixvacuum_cost_delay=0bgwriter_delay=10msbgwriter_lru_maxpages=1000bgwriter_lru_multiplier=10.0bgwriter_flush_after=0max_worker_processes=128max_parallel_workers_per_gather=0old_snapshot_threshold=-1backend_flush_after=0wal_level=replicasynchronous_commit=offfull_page_writes=onwal_buffers=1GBwal_writer_delay=10mswal_writer_flush_after=0checkpoint_timeout=30minmax_wal_size=256GBmin_wal_size=64GBcheckpoint_completion_target=0.05checkpoint_flush_after=0archive_mode=onarchive_command='/bin/date'max_wal_senders=8random_page_cost=1.3parallel_tuple_cost=0parallel_setup_cost=0min_parallel_relation_size=0effective_cache_size=300GBforce_parallel_mode=offlog_destination='csvlog'logging_collector=onlog_truncate_on_rotation=onlog_checkpoints=onlog_connections=onlog_disconnections=onlog_error_verbosity=verboselog_timezone='PRC'vacuum_defer_cleanup_age=0hot_standby_feedback=offmax_standby_archive_delay=300smax_standby_streaming_delay=300sautovacuum=onlog_autovacuum_min_duration=0autovacuum_max_workers=16autovacuum_naptime=45sautovacuum_vacuum_scale_factor=0.1autovacuum_analyze_scale_factor=0.1autovacuum_freeze_max_age=1600000000autovacuum_multixact_freeze_max_age=1600000000vacuum_freeze_table_age=1500000000vacuum_multixact_freeze_table_age=1500000000datestyle='iso, mdy'timezone='PRC'lc_messages='C'lc_monetary='C'lc_numeric='C'lc_time='C'default_text_search_config='pg_catalog.english'shared_preload_libraries='pg_stat_statements'
十五、配置pg_hba.conf我倾向于使用密码验证:
vi pg_hba.confhost replication xx 0.0.0.0/0 md5# 流复制host all postgres 0.0.0.0/0 reject# 拒绝超级用户从网络登录host all all 0.0.0.0/0 md5# 其他用户登录
十六、启动数据库使用以下命令启动:
pg_ctl start
以上就是我在实际工作中对Linux系统的优化步骤和配置经验。希望这些内容能为您的系统优化提供帮助。
转载地址:http://qnxfk.baihongyu.com/