Linux OS buffer tuning for dirty blocks (for Databases/Application Server)

環境 Redhat 5.5 , memory 64GB. 跑的是timesten 資料庫

客戶反映系統告警 ( free memory 不足 , 只剩2GB )

使用以下free -m 指令發現OS buffer使用過多(21GB) , 與其他設備比較相對占用較多緩存

[root@db1 log]# free -m

total used free shared buffers cached

Mem: 64429 61744 2685 0 752 21638

-/+ buffers/cache: 39352 25077

Swap: 65538 0 65538

[root@db1 log]# echo 1 > /proc/sys/vm/drop_caches ==> 要求OS kernel捨棄沒在使用的 cache

[root@db1 log]# free -m

total used free shared buffers cached

Mem: 64429 44140 20289 0 13 5272

-/+ buffers/cache: 38854 25575

Swap: 65538 0 65538

[root@db1 log]#

後來發現以上的做法只是暫解, 如果要一勞永逸須調整Linux kernel vm.dirty_ratio 參數, 相關檔案在 /etc/sysctl.conf

參考資料：

http://blog.chrisgreenough.com/2012/01/oracle-redhat-vm-swappiness-and-huge-pages/

http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html/Deployment_Guide/s3-proc-sys-vm.html

--

Made Kernel changes so dirty pages in memory are written out more frequently

vm.dirty_background_ratio = 3  ==>使用背景daemon pdflush 去 flush回硬碟  (default 10%)

vm.dirty_ratio = 15    ==>使用總系統多少比率的dirty,就會flush回硬碟 (default  40%)

vm.dirty_expire_centisecs = 500  ==>資料可以保持為dirty狀態的最大毫秒數(ms), default (3000ms , 30s)

vm.dirty_writeback_centisecs = 100  ==>資料多久寫回(default 500ms)

Changed the kernel parameter that determines how aggressive swapping is done. Set to 0 so swapping is only done when needed.

vm.swappiness = 1  ==> default 60 , Range from 0 to 100. Lower values mean pages remain longer in memory before swapping them out to disk. Larger values swap out infrequently used memory pages faster. The default value is “60″.

Configured hugepages for Oracle which makes the memory pagesize larger for Oracle processes (this memory is not swappable)

vm.nr_hugepages = 6656  ==> number of huge pages

vm.hugetlb_shm_group = 501  ==> contains group id that is allowed to create SysV shared memory segment using hugetlb page , 在此範例 group id 為 501

如果沒設定 可能遇到如下問題
 : http://www.eygle.com/archives/2011/12/hugepageshugetl.html

客戶解決一個Linux上資料庫無法啟動的問題。
客戶的Linux 5.6 x86-64環境，安裝資料庫後，啟動資料庫報錯：ORA-27125。
Oracle文檔上關於ORA-27125錯誤的描述為：

ORA-27125: unable to create shared memory segment
Cause: shmget() call failed
Action: contact Oracle support

ulimit settings for Oracle:

oracle soft memlock 13631488

oracle hard hemlock 13631488

心得:
如果使用大記憶體的linux  , vm.dirty_ratio 預設40% 是相當不合理的, 且會搶走Application/DB的記憶體.意思是說 OS 會用OS buffer , DB 也用DB BUFFER 造成無謂浪費~