\u0130\u00e7indekiler<\/strong><\/p>\n Bir ev sunucusunda 120b yerel modelleri \u00e7al\u0131\u015ft\u0131rmak<\/strong> \u00e7al\u0131\u015ft\u0131rmak sadece bir merak eylemi de\u011fildir\u2014ayl\u0131k bulut faturalar\u0131n\u0131 d\u00fc\u015f\u00fcrebilir, tam veri egemenli\u011fi sa\u011flayabilir ve gecikmeyi kendi ihtiya\u00e7lar\u0131n\u0131za g\u00f6re ayarlaman\u0131za izin verir. Bu makale, minimum donan\u0131m, yaz\u0131l\u0131m y\u0131\u011f\u0131n\u0131 ve ger\u00e7ek d\u00fcnya tuzaklar\u0131n\u0131 sizinle payla\u015f\u0131yor, b\u00f6ylece modeli kendi rack’\u0131n\u0131zda g\u00fcvenle \u00e7al\u0131\u015ft\u0131rabilirsiniz.<\/p>\n Elektrik faturas\u0131n\u0131 \u00f6deyen bir homelab operat\u00f6r\u00fc i\u00e7in en b\u00fcy\u00fck \u00e7ekicilik maliyettir. 4\u00d7RTX\u202f3090, 128\u202fGB RAM ve 1\u202fTB NVMe s\u00fcr\u00fcc\u00fcyle donat\u0131lm\u0131\u015f bir rig, y\u00fck alt\u0131nda yakla\u015f\u0131k 1.5\u202fkW t\u00fcketir. Bu, GPU’yu g\u00fcnl\u00fck 8 saat \u00e7al\u0131\u015ft\u0131r\u0131rsan\u0131z yakla\u015f\u0131k $90\/ay<\/strong> elektrik maliyeti demektir. Bulut sa\u011flay\u0131c\u0131lar\u0131 her tahmine birka\u00e7 dolar tahsis eder, ancak \u00f6l\u00e7eklendirdik\u00e7e h\u0131zla artar. Yerel bir da\u011f\u0131t\u0131mda, tek seferlik donan\u0131m faturas\u0131 \u00f6dersiniz ve yaln\u0131zca enerji i\u00e7in marjinal maliyet ta\u015f\u0131r.<\/p>\n Paran\u0131n \u00f6tesinde, modeli yerinde \u00e7al\u0131\u015ft\u0131rmak veri egemenli\u011fi endi\u015felerini ortadan kald\u0131r\u0131r ve an\u0131nda tahmin sa\u011flar. \u00d6rne\u011fin bir felaket m\u00fcdahalesi senaryosunda, a\u011fa ba\u011fl\u0131 gecikme veya veri ka\u00e7ak riskine ihtiya\u00e7 duymazs\u0131n\u0131z.<\/p>\n GPT\u2011OSS\u2011120B 117\u202fB parametreyle gelir, ama tahmin s\u0131ras\u0131nda aktif olan sadece yakla\u015f\u0131k 5,1\u202fB. 3,0\u202fB aktif a\u011f\u0131rl\u0131kla bile tek blokta model tutuyorsan\u0131z yakla\u015f\u0131k 450\u202fGB VRAM<\/strong> gerekir. Bu y\u00fczden 4\u00d7RTX\u202f3090 (her biri 24\u202fGB) temel yap\u0131: her kart bir dilim bar\u0131nd\u0131rabilir, birle\u015fik 96\u202fGB VRAM \u00e7o\u011fu toplu boyut i\u00e7in yeterli. Tek 40\u202fGB’l\u0131k GPU (RTX\u202f4090) de \u00e7al\u0131\u015fabilir, ama titiz dilimleme gerekir.<\/p>\n Linux ger\u00e7ek se\u00e7im. Ubuntu 22.04 LTS ve en yeni CUDA 12.2 ile cuDNN 8.9 s\u00fcr\u00fcc\u00fcleri \u00f6nerilir. 64\u2011bit \u00e7ekirdek, do\u011fru NVIDIA s\u00fcr\u00fcc\u00fcs\u00fc ve Model kontrol noktas\u0131 yakla\u015f\u0131k 20\u202fGB, ama \u00e7al\u0131\u015fma seti ve loglar h\u0131zla b\u00fcy\u00fcyebilir. 1\u202fTB NVMe SSD ideal: kontrol noktas\u0131 y\u00fcklemesi ve d\u00fc\u015f\u00fck gecikmeli disk G\/\u00c7 i\u00e7in yeterli h\u0131z, ayn\u0131 zamanda m\u00fctevaz\u0131 bir kasa i\u00e7inde yer al\u0131r.<\/p>\n A\u015fa\u011f\u0131da, GPT\u2011OSS\u2011120B’nin dakikalar i\u00e7inde \u00e7al\u0131\u015fmas\u0131n\u0131 sa\u011flayan, host’u temiz tutan bir Docker\u2011Compose kurulumunu ad\u0131m ad\u0131m anlatan bir ak\u0131\u015f var.<\/p>\n \u0130pucu:<\/strong> B\u00fct\u00e7e dar ise, bir RTX\u202f3090’\u0131 RTX\u202f4090’a de\u011fi\u015ftirin. 40\u202fGB kart, daha k\u00fc\u00e7\u00fck bir GPU ile i\u015f y\u00fck\u00fcn\u00fc payla\u015fabilir, ancak t\u00fcm \u00e7ekirdekleri me\u015fgul tutmak i\u00e7in daha b\u00fcy\u00fck bir toplu i\u015flem gerekir.<\/p>\n<\/blockquote>\n Sistemi yeniden ba\u015flat\u0131n, nvidia-smi’yi kontrol edin ve her GPU’nun 24\u202fGB bo\u015f VRAM g\u00f6sterdi\u011finden emin olun.<\/p>\n Checkpoint klas\u00f6r\u00fcn\u00fc \/mnt\/data\/gpt-oss-120b alt\u0131nda yerle\u015ftirin. Konteyner, 8080 portunda basit bir REST u\u00e7 noktas\u0131 a\u00e7acakt\u0131r.<\/p>\n Bir ka\u00e7 dakika i\u00e7inde GPU’lar aras\u0131nda b\u00f6l\u00fcnmeyi g\u00f6steren g\u00fcnl\u00fckleri g\u00f6r\u00fcrs\u00fcn\u00fcz. Test etmek i\u00e7in:<\/p>\n Operat\u00f6r notu:<\/strong> 120B’yi yerelde \u00e7al\u0131\u015ft\u0131rmak i\u00e7in minimum donan\u0131m ne zaman yeterli olur?<\/strong> 120B’yi yerelde kurarken en s\u0131k yap\u0131lan hata nedir?<\/strong> \u0130lgili ba\u011flam i\u00e7in 120b yerel modelleri \u00e7al\u0131\u015ft\u0131rmak i\u00e7in minimum gereksinimler rehberi<\/a> ve openai gpt oss 120b’yi yerelde \u00e7al\u0131\u015ft\u0131rma<\/a> inceleyin.<\/p>\n GPT\u2011OSS\u2011120B’yi ev sunucunuzda \u00e7al\u0131\u015ft\u0131rmak i\u00e7in tek seferlik $5,000<\/strong>‘dan az bir yat\u0131r\u0131m gerekiyor, bu da bulut tahminine g\u00f6re ayl\u0131k yakla\u015f\u0131k 90 $ elektrik tasarrufu anlam\u0131na gelir. A\u00e7\u0131k bir donan\u0131m kontrol listesi, basit bir Docker kurulum ve birka\u00e7 performans ayar\u0131 ile 120B modeli ara\u015ft\u0131rma, otomasyon ve deney i\u00e7in ger\u00e7ek\u00e7i bir ara\u00e7 haline gelir.<\/p>\n\n
120b yerel modelleri \u00e7al\u0131\u015ft\u0131rmak: Neden \u00d6nemli<\/h2>\n
Ana Kavramlar<\/h2>\n
Model Boyutu ve Bellek<\/h3>\n
\u0130\u015fletim Sistemi ve S\u00fcr\u00fcc\u00fcler<\/h3>\n
nvidia-smi<\/code> \u00e7\u0131kt\u0131s\u0131n\u0131n GPU\u2019lar\u0131 do\u011fru g\u00f6sterdi\u011finden emin olun.<\/p>\nDisk<\/h3>\n
Pratik \u00d6rnekler ve En \u0130yi Uygulamalar<\/h2>\n
1. Rack’i Kurun<\/h3>\n
\n\n
\n \nBile\u015fen<\/th>\n \u00d6rnek<\/th>\n Tahmini Maliyet<\/th>\n<\/tr>\n<\/thead>\n \n 4 \u00d7 NVIDIA RTX\u202f3090<\/td>\n RTX\u202f3090 24\u202fGB<\/td>\n $3,600<\/td>\n<\/tr>\n \n 128\u202fGB DDR5 RAM<\/td>\n 4 \u00d7 32\u202fGB<\/td>\n $400<\/td>\n<\/tr>\n \n 1\u202fTB NVMe SSD<\/td>\n Samsung 970 EVO Plus<\/td>\n $150<\/td>\n<\/tr>\n \n Dual\u2011CPU anakart<\/td>\n ASRock Rack EP4024P2<\/td>\n $200<\/td>\n<\/tr>\n \n 750\u202fW PSU<\/td>\n EVGA SuperNOVA 750 G5<\/td>\n $130<\/td>\n<\/tr>\n \n Kas ve So\u011futma<\/td>\n Supermicro CSE\u2011701-AT<\/td>\n $120<\/td>\n<\/tr>\n \n Toplam<\/strong><\/td>\n <\/td>\n $4,700<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n \n
2. \u0130\u015fletim Sistemi ve S\u00fcr\u00fcc\u00fcler Kurulumu<\/h3>\n
sudo apt update && sudo apt upgrade -y\nsudo apt install build-essential dkms\n# Add NVIDIA repo\nwget https:\/\/developer.download.nvidia.com\/compute\/cuda\/repos\/ubuntu2204\/x86_64\/cuda-ubuntu2204.pin\nsudo mv cuda-ubuntu2204.pin \/etc\/apt\/preferences.d\/cuda-repository-pin-600\nsudo apt-key adv --fetch-keys https:\/\/developer.download.nvidia.com\/compute\/cuda\/repos\/ubuntu2204\/x86_64\/7fa2af80.pub\nsudo add-apt-repository \"deb https:\/\/developer.download.nvidia.com\/compute\/cuda\/repos\/ubuntu2204\/x86_64\/ \/\"\nsudo apt update\nsudo apt install cuda-12-2\n<\/code><\/pre>\n3. Modeli ve Docker G\u00f6r\u00fcnt\u00fcs\u00fcn\u00fc \u00c7ekme<\/h3>\n
sudo apt install docker.io docker-compose\nsudo systemctl enable --now docker\ndocker pull ghcr.io\/openai\/gpt-oss-120b:latest\n<\/code><\/pre>\n4. docker-compose.yml Olu\u015fturma<\/h3>\n
version: \"3.9\"\nservices:\n gpt-oss-120b:\n image: ghcr.io\/openai\/gpt-oss-120b:latest\n deploy:\n resources:\n reservations:\n devices:\n - driver: nvidia\n count: all\n capabilities: [gpu]\n environment:\n - BATCH_SIZE=8\n - MAX_SEQ_LEN=1024\n volumes:\n - \/mnt\/data\/gpt-oss-120b:\/model\n ports:\n - \"8080:8080\"\n<\/code><\/pre>\n5. Ba\u015flatma<\/h3>\n
docker-compose up -d\n<\/code><\/pre>\ncurl -X POST http:\/\/localhost:8080\/v1\/chat\/completions \\\n -H \"Content-Type: application\/json\" \\\n -d '{\"model\":\"gpt-oss-120b\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello.\"}],\"max_tokens\":20}'\n<\/code><\/pre>\n6. Performans\u0131 Ayarlama<\/h3>\n
\n\n
\n \nAyar<\/th>\n Etkisi<\/th>\n<\/tr>\n<\/thead>\n \n BATCH_SIZE’i azalt\u0131n<\/td>\n VRAM kullan\u0131m\u0131n\u0131 d\u00fc\u015f\u00fcr\u00fcr, gecikmeyi art\u0131r\u0131r<\/td>\n<\/tr>\n \n MAX_SEQ_LEN’i art\u0131r\u0131n<\/td>\n Daha fazla ba\u011flam, daha fazla bellek<\/td>\n<\/tr>\n \n TF32’yi etkinle\u015ftirin<\/td>\n RTX\u202f3090’da hafif h\u0131zlanma, bellek \u00fczerinde g\u00f6z ard\u0131 edilebilir etki<\/td>\n<\/tr>\n \n NVLink k\u00f6pr\u00fclerini kullan\u0131n<\/td>\n GPU’lar aras\u0131 bant geni\u015fli\u011fini art\u0131r\u0131r, daha b\u00fcy\u00fck par\u00e7alar i\u00e7in faydal\u0131d\u0131r<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n S\u0131kla\u015f\u0131lan Hatalar & Sorun Giderme<\/h2>\n
\n\n
\n \nBelirti<\/th>\n Muhtemel Neden<\/th>\n \u00c7\u00f6z\u00fcm<\/th>\n<\/tr>\n<\/thead>\n \n CUDA out of memory<\/code><\/td>\nYararlan\u0131labilir VRAM i\u00e7in paket \u00e7ok b\u00fcy\u00fck<\/td>\n BATCH_SIZE<\/code> de\u011ferini d\u00fc\u015f\u00fcr\u00fcn veya gradyan kontrol noktas\u0131n\u0131 etkinle\u015ftirin<\/td>\n<\/tr>\n\n driver version mismatch<\/code><\/td>\nDocker imaj\u0131 eski CUDA ile olu\u015fturulmu\u015f<\/td>\n Host CUDA’y\u0131 konteynerle e\u015fle\u015ftirin veya daha yeni bir imaj \u00e7ekin<\/td>\n<\/tr>\n \n Yava\u015f \u00e7\u0131kar\u0131m<\/td>\n Disk I\/O darbo\u011faz\u0131<\/td>\n Kontrol noktas\u0131n\u0131 NVMe’ye ta\u015f\u0131y\u0131n veya takas alan\u0131n\u0131 art\u0131r\u0131n<\/td>\n<\/tr>\n \n Ba\u015flang\u0131\u00e7ta rastgele \u00e7\u00f6kmeler<\/td>\n Eksik b\u00f6lme<\/td>\n nvidia-smi<\/code> t\u00fcm GPU’lar\u0131 g\u00f6sterdi\u011finden docker-compose up<\/code> \u00f6ncesinde do\u011frulay\u0131n<\/td>\n<\/tr>\n\n \u00c7\u0131kar\u0131m s\u0131ras\u0131nda y\u00fcksek CPU kullan\u0131m\u0131<\/td>\n CPU, Python yorumlay\u0131c\u0131s\u0131 taraf\u0131ndan darbo\u011fazlan\u0131yor<\/td>\n Kodda torch.no_grad()<\/code> ve torch.backends.cudnn.benchmark=True<\/code> kullan\u0131n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nnvidia-smi<\/code> g\u00fcnl\u00fcklerini izleyin; bellek kullan\u0131labilirli\u011finde ani d\u00fc\u015f\u00fc\u015f genellikle OOM \u00e7\u00f6kmesi \u00f6ncesinde gelir.<\/p>\nFAQ<\/h2>\n
\n
\n 4\u00d7RTX\u202f3090, 128\u202fGB RAM ve 1\u202fTB NVMe temel donan\u0131m. Sadece 3 GPU veya 96\u202fGB RAM varsa, toplama boyutunu k\u00fc\u00e7\u00fcltmeniz gerekir; b\u00f6lmeleri daha agresif b\u00f6lmelisiniz.<\/p>\n<\/li>\n
\n GPU bellek par\u00e7alanmas\u0131n\u0131 g\u00f6z ard\u0131 edip modeli tek bir karta y\u00fcklemeye \u00e7al\u0131\u015fmak. Her zaman modeli t\u00fcm GPU’lar aras\u0131nda b\u00f6l\u00fcn ve toplama boyutunu, tek bir GPU’nun haf\u0131za s\u0131n\u0131rlar\u0131 i\u00e7inde kalacak \u015fekilde d\u00fc\u015f\u00fck tutun.<\/p>\n<\/li>\n<\/ul>\nSonu\u00e7<\/h2>\n