Ollama, vLLM, llama.cpp<\/strong> \u00e7al\u0131\u015ft\u0131rman\u0131n ger\u00e7e\u011fi, d\u00fc\u015fen verim, termal darbelendirme ve c\u00fczdan\u0131n\u0131z\u0131 token \u00fcretmekten daha h\u0131zl\u0131 eriten bant geni\u015fli\u011fi darbo\u011fazlar\u0131yla dolu bir oyundur. G\u00fc\u00e7 da\u011f\u0131t\u0131m \u00fcniteleri veya so\u011futma y\u00fckseltmeleri i\u00e7in bir kuru\u015f daha harcamadan \u00f6nce, PCIe bus’un bir otoyol de\u011fil, bir darbo\u011faz oldu\u011funu anlaman\u0131z gerekir.<\/p>\n\u0130\u015fletme donan\u0131m\u0131n\u0131za ikinci bir GPU eklemek do\u011frusal bir y\u00fckseltme de\u011fildir. Yeni ar\u0131za modlar\u0131n\u0131 devreye sokan karma\u015f\u0131k bir entegrasyon g\u00f6revidir. Veri merkezlerinde NVLink bu a\u00e7\u0131\u011f\u0131 kapat\u0131r; ancak homelab ortam\u0131nda PCIe Gen3 veya Gen4 \u015feritleri verinin s\u00fcrekli geri ve ileri kar\u0131\u015ft\u0131r\u0131lmas\u0131n\u0131 zorunlu k\u0131lar, bu da ikinci kart\u0131 varl\u0131ktan ziyade bir y\u00fck haline getirir. 70 milyar parametreli bir modeli iki adet 12GB’l\u0131k kart \u00fczerinde \u00e7al\u0131\u015ft\u0131r\u0131yor veya RTX 3090’lar \u00fczerinde 48GB VRAM’i yayarak devasa bir ba\u011flam penceresini s\u0131k\u0131\u015ft\u0131rmaya \u00e7al\u0131\u015f\u0131yor olabilirsiniz. Bu k\u0131s\u0131tlamalar\u0131n fizi\u011fi, hangi motoru \u00e7al\u0131\u015ft\u0131rman\u0131z gerekti\u011fini belirler.<\/p>\n
Bu k\u0131lavuz, “bulut-native” benchmarklar\u0131n\u0131n hype’inden s\u0131yr\u0131l\u0131r. Elektrik faturas\u0131na, oturma odan\u0131zdaki fan g\u00fcr\u00fclt\u00fcs\u00fcne ve ev internet ba\u011flant\u0131s\u0131 \u00fczerinde 7\/24 \u00e7al\u0131\u015fan bir sistemin istikrar\u0131na bak\u0131yoruz. Ollama, vLLM veya llama.cpp basitli\u011finden ziyade, vLLM’in ham i\u015fleyi\u015f kapasitesini veya llama.cpp’nin gran\u00fcler kontrol\u00fcn\u00fc se\u00e7seniz bile, se\u00e7iminiz fiziksel donan\u0131m topolojinizle uyumlu olmal\u0131d\u0131r.<\/p>\n
Motorlar\u0131n katmanlar\u0131 nas\u0131l b\u00f6ld\u00fc\u011f\u00fcn\u00fc, bellek bant geni\u015fli\u011finde nerede t\u0131kanacaklar\u0131n\u0131 ve fanlar %100 h\u0131zda d\u00f6nd\u00fc\u011f\u00fcnde sistemde RAM’e takas yapmaya ba\u015flad\u0131\u011f\u0131nda neler olaca\u011f\u0131n\u0131 detayland\u0131raca\u011f\u0131z. Bu teorik bir genel bak\u0131\u015f de\u011fil, elektrik faturas\u0131n\u0131 \u00f6deyen operat\u00f6r i\u00e7in bir hayatta kalma k\u0131lavuzudur.<\/p>\n
Ollama vllm llama cpp: \u00c7oklu GPU Ger\u00e7ekli\u011fi: Bant Geni\u015fli\u011fi T\u0131kan\u0131kl\u0131klar\u0131 ve Enerji Maliyetleri<\/h2>\n
\u0130kinci bir ekran kart\u0131 sat\u0131n ald\u0131\u011f\u0131n\u0131zda, sadece bellek sat\u0131n alm\u0131yorsunuz; ayn\u0131 zamanda \u0131s\u0131, g\u00fc\u00e7 t\u00fcketimi ve \u00e7ok daha karma\u015f\u0131k bir sistem mimarisi de sat\u0131n al\u0131yorsunuz. Homelab y\u00f6neticilerinin yapt\u0131\u011f\u0131 en yayg\u0131n hata, ayn\u0131 modeli \u00e7al\u0131\u015ft\u0131ran iki GPU’nun performans\u0131n iki kat\u0131n\u0131 verece\u011fini varsaymakt\u0131r. B\u00fcy\u00fck Dil Modeli (LLM) \u00e7\u0131kar\u0131m\u0131 d\u00fcnyas\u0131nda, bu varsay\u0131m NVLink ve devasa y\u0131\u011f\u0131n boyutlar\u0131 i\u00e7eren \u00e7ok spesifik ko\u015fullar d\u0131\u015f\u0131nda neredeyse hi\u00e7bir zaman do\u011fru de\u011fildir.<\/p>\n
PCIe Bant Geni\u015fli\u011fi T\u0131kan\u0131kl\u0131\u011f\u0131<\/h3>\n
Konsumeri \u00e7oklu GPU kurulumlar\u0131ndaki temel k\u0131s\u0131tlama, Peripheral Component Interconnect (PCIe) veri yolud\u0131r. Modern t\u00fcketici anakartlar\u0131 genellikle PCIe Gen4 x16 yollar\u0131 sunar, ancak bunlar bile b\u00fcy\u00fck modellerin \u00e7\u0131kar\u0131m\u0131 s\u0131ras\u0131nda gereken veri hareketi i\u00e7in yetersizdir. Model iki GPU’ya b\u00f6l\u00fcnd\u00fc\u011f\u00fcnde, kat\u0131klar s\u00fcrekli ileti\u015fim kurmak zorunda kal\u0131r. \u0130leri pass (forward pass) s\u0131ras\u0131nda GPU 0’dan \u00e7\u0131kan \u00e7\u0131kt\u0131, GPU 1 i\u00e7in girdi olur. Geri d\u00f6n\u00fc\u015f (backward pass) veya ba\u011flam olu\u015fturma s\u0131ras\u0131nda ise verinin geri akmas\u0131 gerekir.<\/p>\n
\u0130ki RTX 3090’\u0131n\u0131z oldu\u011funu d\u00fc\u015f\u00fcn\u00fcn. Her birinde 24GB VRAM var. 70B parametreli bir modeli y\u00fcklersiniz. Model tek bir kartta s\u0131\u011fmaz. \u00c7\u0131kar\u0131m motoru kat\u0131klar\u0131 b\u00f6lmek zorundad\u0131r. GPU 0 0-20. katmanlar\u0131, GPU 1 ise 21-50. katmanlar\u0131 i\u015fler. Model bir token \u00fcretti\u011finde, GPU 0 hesaplamas\u0131n\u0131 tamamlar ve sonucu PCIe veri yolu \u00fczerinden GPU 1’e g\u00f6ndermek zorundad\u0131r. PCIe bant geni\u015fli\u011finiz doymu\u015fsa, GPU 1 veriyi bekleyerek at\u0131l durumdad\u0131r. Buna “PCIy stall” (PCIy tak\u0131lmas\u0131) denir.<\/p>\n
Ni\u015fan bir tek GPU \u00e7al\u0131\u015ft\u0131rmada veri hi\u00e7 VRAM’den \u00e7\u0131kmaz. \u00c7oklu GPU PCIe \u00e7al\u0131\u015ft\u0131rmada ise, \u00fcretilen her token i\u00e7in veri anakart \u00fczerinde defalarca dola\u015f\u0131r. 70B’lik bir model i\u00e7in veri hareketi b\u00fcy\u00fckt\u00fcr. PCIe Gen3 x16 ile teorik bant geni\u015fli\u011fi yakla\u015f\u0131k 16 GB\/s’dir. Ger\u00e7ek d\u00fcnya verimlili\u011fi genellikle protokol y\u00fck\u00fc nedeniyle daha d\u00fc\u015f\u00fckt\u00fcr. E\u011fer \u00e7\u0131kar\u0131m motorunuz 20 GB\/s ara GPU trafi\u011fi gerektiriyorsa, hesaplamay\u0131 bile ba\u015flatmadan \u00f6nce darbo\u011faza tak\u0131l\u0131rs\u0131n\u0131z. Bu da standart bir t\u00fcketici anakart\u0131na ikinci bir GPU eklemenin, umdu\u011funuz 2.0x h\u0131zlanman\u0131n yerine yaln\u0131zca 1.3x ile 1.5x h\u0131zlanma sa\u011flayabilece\u011fi anlam\u0131na gelir.<\/p>\n
G\u00fc\u00e7 ve Is\u0131 Denklemi<\/h3>\n
Bunu muhtemelen bulut \u00fccretleri \u00f6demeden daha b\u00fcy\u00fck modelleri yerel olarak \u00e7al\u0131\u015ft\u0131rmak istedi\u011finiz i\u00e7in yap\u0131yorsunuz. Ancak g\u00fc\u00e7 maliyetini hesaba katmal\u0131s\u0131n\u0131z. Tam y\u00fck alt\u0131nda modern bir GPU 250W ile 350W aras\u0131nda g\u00fc\u00e7 \u00e7ekebilir. \u0130kisini \u00e7al\u0131\u015ft\u0131rmak, ev y\u00fck\u00fcn\u00fcze 500W ile 700W eklemek demektir. 24 saatlik bir d\u00f6nemde bu, ek olarak 12 ile 16.8 kWh anlam\u0131na gelir. Yerel elektrik oranlar\u0131n\u0131za ba\u011fl\u0131 olarak bu, ayl\u0131k \u00f6nemli bir giderdir.<\/p>\n
Evinizdeki elektrik oran\u0131 d\u00fc\u015f\u00fcktse, diyelim ki kWh ba\u015f\u0131na 0.10$, iki GPU’lu bir kurulumu \u00e7al\u0131\u015ft\u0131rmak i\u00e7in ayl\u0131k 50$ ile 70$ aras\u0131 ek \u00fccret \u00f6d\u00fcyor olabilirsiniz. E\u011fer California veya Avrupa’n\u0131n baz\u0131 b\u00f6lgeleri gibi daha y\u00fcksek oranl\u0131 bir yerde ya\u015f\u0131yorsan\u0131z, bu rakam ikiye katlanabilir. \u00dcstelik g\u00fc\u00e7 denklemin sadece yar\u0131s\u0131d\u0131r; di\u011fer yar\u0131s\u0131 \u0131s\u0131d\u0131r.<\/p>\n
K\u00fc\u00e7\u00fck bir kasada (standart bir 1U veya 2U sunucu kasas\u0131nda veya hatta modifiye edilmi\u015f bir ATX kulede) iki GPU, bir \u0131s\u0131 adas\u0131 yarat\u0131r. Kasan\u0131n i\u00e7indeki hava \u00e7ok \u00e7abuk \u0131s\u0131n\u0131r. \u00c7o\u011fu t\u00fcketici ekran kart\u0131n\u0131n maksimum s\u0131cakl\u0131k limiti 83\u00b0C ile 90\u00b0C aras\u0131ndad\u0131r. Bu e\u015fi\u011fe ula\u015ft\u0131klar\u0131nda h\u0131z d\u00fc\u015f\u00fcr\u00fcrler (throttle). H\u0131z d\u00fc\u015f\u00fcrme, her tokenin \u00fcretimi i\u00e7in gereken s\u00fcreyi art\u0131r\u0131r, bu da daha fazla<\/em> \u0131s\u0131 \u00fcretir. Sonu\u00e7 olarak, donan\u0131m\u0131n kendini so\u011futmak i\u00e7in daha fazla \u00e7al\u0131\u015ft\u0131\u011f\u0131 ve performans\u0131 daha da k\u00f6t\u00fcle\u015ftirdi\u011fi bir olumsuz geri besleme d\u00f6ng\u00fcs\u00fcnde olursunuz.<\/p>\nBunu daha fazla fanla \u00e7\u00f6zebilece\u011finizi d\u00fc\u015f\u00fcnebilirsiniz, ancak h\u0131zla ses s\u0131n\u0131rlar\u0131na tak\u0131l\u0131rs\u0131n\u0131z. \u0130ki 3090’\u0131 %100 fan h\u0131z\u0131nda \u00e7al\u0131\u015ft\u0131rmak, u\u00e7a\u011f\u0131n kalk\u0131\u015f yapt\u0131\u011f\u0131 gibi bir g\u00fcr\u00fclt\u00fc yarat\u0131r. Evinizde veya ofisinizde bu g\u00fcr\u00fclt\u00fcy\u00fc tolere edemiyorsan\u0131z, kartlar\u0131 a\u015f\u0131nd\u0131rmak (underclock) veya gerilimi d\u00fc\u015f\u00fcrmek (undervolt) zorunda kal\u0131rs\u0131n\u0131z. Gerilim d\u00fc\u015f\u00fcrme, manuel ayarlama gerektirir ve yanl\u0131\u015f yap\u0131l\u0131rsa karars\u0131zl\u0131\u011fa yol a\u00e7abilir.<\/p>\n
Neden “Daha Fazla GPU” Her Zaman “Daha H\u0131zl\u0131” De\u011fildir<\/h3>\n
“Daha fazla GPU daha h\u0131zl\u0131d\u0131r” s\u00f6z\u00fc, pazarlama departmanlar\u0131 taraf\u0131ndan sat\u0131lan bir yaland\u0131r. Ger\u00e7eklik daha incelidir. Bir GPU eklemek kapasiteyi (\u00e7al\u0131\u015ft\u0131rabilece\u011finiz model boyutunu) art\u0131r\u0131r, ancak yukar\u0131da belirtilen y\u00fcklerden dolay\u0131 verimlili\u011fi (token alma h\u0131z\u0131n\u0131) genellikle d\u00fc\u015f\u00fcr\u00fcr.<\/p>\n
Belirli bir senaryoya bakal\u0131m. 13B parametreli bir modeliniz var. Bu, tek bir 12GB kart\u0131n rahat\u00e7a s\u0131\u011fabilece\u011fi bir boyuttad\u0131r. Tek bir kartta \u00e7al\u0131\u015ft\u0131rd\u0131\u011f\u0131n\u0131zda y\u00fcksek token verimlili\u011fi, d\u00fc\u015f\u00fck gecikme ve d\u00fc\u015f\u00fck g\u00fc\u00e7 t\u00fcketimi elde edersiniz. Ancak bunu iki 8GB kart \u00fczerinde \u00e7al\u0131\u015ft\u0131rmay\u0131 (modeli b\u00f6lerek) denerseniz, PCIe y\u00fck\u00fc getirmi\u015f, g\u00fc\u00e7 t\u00fcketimi ikiye katlanm\u0131\u015f ve \u0131s\u0131 artm\u0131\u015f olursunuz. B\u00fcy\u00fck ihtimalle tek kartta \u00e7al\u0131\u015ft\u0131rmaya g\u00f6re daha yava\u015f<\/em> performans al\u0131rs\u0131n\u0131z. K\u00fc\u00e7\u00fck y\u0131\u011f\u0131n boyutlar\u0131nda “\u00f6l\u00e7ek verimsizli\u011fi” budur.<\/p>\n\u00c7oklu GPU, model, tek bir t\u00fcketici kart\u0131n\u0131n VRAM kapasitesinden b\u00fcy\u00fck oldu\u011funda anlaml\u0131d\u0131r. \u00d6rne\u011fin, bir 70B model (yakla\u015f\u0131k 35GB FP16 veya 30GB Q4_K_M i\u00e7in) tek bir 24GB kartta s\u0131\u011fmaz. Zorunlu<\/em> olarak iki kart kullanman\u0131z gerekir. Bu spesifik durumda, m\u00fcbadele kabul edilebilir: modeli \u00e7al\u0131\u015ft\u0131rmay\u0131 sa\u011flars\u0131n\u0131z, ancak gecikme cezas\u0131n\u0131 kabul edersiniz. Model tek bir kartta s\u0131\u011fabiliyorsa, onu b\u00f6lme. G\u00fc\u00e7 ve donan\u0131m karma\u015f\u0131kl\u0131\u011f\u0131 i\u00e7in hi\u00e7bir getiri elde etmeden para harcam\u0131\u015f olursunuz.<\/p>\nKarars\u0131zl\u0131\u011f\u0131n Ger\u00e7ek D\u00fcnya Maliyeti<\/h3>\n
\u00c7oklu GPU kurulumlar\u0131n\u0131n g\u00f6r\u00fcnmeyen bir maliyeti de operasyonel karars\u0131zl\u0131kt\u0131r. T\u00fcketici s\u00fcr\u00fcc\u00fcleri, LLM \u00e7\u0131kar\u0131m\u0131n\u0131n s\u00fcrd\u00fcr\u00fclen, a\u011f\u0131r y\u00fck\u00fc i\u00e7in tasarlanmam\u0131\u015ft\u0131r. \u0130ki kart \u00e7al\u0131\u015ft\u0131rd\u0131\u011f\u0131n\u0131zda bellek y\u00fck\u00fc da\u011f\u0131t\u0131l\u0131r, ancak b\u00f6l\u00fcnmeyi y\u00f6netmek i\u00e7in gereken sistem belle\u011fi (RAM) kullan\u0131m\u0131 patlama yapabilir. \u0130\u015fletim sisteminin bellek doluluk (OOM) \u00f6ld\u00fcr\u00fcc\u00fcs\u00fc tetiklenirse, t\u00fcm kurulumunuz \u00e7\u00f6ker.<\/p>\n
Ayr\u0131ca ba\u015far\u0131s\u0131zl\u0131k noktalar\u0131 olu\u015fturursunuz. Bir GPU s\u00fcr\u00fcc\u00fcs\u00fc \u00e7\u00f6kerse veya kart a\u015f\u0131r\u0131 \u0131s\u0131narak kapan\u0131rsa, t\u00fcm \u00e7\u0131kar\u0131m i\u015flemi durur. \u00c7\u0131kar\u0131m motoru zarif bir \u015fekilde toparlanamayabilir. Tek GPU’lu bir kurulumda, bir \u00e7\u00f6k\u00fc\u015f yeniden ba\u015flatmad\u0131r. \u00c7oklu GPU kurulumunda ise, bekleyen t\u00fcm istekler i\u00e7in ba\u011flam penceresini kaybetmi\u015f olabilirsiniz. Maliyet kadar “\u00e7al\u0131\u015fma s\u00fcresine” (uptime) \u00f6nem veren bir homelab y\u00f6neticisi i\u00e7in bu, kritik bir riskdir.<\/p>\n
Kendinize sorman\u0131z gereken \u015fey: 70B’lik bir modeli \u00e7al\u0131\u015ft\u0131rabilme yetene\u011fi, artan karma\u015f\u0131l\u0131k de\u011ferinde mi? E\u011fer tek bir 24GB kart \u00fczerinde (Llama 3 34B gibi) nicemlenmi\u015f bir 34B model kullanabilirseniz, y\u00fcksek performans, d\u00fc\u015f\u00fck gecikme ve kararl\u0131l\u0131k elde edersiniz. E\u011fer 70B modelin saf zekas\u0131na ihtiyac\u0131n\u0131z varsa, iki karta ihtiyac\u0131n\u0131z var. Ancak PCIe darbo\u011fac\u0131n\u0131 kabul etmelisiniz.<\/p>\n
Ollama: \u00c7oklu Cihaz \u00c7al\u0131\u015ft\u0131rmas\u0131 \u0130\u00e7in Pratik Bir Sar\u0131c\u0131<\/h2>\n
Ollama, LLM’ler i\u00e7in “tamam\u0131 i\u015fe yarar” \u00e7\u00f6z\u00fcm olarak konumland\u0131rd\u0131. Bir homelab operat\u00f6r\u00fc i\u00e7in bu en g\u00fc\u00e7l\u00fc sat\u0131\u015f noktas\u0131d\u0131r. Python ortamlar\u0131, ba\u011f\u0131ml\u0131l\u0131k y\u00f6netimi ve Docker konteynerlerinin karma\u015f\u0131kl\u0131\u011f\u0131n\u0131 soyutlar. Ancak \u00e7oklu GPU deste\u011fi sundu\u011funuzda, “sadece \u00e7al\u0131\u015f\u0131r” b\u00fcy\u00fcs\u00fc daha k\u0131r\u0131lgan hale gelir. Ollama’n\u0131n \u00e7oklu GPU uygulamas\u0131 pragmatiktir ancak tek i\u015flemli bir sar\u0131c\u0131 mimarisiyle s\u0131n\u0131rl\u0131d\u0131r.<\/p>\n
Ollama Model B\u00f6lme Nas\u0131l Yapar<\/h3>\n
Ollama, katmanl\u0131 bir b\u00f6lme yakla\u015f\u0131m\u0131 kullan\u0131r. Tek bir GPU’ya s\u0131\u011fmayan bir model istedi\u011finizde, Ollama sinir a\u011f\u0131 katmanlar\u0131n\u0131 mevcut cihazlar aras\u0131nda b\u00f6l\u00fc\u015ft\u00fcrmeye \u00e7al\u0131\u015f\u0131r. Bunu otomatik olarak yapar, ancak davran\u0131\u015f\u0131 genellikle ortam de\u011fi\u015fkenleriyle etkileyebilirsiniz. Motor, ilk GPU’ya s\u0131\u011fmayan en b\u00fcy\u00fck katman\u0131 belirler ve onu sonrakine ta\u015f\u0131r.<\/p>\n
\u0130ki RTX 3090’l\u0131 tipik bir kurulumda Ollama, ilk 20 katman\u0131 GPU 0’a ve kalan katmanlar\u0131 GPU 1’e atayabilir. Kritik nokta \u015fudur: Bu b\u00f6l\u00fcnme model y\u00fckleme a\u015famas\u0131nda ger\u00e7ekle\u015fir. Katmanlar \u00e7\u0131kar\u0131m (inference) s\u0131ras\u0131nda dinamik olarak i\u00e7eri al\u0131nmaz veya \u00e7\u0131kar\u0131lmaz; sabittir. Model y\u00fcklendikten sonra GPU 0 alt katmanlardan, GPU 1 ise \u00fcst katmanlardan sorumludur.<\/p>\n
Bu statik b\u00f6lme basit i\u015f y\u00fckleri i\u00e7in verimlidir ancak darbo\u011fazlar yaratabilir. Katmanlar aras\u0131ndaki hesaplama y\u00fck\u00fc dengesizse (ki bu bazen olur), bir GPU i\u015fini di\u011ferinden \u00f6nce bitirebilir, ancak bir sonraki token olu\u015fturulmadan \u00f6nce di\u011ferinin ileri ge\u00e7i\u015fi (forward pass) bitirmesini beklemek zorunda kal\u0131r. Bu senkronizasyon maliyeti tasar\u0131m\u0131n do\u011fas\u0131nda vard\u0131r.<\/p>\n
Kurulum ve Yap\u0131land\u0131rma Kolayl\u0131\u011f\u0131<\/h3>\n
Operat\u00f6rlerin Ollama’y\u0131 se\u00e7mesinin birincil nedeni kurulum kolayl\u0131\u011f\u0131d\u0131r. Tek bir komutla bir sunucu ba\u015flatabilirsiniz: ollama serve<\/code>. \u00c7oklu GPU ortamlar\u0131nda CUDA s\u00fcr\u00fcc\u00fclerinin do\u011fru \u015fekilde haritaya eklenmesini sa\u011flamak i\u00e7in genellikle konteynerize bir kurulum gerekir.<\/p>\nOllama’n\u0131n \u00e7oklu GPU kullanmak i\u00e7in bir Docker konteynerinde nas\u0131l yap\u0131land\u0131r\u0131laca\u011f\u0131na dair \u00f6rnek budur. Bunun i\u00e7in NVIDIA Container Toolkit gereklidir.<\/p>\n
docker run -d --name ollama-multi-gpu --gpus all \\\n -p 11434:11434 \\\n -v ollama:\/root\/.ollama \\\n ollama\/ollama:latest\n<\/code><\/pre>\n--gpus all<\/code> bayra\u011f\u0131 hayati \u00f6nem ta\u015f\u0131r. Docker’a kullan\u0131ma haz\u0131r t\u00fcm GPU’lar\u0131 konteynerle payla\u015fmas\u0131 talimat\u0131n\u0131 verir. Ollama ard\u0131ndan mevcut CUDA cihazlar\u0131n\u0131 tarar ve model katmanlar\u0131n\u0131 buna g\u00f6re atar. Hangi GPU’lar\u0131n kullan\u0131ld\u0131\u011f\u0131n\u0131 kontrol loglar\u0131n\u0131 inceleyerek veya nvidia-smi<\/code> komutuyla do\u011frulayabilirsiniz.<\/p>\nAncak s\u0131n\u0131rlamalar vard\u0131r. Ollama’n\u0131n \u00e7oklu GPU deste\u011fi nispeten yenidir ve vLLM kadar olgun de\u011fildir. K\u00fc\u00e7\u00fck topluluklar i\u00e7in gecikmeyi optimize eder, y\u00fcksek ak\u0131\u015fl\u0131 hizmet sunma i\u00e7in de\u011fil. \u00c7oklu kullan\u0131c\u0131 i\u00e7in bir API sunucusu \u00e7al\u0131\u015ft\u0131r\u0131yorsan\u0131z, Ollama e\u015fzamanl\u0131 isteklerde zorlanabilir. Tek kullan\u0131c\u0131 etkile\u015fimi veya k\u00fc\u00e7\u00fck bir istek grubu i\u00e7in m\u00fckemmeldir, ancak kurumsal motorlar sunan geli\u015fmi\u015f istek zamanlama yetene\u011finden yoksundur.<\/p>\n
E\u015fzamanl\u0131l\u0131k S\u0131n\u0131rlamalar\u0131<\/h3>\n
Ollama yerel \u00e7al\u0131\u015fan bir ki\u015fisel asistan i\u00e7in harika olsa da, ayn\u0131 anda \u00e7oklu kullan\u0131c\u0131y\u0131 y\u00f6netmeniz gerekti\u011finde yetersiz kal\u0131r. Ollama’daki varsay\u0131lan \u00e7\u0131kar\u0131m motoru istekleri s\u0131rayla veya k\u00fc\u00e7\u00fck topluluklar halinde i\u015fler. \u0130ki kullan\u0131c\u0131 ayn\u0131 anda istek g\u00f6nderdi\u011finde, Ollama bunlar\u0131 kuyru\u011fa al\u0131r.<\/p>\n
\u00c7oklu GPU kurulumunda bu kuyrulama darbo\u011faza d\u00f6n\u00fc\u015febilir. 48GB VRAM’iniz olsa bile, topluluk boyutu k\u00fc\u00e7\u00fckse sistem her iki GPU’yu da verimli kullanmayabilir. PCIe busu katmanlar aras\u0131nda veri aktar\u0131m\u0131 i\u00e7in darbo\u011faz haline gelir ve Ollama’n\u0131n tek i\u015flemli mimarisi, \u00f6nemli bir i\u00e7 karma\u015f\u0131kl\u0131k olmadan istek i\u015fleme i\u015flemi \u00e7oklu GPU \u00e7iftleri aras\u0131nda kolayca paralelle\u015ftiremez.<\/p>\n
Operat\u00f6r Notlar\u0131: “Ollama” Takas Dengesi<\/h3>\n
Ollama’y\u0131 se\u00e7erseniz, esneklik yerine konforu al\u0131rs\u0131n\u0131z. Y\u00f6netimi kolay sa\u011flam bir sistem elde edersiniz, ancak katman b\u00f6lme mant\u0131\u011f\u0131n\u0131 ince ayar yapma veya haf\u0131za d\u00fczenini dinamik olarak de\u011fi\u015ftirme yetene\u011finden yoksun kal\u0131rs\u0131n\u0131z. Haf\u0131za d\u00fczenine a\u015f\u0131r\u0131 duyarl\u0131 (baz\u0131 \u00f6zel nicemleme formatlar\u0131 gibi) bir modeli \u00e7al\u0131\u015ft\u0131rman\u0131z gerekiyorsa, Ollama gerekli ince ayar butonlar\u0131n\u0131 (knobs) sunmayabilir.<\/p>\n
Ayr\u0131ca kaynak maliyetine dikkat edin. Ollama bellekte kalan bir sunucu i\u015flemi \u00e7al\u0131\u015ft\u0131r\u0131r. Birden fazla model \u00e7al\u0131\u015ft\u0131r\u0131yorsan\u0131z, Ollama bunlar\u0131n hepsini y\u00fckler ve RAM ve VRAM t\u00fcketir. Sisteminizi \u00e7ok zorlarsan\u0131z bu OOM (Out of Memory) hatalar\u0131na yol a\u00e7abilir. Kullan\u0131lmayan modelleri durdurmak konusunda titiz olman\u0131z gerekir.<\/p>\n
vLLM: Y\u00fcksek E\u015fzamanl\u0131l\u0131k \u0130\u015f Y\u00fckleri \u0130\u00e7in Bant Geni\u015fli\u011fi Canavar\u0131<\/h2>\n
E\u011fer Ollama pratik bir sarmalay\u0131c\u0131 (wrapper) ise, vLLM \u00f6l\u00e7eklenebilirlik i\u00e7in tasarlanm\u0131\u015f y\u00fcksek performansl\u0131 motordur. UC Berkeley ara\u015ft\u0131rmac\u0131lar\u0131 taraf\u0131ndan geli\u015ftirilen vLLM, PagedAttention ad\u0131 verilen bir teknik kullanarak LLM sunumundaki bellek darbo\u011faz\u0131n\u0131 \u00e7\u00f6zmeye \u00f6zel olarak kurgulanm\u0131\u015ft\u0131r. Birden fazla kullan\u0131c\u0131y\u0131 y\u00f6neten \u00e7oklu GPU’lu bir istemci sunucusu kurmaya \u00e7al\u0131\u015fan bir homelab operat\u00f6r\u00fc i\u00e7in, \u00f6\u011frenme e\u011frisinin daha dik olmas\u0131 ve kaynak y\u00fck\u00fc daha fazla olmas\u0131na ra\u011fmen, vLLM genellikle \u00fcst\u00fcn bir tercihtir.<\/p>\n
PagedAttention Mekanizmas\u0131<\/h3>\n
Traditional LLM serving engines store the attention key-value (KV) cache in a contiguous block of memory. As the context window grows, this memory grows linearly. To save memory, you have to reduce the batch size or the context window. This is inefficient.<\/p>\n
vLLM, PagedAttention’\u0131 tan\u0131t\u0131r. Bu mekanizma KV \u00f6nbelle\u011fini birbirine ba\u011fl\u0131 olmayan k\u00fc\u00e7\u00fck bloklara b\u00f6ler. \u0130\u015fletim sistemlerinin fiziksel RAM sayfalar\u0131n\u0131 y\u00f6netme bi\u00e7imine benzer \u015fekilde, bu bloklar\u0131 birbirine ba\u011flamak i\u00e7in bir bellek y\u00f6netim tablosu kullan\u0131r. Bu sayede vLLM \u015funlar\u0131 yapabilir:
\n1. Bellek verimlili\u011fi:<\/strong> Ayn\u0131 \u00f6nekle (prefix) sahip istekler varsa, ayn\u0131 bellek blo\u011fu istekler aras\u0131nda payla\u015f\u0131labilir (tekrar \u00f6nleme).
\n2. Par\u00e7alanmay\u0131 ortadan kald\u0131r\u0131r:<\/strong> Bellek sabit boyutlu bloklar halinde tahsis edilir, bu da bo\u015fa harcanan alan kalmaz demektir.
\n3. B\u00fcy\u00fck batch’leri destekler:<\/strong> Bellek daha verimli kullan\u0131ld\u0131\u011f\u0131 i\u00e7in, ayn\u0131 donan\u0131m \u00fczerinde daha fazla e\u015fzamanl\u0131 istek s\u0131\u011fd\u0131rabilirsiniz.<\/p>\nBir \u00e7oklu GPU kurulumunda, bu verimlilik kritiktir. Bir modeli iki GPU’ya b\u00f6ld\u00fc\u011f\u00fcn\u00fczde, KV \u00f6nbelle\u011fi i\u00e7in bellek fazlal\u0131\u011f\u0131 yine de mevcuttur. vLLM’in bu belle\u011fi y\u00f6netebilme kabiliyeti, bellek tavan\u0131na \u00e7arpmadan \u00f6nce daha fazla iste\u011fi \u00e7al\u0131\u015ft\u0131rabilmenizi sa\u011flar.<\/p>\n
Da\u011f\u0131t\u0131k Sunum Yetenekleri<\/h3>\n
vLLM, tensor paralelli\u011fi (TP) ve pipeline paralelli\u011fi (PP) kullanan da\u011f\u0131t\u0131k \u00e7\u0131kar\u0131m i\u00e7in yerel deste\u011fe sahiptir. vLLM’\u00fcn \u00e7oklu GPU’lu bir homelab ortam\u0131nda parlad\u0131\u011f\u0131 yer buras\u0131d\u0131r.<\/p>\n
\n- Tensor Paralelli\u011fi (TP):<\/strong> Model GPU’lar aras\u0131nda b\u00f6l\u00fcn\u00fcr. Her GPU, ayn\u0131 katman\u0131n bir dilimini i\u015fler. \u00d6rne\u011fin, bir katmanda 1024 giri\u015f kanal\u0131 varsa, GPU 0 ilk 512’sini, GPU 1 son 512’sini i\u015fler. Her tokende kilitle\u015fmi\u015f olarak (lockstep) \u00e7al\u0131\u015f\u0131rlar. Bu, y\u00fcksek h\u0131zl\u0131 ba\u011flant\u0131lar (NVLink gibi) gerektirir ancak devasa h\u0131z art\u0131\u015flar\u0131 sunar.<\/li>\n
- Pipeline Paralelli\u011fi (PP):<\/strong> Farkl\u0131 GPU’lar modelin farkl\u0131 katmanlar\u0131n\u0131 i\u015fler. GPU 0 0-10 katmanlar\u0131n\u0131, GPU 1 11-20 katmanlar\u0131n\u0131 vb. i\u015fler. Bu, Ollama’n\u0131n yakla\u015f\u0131m\u0131na benzer ancak daha iyi batch planlamas\u0131 ve \u00f6rt\u00fc\u015fme sa\u011flar.<\/li>\n<\/ul>\n
PCIe \u00fczerinden ba\u011fl\u0131 t\u00fcketici kartlar\u0131 i\u00e7in vLLM yine de TP veya PP kullanabilir, ancak performans kazan\u00e7lar\u0131 veriyolu ile s\u0131n\u0131rlanacakt\u0131r. Ancak vLLM, Ollama’ya k\u0131yasla gecikme fazlal\u0131\u011f\u0131n\u0131 daha iyi y\u00f6netecek \u015fekilde optimize edilmi\u015ftir. Mevcut bant geni\u015fli\u011finin kullan\u0131m\u0131n\u0131 maksimize etmek i\u00e7in daha sofistike bir zamanlay\u0131c\u0131 kullan\u0131r.<\/p>\n
Y\u00fcksek E\u015fzamanl\u0131l\u0131k vs. D\u00fc\u015f\u00fck Gecikme<\/h3>\n
vLLM, bant geni\u015fli\u011fi (throughput) i\u00e7in tasarlanm\u0131\u015ft\u0131r. Ayn\u0131 anda 100 kullan\u0131c\u0131ya hizmet veren bir web siteniz veya API’niz varsa, bu senaryoda vLLM Ollama’y\u0131 ezer ge\u00e7er. Bir\u00e7ok aktif iste\u011fe ra\u011fmen y\u00fcksek bir token olu\u015fturma h\u0131z\u0131n\u0131 koruyabilir.<\/p>\n
Bununla birlikte, vLLM de bedelini ister. llama.cpp veya Ollama’dan daha y\u00fcksek bir bellek fazlal\u0131\u011f\u0131na sahiptir. PagedAttention uygulamas\u0131 ek metadata yap\u0131lar\u0131 gerektirir. Ayr\u0131ca vLLM’i dikkatli bir \u015fekilde yap\u0131land\u0131rman\u0131z gerekir. GPU say\u0131s\u0131n\u0131, tensor paralellik boyutunu ve pipeline paralellik boyutunu belirtmeniz gerekir.<\/p>\n
# docker-compose.yaml example for vLLM multi-GPU serving\nversion: '3.8'\nservices:\n vllm:\n image: vllm\/vllm-openai:latest\n command:\n - --model\n - meta-llama\/Llama-3-70B-Instruct\n - --tensor-parallel-size\n - \"2\"\n - --dtype\n - float16\n - --max-model-len\n - \"8192\"\n - --gpu-memory-utilization\n - \"0.95\"\n deploy:\n resources:\n reservations:\n devices:\n - driver: nvidia\n count: 2\n capabilities: [gpu]\n ports:\n - \"8000:8000\"\n volumes:\n - .\/models:\/models\n<\/code><\/pre>\nBu yap\u0131land\u0131rma, vLLM’in Llama 3 70B modelini 2 boyutunda tensor paralelli\u011fi ile \u00e7al\u0131\u015ft\u0131raca\u011f\u0131n\u0131 belirtir. \u0130ki GPU’ya sahip oldu\u011funuzu varsayar. VRAM kullan\u0131m\u0131n\u0131 en \u00fcst d\u00fczeye \u00e7\u0131karmak i\u00e7in gpu-memory-utilization<\/code> bayra\u011f\u0131 0.95 olarak ayarlanm\u0131\u015ft\u0131r; g\u00fcvenlik i\u00e7in k\u00fc\u00e7\u00fck bir tampon b\u0131rak\u0131l\u0131r.<\/p>\nOperat\u00f6r Notlar\u0131: Karma\u015f\u0131kl\u0131k Vergisi<\/h3>\n
Homelab ortam\u0131nda vLLM kullanmak s\u0131radan bir i\u015f de\u011fildir. Docker, a\u011f ve vLLM yap\u0131land\u0131rma bayraklar\u0131n\u0131 anlaman\u0131z gerekir. Tensor paralellik boyutunu yanl\u0131\u015f yap\u0131land\u0131r\u0131rsan\u0131z, sunucu \u00e7\u00f6ker. Model uzunlu\u011funu \u00e7ok y\u00fcksek ayarlarsan\u0131z, bellek yetersiz kal\u0131r.<\/p>\n
Ancak, “\u00fcretim benzeri stabilite” ve bant geni\u015fli\u011fine de\u011fer veren bir homelab operat\u00f6r\u00fc i\u00e7in vLLM en iyi ara\u00e7t\u0131r. Bellek y\u00f6netimi ve istek zamanlamas\u0131 gibi zorlu i\u015fleri halleder; b\u00f6ylece siz altyap\u0131ya de\u011fil, i\u015f y\u00fck\u00fcne odaklanabilirsiniz.<\/p>\n
llama.cpp: Gran\u00fcler Kontrol \u0130\u00e7in Ham Motor<\/h2>\n
Ollama bir kabuk, vLLM ise optimize edilmi\u015f bir sunucu ise, llama.cpp do\u011frudan \u00e7al\u0131\u015fan ham motordur. Bir C\/C++ k\u00fct\u00fcphanesi olarak bir\u00e7ok arac\u0131n (Ollama dahil) arkas\u0131ndaki itici g\u00fc\u00e7t\u00fcr. llama.cpp’yi do\u011frudan \u00e7al\u0131\u015ft\u0131rmak, donan\u0131m, model quantization (nicemleme) ve bellek d\u00fczeni \u00fczerinde en geni\u015f kontrol\u00fc sa\u011flar. GPU’lar\u0131n\u0131zdan her bir son damla performans\u0131 s\u0131kmak isteyen operat\u00f6r i\u00e7in llama.cpp krald\u0131r.<\/p>\n
GGUF Format\u0131 ve Quantization (Nicemleme)<\/h3>\n
llama.cpp’nin g\u00fcc\u00fcn\u00fcn kalbi GGUF format\u0131ndad\u0131r. Genellikle FP16 veya FP32 olan standart PyTorch modellerinin aksine, GGUF modelleri bellek hacmini d\u00fc\u015f\u00fcrmek i\u00e7in nicemleme kullan\u0131r. Yayg\u0131n nicemleme se\u00e7enekleri Q4_K_M (4-bit) ve Q8_0 (8-bit)’dir.<\/p>\n
Nicemleme, modelin gerektirdi\u011fi belle\u011fi 4’ten 8’e kadar azalt\u0131r. FP16 format\u0131nda 140GB gerektiren bir 70B model, Q4_K_M olarak nicemlendi\u011finde yakla\u015f\u0131k 40GB VRAM’e s\u0131\u011fd\u0131r\u0131labilir. \u0130\u015fte llama.cpp’nin homelab toplulu\u011funda bu kadar pop\u00fcler olmas\u0131n\u0131n sebebi: Aksi halde imkans\u0131z olacak devasa modelleri t\u00fcketiciler seviyesinde donan\u0131mda \u00e7al\u0131\u015ft\u0131rman\u0131za izin verir.<\/p>\n
llama.cpp’yi \u00e7oklu GPU i\u00e7in kullan\u0131rken, modeli s\u0131\u011fd\u0131rmak i\u00e7in genellikle bu nicemleme avantaj\u0131n\u0131 kullan\u0131rs\u0131n\u0131z. Ancak model VRAM’den fazlaysa, katmanlar\u0131 sistem RAM’ine ta\u015f\u0131mak (offload) se\u00e7ene\u011finiz de vard\u0131r. Bu se\u00e7enek her zaman \u00e7al\u0131\u015fmaz; bellek yolundaki gecikme (latency) ciddi \u015fekilde artar ve bunu bir \u00fcretim ortam\u0131nda beklersiniz. Alternatif \u00e7\u00f6z\u00fcmler genelde buna izin vermez veya \u00e7ok daha yava\u015ft\u0131r.<\/p>\n
Manuel Tens\u00f6r B\u00f6l\u00fcmleme ve Katman Kontrol\u00fc<\/h3>\n
llama.cpp’de modeli nas\u0131l b\u00f6ld\u00fc\u011f\u00fcn\u00fcz konusunda a\u00e7\u0131k bir kontrol\u00fcn\u00fcz vard\u0131r. Katmanlar\u0131 GPU’ya ta\u015f\u0131ma say\u0131s\u0131n\u0131 belirtmek i\u00e7in -ngl<\/code> bayra\u011f\u0131n\u0131 (offload edilecek katman say\u0131s\u0131) kullan\u0131rs\u0131n\u0131z.<\/p>\n\u00c7oklu GPU kurulumlar\u0131nda tipik komut sat\u0131r\u0131 \u015f\u00f6yle g\u00f6r\u00fcn\u00fcr:<\/p>\n
.\/llama-server -m models\/llama-3-70b.Q4_K_M.gguf \\\n --ctx-size 8192 \\\n --n-ctx 8192 \\\n --ngl 80 \\\n --threads 12 \\\n --gpu-layers 20\n<\/code><\/pre>\nDurak, yukar\u0131daki komut tek bir GPU i\u00e7indir. \u00c7oklu GPU i\u00e7in llama.cpp farkl\u0131 bir mekanizma kullan\u0131r. GPU say\u0131s\u0131n\u0131 ve da\u011f\u0131l\u0131m\u0131 belirtmeniz gerekir. Son s\u00fcr\u00fcmlerde llama.cpp otomatik olarak birden fazla GPU’yu alg\u0131lar ve katmanlar\u0131 b\u00f6ler, ancak \u00e7evre de\u011fi\u015fkenleri veya --split-mode<\/code> gibi belirli bayraklar kullanarak bunu manuel olarak da kontrol edebilirsiniz.<\/p>\n\u00d6rne\u011fin, belirli bir b\u00f6l\u00fcnmeyi zorlamak isteyebilirsiniz:<\/p>\n
export LLAMA_MAX_BATCH_SIZE=128\n.\/llama-server -m models\/llama-3-70b.Q4_K_M.gguf --gpu-layers 40\n<\/code><\/pre>\nAsl\u0131nda, ger\u00e7ek \u00e7oklu GPU da\u011f\u0131l\u0131m\u0131 i\u00e7in genellikle toplam katmanlara g\u00f6re --ngl<\/code> bayra\u011f\u0131na g\u00fcvenmeniz gerekir. Toplam 80 katn\u0131n\u0131z ve iki GPU’nuz varsa, her GPU i\u00e7in --ngl 40<\/code> ayarlayabilirsiniz; ancak llama.cpp da\u011f\u0131t\u0131m mant\u0131\u011f\u0131n\u0131 mevcut VRAM’e g\u00f6re kendi i\u00e7inde halleder. Modeli e\u015fit \u015fekilde b\u00f6lmeye \u00e7al\u0131\u015f\u0131r. Farkl\u0131 VRAM’lere sahipseniz (\u00f6rne\u011fin biri 24GB, di\u011feri 12GB), llama.cpp daha ak\u0131ll\u0131 davran\u0131r ve 24GB’l\u0131k karta daha fazla, 12GB’l\u0131k karta daha az katman y\u00fckler. Ama unutmay\u0131n, karars\u0131z donan\u0131m yap\u0131lar\u0131 her zaman beklenmedik hatalar yarat\u0131r.<\/p>\nD\u00fc\u015f\u00fck Seviye Kontrol ve Esneklik<\/h3>\n
llama.cpp’nin en b\u00fcy\u00fck avantaj\u0131 esnekliktir. \u00d6zel yap\u0131lar olu\u015fturabilirsiniz. Donan\u0131m\u0131n\u0131za \u00f6zel optimizasyonlar i\u00e7in belirli kernel’leri (CUDA, ROCm, Metal) aktif edebilir, kullan\u0131m durumunuza \u00f6zel bellek d\u00fczenini ince ayara getirebilirsiniz.<\/p>\n
\u00d6rne\u011fin, \u00e7ok b\u00fcy\u00fck bir ba\u011flam penceresi (\u00f6rn. 128k token) gerektiren bir model \u00e7al\u0131\u015ft\u0131r\u0131yorsan\u0131z, llama.cpp ba\u015fka motorlar\u0131n eri\u015fime a\u00e7mad\u0131\u011f\u0131 farkl\u0131 kayd\u0131rma penceresi tekniklerini veya KV \u00f6nbellek atma politikalar\u0131n\u0131 denemenize olanak tan\u0131r. Alternatif ara\u00e7lar genellikle bu d\u00fczeyde k\u0131s\u0131tland\u0131r\u0131lm\u0131\u015ft\u0131r.<\/p>\n
Operat\u00f6r Notlar\u0131: Bak\u0131m Y\u00fck\u00fc<\/h3>\n
llama.cpp’yi do\u011frudan \u00e7al\u0131\u015ft\u0131rmak ciddi bir i\u015f y\u00fck\u00fcd\u00fcr. Binary’leri derlemeniz, model dosyalar\u0131n\u0131 y\u00f6netmeniz ve komut sat\u0131r\u0131 arg\u00fcmanlar\u0131n\u0131n do\u011fru oldu\u011fundan emin olman\u0131z gerekir. Bunu bir API \u00fczerinden sunmak istiyorsan\u0131z, llama-server<\/code> binary’sini \u00e7al\u0131\u015ft\u0131r\u0131p HTTP isteklerini kendiniz y\u00f6netmeniz veya bunlar\u0131 text-generation-webui<\/code> veya oobabooga<\/code> gibi bir\u6846\u67b6\u0131n i\u00e7ine sarmalaman\u0131z gerekir.<\/p>\nAncak tam olarak ne oldu\u011funu anlamak isteyen operat\u00f6r i\u00e7in llama.cpp en iyi ara\u00e7t\u0131r. Loglar\u0131 g\u00f6rebilir, parametreleri ince ayarlayabilir ve spesifik donan\u0131m topolojinize g\u00f6re optimize edebilirsiniz. Di\u011ferleri sadece “\u00e7al\u0131\u015fs\u0131n” derken, siz sistemi y\u00f6netirsiniz. Beklenen performans ve stabilite i\u00e7in bu maliyeti \u00f6demelisiniz.<\/p>\n
Don\u00f6r Topolojisi \u00d6nemlidir: NVLink vs. PCIe Darbo\u011fazlar\u0131<\/h2>\n
Ollama, vLLM veya llama.cpp aras\u0131nda se\u00e7im yapmak, fiziksel donan\u00f6r topolojisinden ikincildir. D\u00fcnyan\u0131n en iyi yaz\u0131l\u0131m\u0131na sahip olsan\u0131z bile, e\u011fer donan\u00f6r topolojiniz hatal\u0131ysa performans\u0131n\u0131z berbat olur. En kritik fakt\u00f6r GPU’lar aras\u0131ndaki ba\u011flant\u0131d\u0131r.<\/p>\n
NVLink Avantaj\u0131 (ve Yoklu\u011fu)<\/h3>\n
Veri merkezlerinde GPU’lar NVLink (Nvidia Y\u00fcksek H\u0131zl\u0131 \u0130leti\u015fim) ile ba\u011flan\u0131r. NVLink, muazzam bant geni\u015fli\u011fi (H100 i\u00e7in 900 GB\/s’e kadar) ve d\u00fc\u015f\u00fck gecikme s\u00fcresi sa\u011flar. Bu, GPU’lar aras\u0131nda s\u0131k\u0131 bir birle\u015fim anlam\u0131na gelir. \u0130ki H100 aras\u0131nda NVLink ile \u00e7al\u0131\u015fan bir modeli \u00e7al\u0131\u015ft\u0131rd\u0131\u011f\u0131n\u0131zda, neredeyse tek devasa bir GPU gibi davran\u0131rlar. Overhead ihmal edilebilir d\u00fczeydedir.<\/p>\n
T\u00fcketici kartlar\u0131 (RTX 3090, 4090) NVLink’e sahip de\u011fildir. NVIDIA, t\u00fcketici kartlar\u0131ndan NVLink ba\u011flant\u0131s\u0131n\u0131 kald\u0131rd\u0131. PCIe bus’\u0131 ile ba\u015f ba\u015fa kal\u0131rs\u0131n\u0131z. Bu temel bir k\u0131s\u0131tt\u0131r.<\/p>\n
PCIe Gen3 vs. Gen4 vs. Gen5<\/h3>\n
T\u00fcketici donan\u0131mda \u00e7oklu-GPU \u00e7\u0131kar\u0131m\u0131 i\u00e7in s\u0131n\u0131rlay\u0131c\u0131 fakt\u00f6r PCIe bus h\u0131z\u0131d\u0131r.
\n* PCIe Gen3 x16:<\/strong> ~16 GB\/s teorik bant geni\u015fli\u011fi.
\n* PCIe Gen4 x16:<\/strong> ~32 GB\/s teorik bant geni\u015fli\u011fi.
\n* PCIe Gen5 x16:<\/strong> ~64 GB\/s teorik bant geni\u015fli\u011fi.<\/p>\nUygulamada, protokol overhadd\u0131 nedeniyle teorik bant geni\u015fli\u011finin %70-80’ini al\u0131rs\u0131n\u0131z.<\/p>\n
E\u011fer PCIe Gen4 x16 slotunda iki RTX 3090’\u0131n\u0131z varsa, teorik olarak 32 GB\/s’lik bir borunuz var demektir. 70B’lik bir model i\u00e7in veri hareketi yo\u011fundur. Bir katman\u0131n \u00e7\u0131kt\u0131s\u0131n\u0131 bir sonrakinin giri\u015fine ta\u015f\u0131yorsunuz. Model b\u00fcy\u00fckse, veri hareketi PCIe bus’\u0131 doyurabilir. Bu, GPU’lar\u0131n hesaplama yapmaktan ziyade veri bekledikleri anlam\u0131na gelir.<\/p>\n
Ba\u011flam De\u011fi\u015ftirme \u00dczerindeki Gecikme Etkisi<\/h3>\n
\u00c7oklu-GPU kurulumlar\u0131nda her token \u00fcretimi, birden fazla veri aktar\u0131m turu i\u00e7erir. Y\u00fcksek gecikmeli bir ba\u011flant\u0131n\u0131z varsa (PCIe gibi), token ba\u015f\u0131na gecikme artar. Bu do\u011frudan “ilk tokena kadar ge\u00e7en s\u00fcre” (TTFT) ve toplam token \u00fcretim h\u0131z\u0131n\u0131 etkiler.<\/p>\n
Bir tek kullan\u0131c\u0131l\u0131 senaryo i\u00e7in bu kabule edilebilir olabilir. saniyede 5 token ile 8 token aras\u0131ndaki fark fark edilmeyebilir. Ancak y\u00fcksek e\u015fzamanl\u0131l\u0131k gerektiren bir sistemde bu gecikme birikir. 100 e\u015fzamanl\u0131 iste\u011finiz varsa ve her iste\u011fin PCIe aktar\u0131m\u0131n\u0131 beklemesi gerekiyorsa, sisteminiz yava\u015flar.<\/p>\n
Kar\u015f\u0131la\u015ft\u0131rma: Veri Merkezi vs. T\u00fcketici<\/h3>\n
\n\n\n| \u00d6zellik<\/th>\n | Veri Merkezi (A100\/H100 + NVLink)<\/th>\n | T\u00fcketici (RTX 3090\/4090 + PCIe)<\/th>\n<\/tr>\n<\/thead>\n |
\n\nBa\u011flant\u0131<\/strong><\/td>\n| NVLink (900 GB\/s)<\/td>\n | PCIe Gen4 (32 GB\/s)<\/td>\n<\/tr>\n | \nGecikme<\/strong><\/td>\n| \u00c7ok D\u00fc\u015f\u00fck<\/td>\n | Orta ile Y\u00fcksek Aras\u0131<\/td>\n<\/tr>\n | \n\u00d6l\u00e7eklenebilirlik<\/strong><\/td>\n| Do\u011frusal (8+ GPU’ya kadar)<\/td>\n | Azalan Getiriler (1.5x – 1.8x)<\/td>\n<\/tr>\n | \nMaliyet<\/strong><\/td>\n| Kart ba\u015f\u0131na $10.000+<\/td>\n | Kart ba\u015f\u0131na $1.500<\/td>\n<\/tr>\n | \nG\u00fc\u00e7 T\u00fcketimi<\/strong><\/td>\n| Kart ba\u015f\u0131na ~300-450W<\/td>\n | Kart ba\u015f\u0131na ~250-350W<\/td>\n<\/tr>\n | \nStabilite<\/strong><\/td>\n| Kurumsal Derece<\/td>\n | T\u00fcketici Derecesi (Termal\/S\u00fcr\u00fcc\u00fc sorunlar\u0131)<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n G\u00f6rd\u00fc\u011f\u00fcn\u00fcz gibi, t\u00fcketici kurulumu b\u00fcy\u00fck bir tavizdir. Para tasarrufu sa\u011flars\u0131n\u0131z ama performanstan kaybedersiniz. Benchmarklarda okudu\u011funuz “do\u011frusal \u00f6l\u00e7eklenebilirlik” t\u00fcketici kartlar i\u00e7in bir efsanedir.<\/p>\n Bellek Hiyerar\u015fisi: VRAM, RAM ve Takas Riskleri<\/h2>\n\u00c7oklu GPU kulland\u0131\u011f\u0131n\u0131zda, karma\u015f\u0131k bir bellek hiyerar\u015fisiyle u\u011fra\u015f\u0131yorsunuz demektir. Her GPU \u00fczerinde VRAM, sistem RAM’i ve potansiyel olarak disk (takas) mevcuttur. \u00c7\u00f6kmeleri \u00f6nlemenin anahtar\u0131, bu katmanlar\u0131n nas\u0131l etkile\u015fime girdi\u011fini anlamakt\u0131r.<\/p>\n VRAM Hiyerar\u015fisi<\/h3>\n\u00c7oklu GPU kurulumunda toplam VRAM, t\u00fcm GPUlardaki VRAM’lerin toplam\u0131d\u0131r. Ancak bunlar\u0131 toplay\u0131p t\u00fcm\u00fcn\u00fc do\u011frudan kullanabilece\u011finizi varsayamazs\u0131n\u0131z. Katmanlar b\u00f6l\u00fcnmelidir. \u0130ki adet 24GB kart\u0131n\u0131z olsa, toplamda 48GB’a sahipsiniz demektir. Fakat 48GB a\u011f\u0131rl\u0131\u011f\u0131 tek bir katmanda tutamazs\u0131n\u0131z. A\u011f\u0131rl\u0131klar kartlar aras\u0131nda b\u00f6l\u00fcn\u00fcr.<\/p>\n E\u011fer bir katman, tek bir kart\u0131n VRAM kapasitesinden b\u00fcy\u00fckse, bu katman birden fazla karta b\u00f6l\u00fcnmek zorundad\u0131r. Bu da veri hareketini art\u0131r\u0131r. E\u011fer bir katman tek bir karta bile s\u0131\u011famazsa, sistem \u00e7\u00f6ker veya CPU’ya kayd\u0131rma (offloading) moduna d\u00fc\u015fer.<\/p>\n CPU Kayd\u0131rma ve RAM Takas\u0131<\/h3>\nVRAM doldu\u011funda, \u00e7\u0131kar\u0131m motoru baz\u0131 katmanlar\u0131 sistem RAM’ine kayd\u0131rabilir. Buna “CPU kayd\u0131rma” denir. Bu, performans katili bir i\u015flemdir. RAM, VRAM’den \u00e7ok daha yava\u015ft\u0131r. Bant geni\u015fli\u011fi fark\u0131 10x ile 20x aras\u0131nda de\u011fi\u015fir.<\/p>\n E\u011fer modeliniz toplam VRAM kapasitesinden b\u00fcy\u00fckse, iki se\u00e7ene\u011finiz var:<\/p>\n \n- Quantize (Nicemleme):<\/strong> A\u011f\u0131rl\u0131\u011f\u0131n hassasiyetini d\u00fc\u015f\u00fcr\u00fcn (\u00f6rn. FP16’dan Q4_K_M’e). Bu, bellek ayak izini azalt\u0131r.<\/li>\n
- Offload (Kayd\u0131rma):<\/strong> Baz\u0131 katmanlar\u0131 RAM’e ta\u015f\u0131y\u0131n.<\/li>\n<\/ol>\n
E\u011fer kayd\u0131rma se\u00e7ene\u011fini se\u00e7erseniz, token \u00fcretme h\u0131z\u0131n\u0131z dramatik \u015fekilde d\u00fc\u015fer. 1 saniyede 10 tokenden 1 tokene gerileyebilirsiniz. Bunun sebebi CPU’nun RAM’i beklemesidir. Alternatif olarak bellek y\u00f6netimini elden ge\u00e7irip RAM’e y\u00fck\u00fc azaltabilirsiniz, ama o zaman da i\u015flemci g\u00fcc\u00fc gerekir.<\/p>\n Swap Dosyalar\u0131 Riski<\/h3>\nE\u011fer sistem RAM’i dolarsa, i\u015fletim sistemi diski takas (swap) alan\u0131 olarak kullanmaya ba\u015flar. Bu, LLM \u00e7\u0131kar\u0131m\u0131 i\u00e7in felakettir. Disk G\/\u00c7 i\u015flemi, RAM’e k\u0131yasla iki b\u00fcy\u00fckl\u00fck derecesi daha yava\u015ft\u0131r. Sistem donar ve verileriniz kaybolabilir.<\/p>\n Bunu \u00f6nlemek i\u00e7in, kayd\u0131r\u0131lan katmanlar\u0131 tutabilecek kadar sistem RAM’inize sahip oldu\u011funuzdan emin olmal\u0131s\u0131n\u0131z. 40GB civar\u0131 yer kaplayan Q4_K_M format\u0131nda bir 70B modeliniz varsa ve 64GB RAM’iniz varsa, i\u015fletim sistemi ve di\u011fer i\u015flemler i\u00e7in geriye sadece 24GB kal\u0131r. Bu s\u0131k\u0131\u015f\u0131k bir durumdur. Sistem bellek kullan\u0131m\u0131n\u0131 s\u00fcrekli izlemeniz gerekir.<\/p>\n Operat\u00f6r Notlar\u0131: “Swap” Pani\u011fi<\/h3>\nSisteminizde takas (swap) g\u00f6rd\u00fc\u011f\u00fcn\u00fcz an, \u00e7\u0131kar\u0131m\u0131 hemen durdurun. Takas veriyi bozabilir ve GPU s\u00fcr\u00fcc\u00fcs\u00fcn\u00fc \u00e7\u00f6kebilir. Her zaman bir g\u00fcvenlik pay\u0131 b\u0131rak\u0131n. RAM’inizin %100’\u00fcn\u00fc kullanmay\u0131n. \u0130\u015fletim sistemi i\u00e7in en az %10-15 bo\u015fluk b\u0131rak\u0131n. Alternatif olarak sanal bellek boyutunu k\u0131s\u0131tlay\u0131p sistemi korumaya almak da bir se\u00e7enektir.<\/p>\n Da\u011f\u0131t\u0131m \u00d6ncesi Donan\u0131m Denetimi: Stabilite i\u00e7in Kontrol Listesi<\/h2>\nModelleri \u00e7al\u0131\u015ft\u0131rmaya ba\u015flamadan \u00f6nce donan\u0131m\u0131 denetlemeniz \u015fart. Bu bir l\u00fcks de\u011fil. Ba\u011f\u0131rs\u0131z\u0131z denetim, da\u011f\u0131t\u0131m\u0131n \u00e7\u00f6kmesiyle sonu\u00e7lan\u0131r. Kurulumunuzu do\u011frulamak i\u00e7in bu listeyi kullan\u0131n.<\/p>\n \n- \u2610 GPU Uyumlulu\u011fu:<\/strong> T\u00fcm GPU’lar\u0131n NVIDIA veya AMD (tutarl\u0131) oldu\u011fundan emin olun. Ayn\u0131 \u00e7oklu-GPU kurulumunda NVIDIA ve AMD kartlar\u0131 kar\u0131\u015ft\u0131rmay\u0131n (s\u00fcr\u00fcc\u00fc uyumsuzluklar\u0131 ka\u00e7\u0131n\u0131lmaz olur).<\/li>\n
- \u2610 PCIe Topolojisi:<\/strong> T\u00fcm GPU’lar\u0131n ayn\u0131 PCIe k\u00f6k k\u00fcmesinde olup olmad\u0131\u011f\u0131n\u0131 kontrol edin. NVLink veya y\u00fcksek h\u0131zl\u0131 bir ara ba\u011flay\u0131c\u0131n\u0131z yoksa, GPU’lar\u0131 farkl\u0131 CPU yuvalar\u0131na b\u00f6lmekten ka\u00e7\u0131n\u0131n. Intel’nin QuickPath veya AMD’nin Infinity Fabric’i gibi alternatifler bile tam bant geni\u015fli\u011fi sa\u011flamaz, gecikme artar.<\/li>\n
- \u2610 G\u00fc\u00e7 Kayna\u011f\u0131:<\/strong> PSU’nun yeterli marj\u0131 oldu\u011fundan emin olun. \u0130ki adet 3090, anl\u0131k olarak 700W’\u0131 ge\u00e7ebilir. Yeterli raylere sahip en az 1000W+ bir PSU \u015fartt\u0131r. Ucuz veya kalitesiz g\u00fc\u00e7 kaynaklar\u0131 y\u00fck alt\u0131nda an\u0131nda kilitlenir.<\/li>\n
- \u2610 So\u011futma:<\/strong> Hava ak\u0131\u015f\u0131n\u0131 do\u011frulay\u0131n. K\u00fc\u00e7\u00fck bir kasa i\u00e7inde iki GPU \u0131s\u0131ya kar\u015f\u0131 diren\u00e7 g\u00f6stermez. S\u0131v\u0131 so\u011futma veya y\u00fcksek statik bas\u0131n\u00e7l\u0131 fanlar \u015fartt\u0131r. Fanlar t\u0131kal\u0131 oldu\u011funda sistem koruma kipiyle kapan\u0131r, bu bir sorun de\u011fil, bir kurtar\u0131c\u0131d\u0131r.<\/li>\n
- \u2610 S\u00fcr\u00fcc\u00fc S\u00fcr\u00fcm\u00fc:<\/strong> CUDA s\u00fcr\u00fcc\u00fclerinin g\u00fcncel oldu\u011fundan emin olun. Eski s\u00fcr\u00fcc\u00fcler, OOM (Yetersiz Bellek) hatalar\u0131n\u0131n en yayg\u0131n sebebidir. Neden? \u00c7\u00fcnk\u00fc yeni modeller eski s\u00fcr\u00fcc\u00fclerin bellek y\u00f6netimini atlar.<\/li>\n
- \u2610 Sistem RAM’i:<\/strong> \u00d6nekleme (offloading) i\u00e7in yeterli sistem RAM’iniz oldu\u011fundan emin olun. Beklenen \u00f6nekleme durumunda modeli boyutunun en az 2 kat\u0131 RAM hedefleyin. DDR5 kullanmak zorunda de\u011filseniz, eski DDR4 bile yeterli olabilir ama h\u0131z fark\u0131n\u0131 g\u00f6receksiniz.<\/li>\n
- \u2610 Swap Alan\u0131:<\/strong> Swap’\u0131 devre d\u0131\u015f\u0131 b\u0131rak\u0131n veya \u00e7\u00f6k\u00fc\u015fleri \u00f6nlemek i\u00e7in \u00e7ok b\u00fcy\u00fck bir dosya olarak ayarlay\u0131n (ama de\u011fi\u015ftirme i\u015fleminden tamamen ka\u00e7\u0131nmak en iyisidir). Swap kullan\u0131m\u0131, SSD \u00f6mr\u00fcn\u00fc h\u0131zla t\u00fcketir ve performans\u0131 fel\u00e7 eder. Sanal belle\u011fe ihtiyac\u0131n\u0131z varsa, fiziksel RAM eksikli\u011fine dikkat edin.<\/li>\n<\/ul>\n
\u00c7oklu GPU Konfig\u00fcrasyonu Do\u011frulama: Ad\u0131m Ad\u0131m Rehber<\/h2>\nDonan\u0131m denetimi tamamland\u0131ktan sonra, yaz\u0131l\u0131m\u0131n do\u011fru yap\u0131land\u0131r\u0131ld\u0131\u011f\u0131n\u0131 teyit etmelisin. \u00c7oklu GPU kurulumunun beklendi\u011fi gibi \u00e7al\u0131\u015fmas\u0131n\u0131 sa\u011flamak i\u00e7in \u015fu ad\u0131mlar\u0131 izle.<\/p>\n \n- \u2610 Cihaz Tan\u0131ma:<\/strong> T\u00fcm GPU’lar\u0131n g\u00f6r\u00fcn\u00fcr ve sa\u011fl\u0131kl\u0131 oldu\u011funu do\u011frulamak i\u00e7in
nvidia-smi<\/code> komutunu \u00e7al\u0131\u015ft\u0131r. Alternatif olarak, s\u00fcr\u00fcc\u00fc sorunlar\u0131nda `lspci`’yi kontrol et ama as\u0131l kan\u0131t burada.<\/li>\n- \u2610 CUDA Ba\u011flam\u0131:<\/strong> T\u00fcm cihazlarda CUDA ba\u011flam\u0131n\u0131n olu\u015fturuldu\u011fundan emin ol. Bu basit ad\u0131ml\u0131 atlanmazsa sistem sessizce hata vermez ama i\u015flemez.<\/li>\n
- \u2610 Katman Ayr\u0131m\u0131:<\/strong> Katmanlar\u0131n GPU’lar aras\u0131nda b\u00f6l\u00fcnd\u00fc\u011f\u00fcn\u00fc teyit etmek i\u00e7in loglara bak. “Layer 0-20 on GPU 0, Layer 21-50 on GPU 1” gibi mesajlar\u0131 ara. Bir kart\u0131n y\u00fck\u00fc ta\u015f\u0131rken di\u011feri bo\u015fsa, ayr\u0131m mant\u0131\u011f\u0131n bozuk demektir.<\/li>\n
- \u2610 Bellek Kullan\u0131m\u0131:<\/strong> Her kart\u0131n VRAM kullan\u0131m\u0131n\u0131 izle. Dengeli olmal\u0131. Bir kart t\u0131ka doluyken di\u011ferinin bo\u015f olmas\u0131, payla\u015f\u0131m mant\u0131\u011f\u0131n\u0131n yanl\u0131\u015f kuruldu\u011funun en net kan\u0131t\u0131d\u0131r.<\/li>\n
- \u2610 Performans Taban\u0131:<\/strong> K\u00fc\u00e7\u00fck bir test iste\u011fi g\u00f6nder. Token olu\u015fturma h\u0131z\u0131n\u0131 \u00f6l\u00e7. Tek GPU performans\u0131yla k\u0131yaslayarak h\u0131zlanma (speedup) elde edip etmedi\u011fini netle\u015ftir. Tekil bir GPU’nun \u0131s\u0131nma sorunuyla yava\u015flamas\u0131, \u00e7iftli\u011fin avantaj\u0131n\u0131 maskeleyebilir.<\/li>\n
- \u2610 Stres Testi:<\/strong> 30 dakikal\u0131k uzun s\u00fcreli bir \u00e7\u0131kar\u0131m (inference) \u00e7al\u0131\u015ft\u0131r. Termal darbe (throttling) veya s\u00fcr\u00fcc\u00fc \u00e7\u00f6k\u00fc\u015flerini kontrol et. \u0130lk bozulan genellikle so\u011futma sistemidir, sadece yaz\u0131l\u0131ma g\u00fcvenme.<\/li>\n
- \u2610 Paralellik Testi:<\/strong> Ayn\u0131 anda birden fazla istek g\u00f6nder. Sistemin y\u00fck alt\u0131nda \u00e7\u00f6kmeden y\u00f6netebildi\u011fini teyit et. Birden fazla ak\u0131\u015fta bellek bariyeri olmaks\u0131z\u0131n \u00e7al\u0131\u015fmazsa, mimarin ger\u00e7ek bir \u00e7oklu kullan\u0131c\u0131 senaryosu i\u00e7in haz\u0131r de\u011fil demektir.<\/li>\n<\/ul>\n
Inference Motoru \u00d6zellik Matrisi<\/h2>\nDo\u011fru motoru se\u00e7menize yard\u0131mc\u0131 olmak i\u00e7in Ollama, vLLM ve llama.cpp’nin ana metrikler \u00fczerindeki do\u011frudan kar\u015f\u0131la\u015ft\u0131rmas\u0131. Not edin: En ucuz \u00e7\u00f6z\u00fcm her zaman en az sorunu ya\u015famaz; beklenen \u00f6m\u00fcr ve faturalara g\u00f6re karar verin.<\/p>\n \n\n\n| \u00d6zellik<\/th>\n | Ollama<\/th>\n | vLLM<\/th>\n | llama.cpp<\/th>\n<\/tr>\n<\/thead>\n | \n\nKullan\u0131m Kolayl\u0131\u011f\u0131<\/strong><\/td>\n| Y\u00fcksek (Tek komut)<\/td>\n | Orta (Yap\u0131land\u0131rma gerekir)<\/td>\n | D\u00fc\u015f\u00fck (CLI \/ \u00d6zel derleme)<\/td>\n<\/tr>\n | \n\u00c7oklu GPU Deste\u011fi<\/strong><\/td>\n| Temel (Katman ayr\u0131m\u0131)<\/td>\n | Geli\u015fmi\u015f (TP\/PP)<\/td>\n | Geli\u015fmi\u015f (Katman ayr\u0131m\u0131)<\/td>\n<\/tr>\n | \n\u0130\u015fletme E\u015fzamanl\u0131l\u0131\u011f\u0131<\/strong><\/td>\n| D\u00fc\u015f\u00fck<\/td>\n | Y\u00fcksek<\/td>\n | Orta<\/td>\n<\/tr>\n | \nGe\u00e7ikme (Latency)<\/strong><\/td>\n| D\u00fc\u015f\u00fck<\/td>\n | D\u00fc\u015f\u00fck<\/td>\n | \u00c7ok D\u00fc\u015f\u00fck<\/td>\n<\/tr>\n | \nBellek Verimlili\u011fi<\/strong><\/td>\n| Orta<\/td>\n | Y\u00fcksek (PagedAttention)<\/td>\n | \u00c7ok Y\u00fcksek (GGUF)<\/td>\n<\/tr>\n | \nDonan\u0131m Esnekli\u011fi<\/strong><\/td>\n| D\u00fc\u015f\u00fck (Sadece CUDA)<\/td>\n | Orta (CUDA\/ROCm)<\/td>\n | Y\u00fcksek (CUDA\/ROCm\/Metal)<\/td>\n<\/tr>\n | \nBak\u0131m Y\u00fck\u00fc<\/strong><\/td>\n| D\u00fc\u015f\u00fck<\/td>\n | Orta<\/td>\n | Y\u00fcksek<\/td>\n<\/tr>\n | \nEn Uygun Oldu\u011fu Kullan\u0131m<\/strong><\/td>\n| Ki\u015fisel kullan\u0131m, tek kullan\u0131c\u0131l\u0131<\/td>\n | API sunumu, y\u00fcksek e\u015fzamanl\u0131l\u0131k<\/td>\n | \u00d6zel derlemeler, maksimum kontrol<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nTopolojiye G\u00f6re \u00c7oklu GPU Performans Beklentileri<\/h2>\n\u00c7oklu GPU kurulumlar\u0131nda performans art\u0131\u015f\u0131 bus topolojisine do\u011frudan ba\u011fl\u0131d\u0131r. Donan\u0131ma g\u00f6re h\u0131zland\u0131rma oranlar\u0131 ciddi farkl\u0131l\u0131klar g\u00f6sterir. Alternatif olarak PCIe \u00fczerinden yap\u0131lan ba\u011flant\u0131lar\u0131n maliyeti daha d\u00fc\u015f\u00fck olsa da, NVLink gibi \u00f6zel ba\u011flant\u0131larda ortaya \u00e7\u0131kacak darbo\u011fazlar sistemin t\u00fcm \u00f6mr\u00fc boyunca yava\u015flat\u0131c\u0131 etki yapar. Beklentinizi geride b\u0131rakacak en zay\u0131f halka hemen fark edilir.<\/p>\n \n\n\n| Topoloji<\/th>\n | Bant Geni\u015fli\u011fi<\/th>\n | Beklenti H\u0131zland\u0131rma<\/th>\n | Notlar<\/th>\n<\/tr>\n<\/thead>\n | \n\nNVLink (H100)<\/strong><\/td>\n| 900 GB\/s<\/td>\n | 1.8x – 2.0x<\/td>\n | Neredeyse do\u011frusal \u00f6l\u00e7ekleme.<\/td>\n<\/tr>\n | \nPCIe Gen5 x16<\/strong><\/td>\n| 64 GB\/s<\/td>\n | 1.5x – 1.8x<\/td>\n | \u0130yi \u00f6l\u00e7ekleme, d\u00fc\u015f\u00fck gecikme.<\/td>\n<\/tr>\n | \nPCIe Gen4 x16<\/strong><\/td>\n| 32 GB\/s<\/td>\n | 1.3x – 1.6x<\/td>\n | Orta seviye \u00f6l\u00e7ekleme.<\/td>\n<\/tr>\n | \nPCIe Gen3 x16<\/strong><\/td>\n| 16 GB\/s<\/td>\n | 1.1x – 1.4x<\/td>\n | Azalan verimlilik.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n Not: Bunlar 70B parametreye sahip bir model i\u00e7in tahminlerdir. Daha k\u00fc\u00e7\u00fck modellerde bu fayda daha azd\u0131r, daha b\u00fcy\u00fck modellerde ise VRAM ihtiyac\u0131ndan dolay\u0131 daha fazla fayda sa\u011flanabilir. Ancak donan\u0131m\u0131n eski olmas\u0131 durumunda so\u011futma ve elektrik maliyetleri performans art\u0131\u015f\u0131yla orant\u0131l\u0131 olmayacakt\u0131r.<\/p>\n Donan\u0131m Ar\u0131za Modu Analizi<\/h2>\n\u00c7oklu GPU kurulumlar\u0131 belirli ar\u0131za modlar\u0131na yatk\u0131nd\u0131r. Bunu anlamak saatler s\u00fcren sorun giderme s\u00fcrecini kurtar\u0131r. \u00c7oklu kartl\u0131 yap\u0131lar tek bir karta g\u00f6re daha fazla nokta at\u0131\u015f\u0131 ar\u0131za \u00fcretir; so\u011futma ve g\u00fc\u00e7 kayna\u011f\u0131 ilk olarak g\u00f6\u00e7 eder.<\/p>\n \n\n\n| Ar\u0131za Modu<\/th>\n | Belirtiler<\/th>\n | Neden<\/th>\n | Y\u00f6netim<\/th>\n<\/tr>\n<\/thead>\n | \n\nTermal Darbelene (Thermal Throttling)<\/strong><\/td>\n| Yava\u015f token \u00fcretimi, y\u00fcksek s\u0131cakl\u0131k<\/td>\n | K\u00f6t\u00fc hava ak\u0131\u015f\u0131, y\u00fcksek ortam s\u0131cakl\u0131\u011f\u0131<\/td>\n | So\u011futmay\u0131 g\u00fc\u00e7lendirin, voltaj\u0131 d\u00fc\u015f\u00fcr\u00fcn (undervolt). Alternatif olarak fan h\u0131z\u0131n\u0131 sabitlemek yerine termal duruma g\u00f6re dinamik kontrol kullan\u0131n.<\/td>\n<\/tr>\n | \nDriver \u00c7\u00f6kmesi<\/strong><\/td>\n| OOM hatas\u0131, sistem tak\u0131lma<\/td>\n | S\u00fcr\u00fcc\u00fc karars\u0131zl\u0131\u011f\u0131, bellek s\u0131z\u0131nt\u0131s\u0131<\/td>\n | S\u00fcr\u00fcc\u00fcleri g\u00fcncelleyin, servisi yeniden ba\u015flat\u0131n. Bilinmeyen bir g\u00fcncellemeden \u00f6nce test ortam\u0131nda do\u011frulay\u0131n; en az\u0131ndan s\u00fcr\u00fcm ge\u00e7mi\u015fini bilin.<\/td>\n<\/tr>\n | \nPCIe T\u0131kanma (Stall)<\/strong><\/td>\n| D\u00fc\u015f\u00fck aktarma h\u0131z\u0131, y\u00fcksek gecikme<\/td>\n | Yol (bus) t\u0131kan\u0131kl\u0131\u011f\u0131, k\u00f6t\u00fc kablo<\/td>\n | PCIe yol h\u0131z\u0131n\u0131 kontrol edin, Gen4 kullan\u0131n. Kablo kalitesinden ve slot yerle\u015fiminden emin olun; \u00e7o\u011fu zaman kablo veya slot de\u011fi\u015ftirme en h\u0131zl\u0131 \u00e7\u00f6z\u00fcmd\u00fcr.<\/td>\n<\/tr>\n | \nG\u00fc\u00e7 Darbesi<\/strong><\/td>\n| Sistem yeniden ba\u015flatma, kapanma<\/td>\n | G\u00fc\u00e7 kayna\u011f\u0131 a\u015f\u0131r\u0131 y\u00fckleme<\/td>\n | G\u00fc\u00e7 kayna\u011f\u0131n\u0131 y\u00fckseltin, g\u00fc\u00e7 \u00e7ekimini kontrol edin. PSU’yu sadece y\u00fckleme de\u011fil, kal\u0131nt\u0131 y\u00fckleme (headroom) ile se\u00e7in; \u00f6mr\u00fcn\u00fc uzatmak i\u00e7in %70’in alt\u0131na d\u00fc\u015f\u00fcrmek daha ak\u0131ll\u0131ca.<\/td>\n<\/tr>\n | \nKatman Ayr\u0131lma Hatas\u0131<\/strong><\/td>\n| Model y\u00fcklenemedi<\/td>\n | VRAM e\u015fle\u015fmesi yok<\/td>\n | --ngl<\/code> parametresini ayarlay\u0131n veya quantization kullan\u0131n. Katmanlar\u0131 VRAM’e s\u0131k\u0131\u015ft\u0131rmak yerine modelin boyutunu ve quantization seviyesini \u00f6nceden hesaplay\u0131n; aksi halde sistem donar.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\nSorun Giderme: \u00c7oklu GPU Ba\u015far\u0131s\u0131z Oldu\u011funda<\/h2>\nEn iyi planlamaya ra\u011fmen \u015feyler ters gider. \u0130\u015fte s\u0131k ya\u015fanan sorunlar ve \u00e7\u00f6z\u00fcmleri.<\/p>\n Sorun 1: GPU 0’da “CUDA out of memory”<\/h3>\nBu genellikle modelin do\u011fru b\u00f6l\u00fcnmedi\u011fini g\u00f6sterir. GPU 0, \u00e7ok fazla katman\u0131 y\u00fcklemeye \u00e7al\u0131\u015f\u0131yor.
\n* D\u00fczeltme:<\/strong> \u0130kinci GPU i\u00e7in | | | | | | | | | | | | | | | | | | | | | | | | | | |