Zaɓi Harshe

Haɗin Token: Gina Hanya Tsakanin Tsinke Token da Haɗa Token don Ingantattun Vision Transformers

Bincike akan Haɗin Token (ToFu), sabuwar hanya da ta haɗa tsinke token da haɗa token don rage farashin lissafi na Vision Transformer yayin da ake kiyaye daidaito.
tokens-market.com | PDF Size: 2.6 MB
Kima: 4.5/5
★ ★ ★ ★ ★
Kimarku
Kun riga kun ƙididdige wannan takarda
Murfin Takardar PDF - Haɗin Token: Gina Hanya Tsakanin Tsinke Token da Haɗa Token don Ingantattun Vision Transformers
Duba Takardar PDF Zazzage PDF Fassara PDF
Gida » Takaddun » Haɗin Token: Gina Hanya Tsakanin Tsinke Token da Haɗa Token don Ingantattun Vision Transformers

1. Gabatarwa & Bayyani

Vision Transformers (ViTs) sun kawo sauyi a fannin hangen nesa na kwamfuta amma suna fama da tsadar lissafi saboda rikitarwar murabba'i na kulawar kai dangane da adadin token na shigarwa. Wannan takarda, Haɗin Token: Gina Hanya Tsakanin Tsinke Token da Haɗa Token, ta gabatar da Haɗin Token (ToFu), hanya ce ta gaurayawa da ke zaɓar tsakanin tsinke token da haɗa token bisa halayen samfurin don inganta daidaiton inganci da aiki.

Babban fahimta shi ne cewa ba tsinke (watsi da token) ko haɗawa (matsakaicin token) ba ne mafi kyau a ko'ina. Takardar ta ba da shawarar hanya mai ka'ida don zaɓar aikin da ya dace a kowane mataki, tare da sabuwar fasahar haɗawa da ake kira MLERP (Multi-token Linear intERPolation) don magance matsalolin canjin rarraba a cikin matsakaicin haɗawa na yau da kullun.

2. Tsarin Aiki na Asali: Haɗin Token (ToFu)

An gina ToFu akan binciken martanin samfurin ga shigarwar da aka haɗa, don tantance dacewarsa don haɗawa ko tsinke.

2.1. Matsalar Tsinke da Haɗawa

Marubutan sun gano ma'auni mai mahimmanci: layin layi na samfurin. Idan matakin samfurin yana amsa kusan layi-layi ga shigarwar da aka haɗa (misali, $f(\alpha x_1 + (1-\alpha)x_2) \approx \alpha f(x_1) + (1-\alpha)f(x_2)$), haɗa token makamantan ta hanyar matsakaici yana da tasiri kuma yana adana bayanai. Duk da haka, a cikin matakan farko/mafi zurfi marasa layi (kamar yadda aka gani a cikin Hoton su na 1), haɗawar layi a sararin shigarwa yana haifar da fitarwa marasa layi sosai, yana sa matsakaicin haɗawa ya zama matsala kuma yana iya haifar da canjin rarraba. A irin waɗannan lokuta, tsinke token marasa mahimmanci madadin ne mai aminci, ko da yake yana da asara.

2.2. Tsarin ToFu

ToFu yana aiki a kowane toshe na transformer:

  1. Ƙimar Muhimmancin Token: Yana ba da maki mahimmanci ga kowane token (misali, bisa ga tsarin kulawa ko gradient).
  2. Kima na Layi: Yana kimanta kusan layin matakin, galibi ana samunsa ta hanyar gwaji ko ta hanyar bincike mai sauƙi.
  3. Aiki Mai Daidaitawa: Don maƙasudin ragin rabon token:
    • A cikin yankuna masu babban layi: Haɗa token mafi ƙarancin mahimmanci tare da maƙwabtansu mafi kama, masu mahimmanci.
    • A cikin yankuna masu ƙananan layi: Tsinke token mafi ƙarancin mahimmanci kai tsaye.

Wannan yana haifar da bututun matsawa mai kuzari, mai sanin yanayi.

2.3. MLERP: Haɗawa Mai Kiyaye Tsarin Norm

Don inganta akan matsakaicin sauƙi, marubutan sun ba da shawarar MLERP, daidaitawar Haɗawar Layi na Spherical (SLERP) don haɗa $K$ token. Don token $t_1, t_2, ..., t_K$ tare da ka'idoji $n_i = ||t_i||$, MLERP da farko yana haɗa hanyoyi a kan sararin samaniya na raka'a sannan ya auna ta matsakaicin ma'auni na asali:

$t_{\text{merged}} = \left( \frac{\sum_{i=1}^K w_i n_i}{\| \sum_{i=1}^K w_i \frac{t_i}{n_i} \|} \right) \left( \sum_{i=1}^K w_i \frac{t_i}{n_i} \right)$

inda $w_i$ ma'auni ne na tushen mahimmanci. Wannan yana adana rarraba tsarin sifa na siffofi, yana rage canjin rarraba da aka haifar da matsakaicin sauƙi kuma yana haifar da ingantaccen aiki, musamman a cikin tsarin marasa layi.

3. Cikakkun Bayanai na Fasaha & Tsarin Lissafi

Takardar ta tsara matsalar raguwar token. Bari wani mataki ya sami $N$ token shigarwa $T = \{t_1, ..., t_N\}$. Manufar ita ce samar da raguwar saiti $T'$ tare da $M < N$ token.

Ma'auni Mai Muhimmanci:

  • Makin Muhimmanci: $I(t_i) = ||\text{Attn}(t_i)||_1$ ko ma'auni na tushen gradient.
  • Ma'aunin Kama: Yawanci kamancen cosine $S(t_i, t_j) = \frac{t_i \cdot t_j}{||t_i|| \, ||t_j||}$.
  • Ma'aunin Layi ($\mathcal{L}$): Ana auna shi ta hanyar karkatar da fitarwar mataki daga haɗawar layi na shigarwa. Ƙaramin $\mathcal{L}$ yana fifita haɗawa; babban $\mathcal{L}$ yana fifita tsinke.

Ana iya amfani da algorithm ɗin ToFu ga samfuran da aka riga aka horar da su ba tare da daidaitawa (zero-shot) ba ko kuma a inganta su tare da horo mai haske.

4. Sakamakon Gwaji & Aiki

Marubutan sun kimanta ToFu akan rarraba hoto (ImageNet tare da ViT-B/16, DeiT) da ayyukan samarwa hoto (samfuran diffusion na latent).

Mahimman Abubuwan Aiki

  • Rarrabawa: ToFu ya sami mafi kyawun daidaito da FLOPs fiye da tsinke kaɗai (misali, DynamicViT) ko hanyoyin haɗawa (ToMe). Misali, a ragin FLOPs na 40%, ToFu ya rasa <0.5% inganci na saman-1 akan ImageNet, yana fi ToMe da kusan 0.3%.
  • Samarwa Hoto: A cikin Stable Diffusion, ToFu yana kiyaye ingantaccen amincin gani (wanda aka auna ta FID) a farashin lissafi da aka rage idan aka kwatanta da ToMe, musamman lokacin rage adadin token mai yawa. Haɗawar MLERP tana nuna fa'ida mafi bayyananne a cikin ayyukan samarwa inda rarraba fitarwa ke da mahimmanci.
  • Ablation: An nuna dabarun daidaitawa (zaɓar haɗawa/tsinke) ya fi amfani da ko dai aiki na musamman a duk matakan. MLERP ya ci gaba da fi matsakaicin haɗawa.

Bayanin Chati (Bisa Hoton Takarda na 1): Hoto yana kwatanta rashin layi na matakan ViT. An haɗa maki siffa biyu na shigarwa (x1, x2) ta layi (layi mai launi). An zana fitarwa (f1-f4) daga matakan MLP daban-daban guda huɗu a cikin ViT. Fitarwar MLP na farko da na ƙarshe (f1, f4) sun nuna babban bambanci daga layi madaidaici, yana nuna ƙarfin rashin layi. Matsakaicin shigarwar biyu (tauraro mai shuɗi) yana zana zuwa wurin fitarwa mai nisa daga matsakaicin fitarwa, yana nuna a zahiri dalilin da yasa matsakaicin haɗawa zai iya gaza a cikin matakan marasa layi.

5. Tsarin Bincike & Misalin Lamari

Lamari: Aiwatar da ToFu zuwa ViT da aka riga aka horar don Aikawa zuwa Geɓe

Yanayi: Mai haɓakawa yana buƙatar gudanar da samfurin ViT-B akan na'urar hannu don rarraba hoto na ainihin lokaci. Cikakken samfurin yana da sauri sosai.

Aiwatar da Tsarin:

  1. Bayanan Martaba: Gudanar da ƙaramin bayanan daidaitawa ta hanyar samfurin. Ga kowane toshe na transformer, lissafta ma'aunin layi $\mathcal{L}$ ta hanyar samfurin nau'i-nau'i na token da duba kuskuren haɗawar fitarwa.
  2. Taswirar Dabarun: Ƙirƙiri bayanan martaba: Tubalan 1-3 (ƙananan layi) → fifita tsinke. Tubalan 4-8 (babban layi) → fifita haɗawar MLERP. Tubalan ƙarshe (ƙananan layi) → fifita tsinke.
  3. Saɓani: Saita kasafin kuɗin raguwar token na duniya (misali, 35%). Aiwatar da tsinke a cikin tubalan ƙananan layi da haɗawar MLERP a cikin tubalan babban layi, girmama kasafin kuɗin kowane toshe da aka samo daga makin mahimmanci.
  4. Ƙima: Aikawa da samfurin da aka matsawa. Hanyar daidaitawa tana tabbatar da raguwar inganci mafi ƙanƙanta idan aka kwatanta da hanyar da ta dace da kowa, saboda tana guje wa haɗawa mai ƙarfi a cikin matakan marasa layi masu mahimmanci.

Wannan misalin yana nuna amfanin ToFu na zahiri azaman tsarin matsawa mai tsari, ba kawai algorithm guda ɗaya ba.

6. Ayyukan Gaba & Hanyoyin Bincike

  • Transformers na Multimodal: Tsawaita ToFu zuwa bidiyo, sauti, ko transformers na multimodal (misali, CLIP, Flamingo) inda motsin token ya fi rikitarwa.
  • Haɗin Kai na Kayan Aiki: Inganta algorithm ɗin yanke shawara na ToFu (tsinke/haɗawa) da aiwatar da MLERP don takamaiman masu hanzari na AI (NPUs, GPUs) don haɓaka haɓakar gudu na ainihi.
  • Haɗin Kai tare da Sauran Fasahohi: Haɗa ToFu tare da ƙididdigewa, distillation ilimi, ko ingantattun hanyoyin kulawa (kamar Linformer) don ribar ingancin da aka haɗa.
  • Binciken Hyperparameter ta atomatik: Amfani da binciken gine-ginen jijiyoyi (NAS) ko koyo mai ƙarfafawa don tantance mafi kyawun rabon tsinke/haɗawa na kowane mataki da kofa na layi ta atomatik.
  • Bayan Hangen Nesa: Bincika ingancinsa a cikin Manyan Samfuran Harshe (LLMs) don matsawa jeri, ko da yake ma'anar token ta bambanta sosai.

7. Nassoshi

  1. Dosovitskiy, A., et al. "An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale." ICLR 2021.
  2. Bolya, D., et al. "Token Merging: Your ViT But Faster." ICLR 2023 (ToMe).
  3. Wang, Y., et al. "DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification." NeurIPS 2021.
  4. Rombach, R., et al. "High-Resolution Image Synthesis with Latent Diffusion Models." CVPR 2022.
  5. Zhu, J.Y., et al. "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks." ICCV 2017 (CycleGAN).
  6. Vaswani, A., et al. "Attention Is All You Need." NeurIPS 2017.

8. Binciken Kwararru & Fahimta Mai Muhimmanci

Fahimta ta Asali: ToFu ba kayan aikin matsawa kawai ba ne; sanin cewa matakan transformer sun bambanta ne a hukumance. Bi da duk matakan da irin wannan farkon matsawa yana da butulci. Hikimar takardar tana cikin hanyar bincikenta—auna layin mataki don ba da "tiyata" daidai (tsinke ko haɗawa). Wannan yana tunawa da yadda masu tarawa na zamani ke bayyana martabar lamba don amfani da ingantattun, matakin ƙwarewa da sau da yawa ba ya nan a cikin binciken ingancin ML.

Kwararar Ma'ana: Hujjar tana da ban sha'awa: 1) Nuna matsakaicin haɗawa ya gaza a cikin matakan marasa layi (Hoto 1). 2) Ba da ma'auni don gano wannan yanayin gazawar (layi). 3) Amfani da ma'aunin don tura token. 4) Gyara aikin da ya gaza (matsakaicin haɗawa) tare da MLERP. Kwararar daga gano matsala zuwa mafita mai ɗimbin sassa yana da tsabta kuma yana da ma'ana.

Ƙarfi & Kurakurai:
Ƙarfi: Hanyar haɗakarwa tana da inganci bisa ka'ida kuma an tabbatar da ita ta hanyar gwaji a cikin ayyuka. MLERP gyara ne mai sauƙi amma mai wayo ga matsala ta gaske (rushewar tsari). Aiwatar da zero-shot fa'ida ce mai mahimmanci na zahiri don tura samfuran da ake da su.
Kurakurai: Takardar ta ɗan rage ƙimar "ƙimar layi." Shin bayanan martaba ne da aka riga aka lissafta (tsaye) ko aka lissafta akan-lokaci (ƙarin farashi na motsi)? Fa'idodin MLERP, ko da yake a bayyane, suna bayyana matsakaici a cikin rarrabuwa; daidai darajarsa ta bayyana mafi bayyananne a cikin ayyukan samarwa, yana daidaitawa da binciken daga wallafe-wallafen samfurin diffusion inda rarraba fitarwa ke da mahimmanci. Kwatancen, ko da yake gaskiya ne, zai iya zama mafi ƙarfi akan hanyoyin ƙididdigewa na zamani bayan horo waɗanda ke ba da fa'idodi masu kusurwa.

Fahimta Mai Aiki: Ga masu aiki: Nan da nan karɓi ToFu/MLERP azaman hanyar farko ta raguwar token don ViTs, musamman don ayyukan samarwa. Ya maye gurbin ToMe azaman dabarun haɗawa na asali. Ga masu bincike: Tsarin "matsawa mai sanin mataki" shine abin da za a ɗauka. Aikin gaba ya kamata ya mayar da hankali kan sarrafa gano yankunan samfurin da suka dace da matsawa da kuma masu mahimmanci na matsawa, watakila yana ɗaukar wahayi daga aikin kan tsinke cibiyar sadarwa a cikin CNNs ko binciken rugujewar yanayi a cikin GANs kamar CycleGAN. Gaba gaba shine gina samfuran waɗanda suke asali masu inganci ta ƙira, ta amfani da fahimta daga irin waɗannan binciken bincike don sanar da binciken gine-gine, wucewa sama da matsawa bayan-bayan.