Top suggestions for modal or multimodalExplore more searches like modal or multimodalPeople interested in modal or multimodal also searched for |
Related Products
- 1000×618
+leads+to+the+phenomenon+of+optimization+imbalance%252C+thus+struggling+to+achieve+satisfactory+performance.+Recently%252C+some+representative+methods+have+been+proposed+to+boost+the+performance%252C+mainly+focusing+on+adaptive+adjusting+the+optimization+of+each+modality+to+rebalance+the+learning+speed+of+dominant+and+non-dominant+modalities.+To+better+facilitate+the+interaction+of+model+information+in+multimodal+learning%252C+in+this+paper%252C+we+propose+a+novel+multimodal+learning+method%252C+called+modal-aware+interactive+enhancement+(MIE).+Specifically%252C+we+first+utilize+an+optimization+strategy+based+on+sharpness+aware+minimization+(SAM)+to+smooth+the+learning+objective+during+the+forward+phase.+Then%252C+with+the+help+of+the+geometry+property+of+SAM%252C+we+propose+a+gradient+modification+strategy+to+impose+the+influence+between+different+modalities+during+the+backward+phase.+Therefore%252C+we+can+improve+the+generalization+ability+and+alleviate+the+modality+forgetting+phenomenon+simultaneously+for+multimodal+learning.+Extensive+experiments+on+widely+used+datasets+demonstrate+that+our+proposed+method+can+outperform+various+state-of-the-art+baselines+to+achieve+the+best+performance.&cdnurl=https%3a%2f%2fth.bing.com%2fth%2fid%2fR.1826a36a654cb1fcb7f7945e9c51c6c5%3frik%3dffYuu0T2%252b027jA%26pid%3dImgRaw%26r%3d0&exph=630&expw=1200&q=Modal+or+Multimodal&FORM=IRPRST&ck=653D0DFA5F7BD19FC84A68DBC07EF897&selectedIndex=67&itb=0)
- 2048×1152
- 1000×528
Some results have been hidden because they may be inaccessible to you.Show inaccessible results
Feedback
